Over the last few months, there have been multiple high profile articles by Gartner, Compliance Week, and now CFO Magazine, criticizing companies for marketing the completion of a SAS 70 audit as a “certification” or proof of security, continuity or privacy compliance.

I wrote an article on this back in June where I discussed SAS 70 my experience marketing the security controls of my previous employer, a global managed security services provider.  The SAS 70 audit was something we did for our financial services clients and we worked collaboratively with them on the audits and reports year over year.   I never considered it a certification.  At the time, it was merely a CPA firm issuing a report confirming that we were doing what we said we were doing.

But not all marketing people think the same.  Over the years, the term SAS 70 “certified” has become commonplace.  Self awarded “seals” of completion have shown up on websites.  So have press releases claiming that the SAS 70 audit confirms that the company can do everything short of walking on water.  Because of this, the SAS 70 standard now regularly takes a beating for not being a universal solution, despite the fact that it never claimed that it worked for any other purpose than those intended.

Now we get articles titles like “The Truth About SAS 70” and “SAS 70 Reports, in Harsh Spotlight Again”.  These catchy titles imply nefariousness and fault in the SAS 70 audit.  But in reading the actual articles you find no claims of fault whatsoever.  Rather, you find significant criticism of those companies that make overreaching claims about the standard itself.  In essence, the jig is up.  Companies must do more to ensure that their marketers understand how to market the completion of a SAS 70 audit.

So speaking the product marketer’s perspective, I recommend the following steps to avoid overreaching and potential customer complaints:

1. Discontinue the use of the term “certification” when referring to the SAS 70 audit.
2. Insure that all marketing materials are free of any claims that a completed SAS 70 audit is evidence of anything beyond the scope of the audit.
3. Avoid all claims that the audit report is proof of security, continuity or privacy compliance.Organizations that display a SAS 70 “certification seal” on their website or in marketing materials should consider discontinuing this practice.  It has become a common practice but is not advisable.  Furthermore, no official “certification seal” exists, so the use of such a seal merely creates the potential for criticism from competitors and customers.
4. Avoid third party “registry” services that seek to list summary information about your organization’s SAS 70 audit in exchange for a fee.  These sites are analogous to claims of “certification” and the use of “certification seals”.  Similar to “certification seals”, the information conveyed is so basic that they offer no significant value.  In addition, many customers may be ill equipped to know the difference between claims of “certification” and claims of “registration”.
5. Share this information with marketing personnel.  Stress the importance of avoiding claims about the SAS 70 audit that are beyond the scope of the audit.

I regularly assist our clients in drafting marketing statements and press releases about their audits.  I recommend that you consult your auditor as well when it comes to these matters as well.

[Created - 25 May 2010]
[Last Updated - 19 August 2010]

In this blog post, I intend to maintain a list of issues I note with SSAE 16.  I post these issues hoping that they will not linger like so many of the issues in the SAS 70 audit standard.

I acknowledge that some of these issues are topics that should be handled in the audit guide, and hopefully they will be.  However, the audit guide is not expected until 2011 and the standard allows for early adoption.  In light of this, many of the following items are needed now.

So in no particular order:

• NEW – Inconsistencies in the Language and Punctuation Between the Common Sections of the Management Assertion Examples
• No Sample Reports
• No Subservice Organization Management Assertions Examples
• No Service Organization Management Representation Letter Examples
• No Subservice Organization Management Representation Letter Examples
• Paragraph Subtitles in Opinion Letter Examples
• Type 1 Carve Out and Inclusive Methods Missing
• Inconsistencies in Opinion Letter Language
• Use of the Term “Their” in Inclusive Method Reporting
• Typos

Want us to add an issue?  Think we’re wrong about a topic?  Let us know and we will be happy to update the topics accordingly.

No Sample Reports
(Originally Posted 5.25.2010)

As a service audit firm, I am confident in our ability to issue an SSAE 16 report today.  However, it is asking a lot of service audit firms to perform audits for “early adopters” without providing example report and greatly increases the risk of form and content errors in reports.

Suggested Solutions to the ASB: Provide comprehensive example Type 1 and Type 2 SSAE 16 reports.

No Subservice Organization Management Assertion Examples
(Originally Posted 5.25.2010)

This is a significant hole in the current guidance.  An experienced service auditor could draft these, but the lack of guidance is a gaping hole in the guidance.

Suggested Solution for the ASB: Provide subservice organization management assertion examples as soon as possible.

No Service Organization Management Representation Letter Examples
(Originally Posted 5.25.2010)

Self-explanatory.

Suggested Solution for the ASB: Provide management representation letter examples for Type 1 and Type 2 examinations.

No Subservice Organization Management Representation Letter Examples
(Originally Posted 5.25.2010)

A long-time issue in the SAS 70 audit guidance rears its ugly head in SSAE 16, but now, it’s an even bigger issue.  Given that subservice organizations are essentially required to provide management representation letters, the lack of example subservice organization management representation letters leaves all service auditors to fend for themselves.

Suggested Solution for the ASB: Provide subservice organization management representation letter examples for Type 1 and Type 2 examinations.

Paragraph Subtitles in the Opinion Letter Examples
(Originally Posted 5.25.2010)

A historical weakness of the SAS 70 audit standard and related audit guide is a lack of clarity when referring to specific opinion letter paragraphs.  Paragraphs had no specific labels/subtitles.  In the case of Type 2 audits, there are actually two opinion paragraphs; however, the guidance typically only referred to “the” opinion paragraph.  In short, this issue increased the risk that of service auditor confusion and error.

My assumption is that the insertion of subtitles in the SSAE 16 sample opinion letter examples is an attempt to improve the guidance.  The subtitles allow for easier references to specific paragraphs within the opinion letter, and from that perspective, they are an improvement.  However, they are not described as required components of a service auditor’s opinion letter, and I assume, perhaps incorrectly, that the ASB did not intend for them to be included in actual reports.

We have discussed this issue with the AICPA.  We were informed that it is auditor preference given that the subtitles are in the example language, but are not required component of the opinion letter.  Given that other similar auditor’s reports do not use subtitles, my hope is that subtitles will not become normal and customary for SSAE 16 opinion letters.

Suggested Solution for the ASB: Clarify whether subtitles should be included in an actual service auditor’s opinion letter so that, at the very least, there is consistency.

Type 1 “Carve Out” and “Inclusive” Reporting Methods Missing
(Originally Posted 5.25.2010)

The nuance differences are minor but do exist.  Certain assumptions have to be made unnecessarily when drafting a Type 1 opinion letter using either method in the absence of Type 1 example language.

Suggested Solution for the ASB: Add Type 1 examples and remove ambiguity.

Inconsistencies and Typos in Opinion Letter Language
(Originally Posted 5.25.2010)

There appears to be “inconsistencies” between base language of example Type 1 templates and between Type 1 and Type 2 templates.  As best I can tell, this is simply a lack of attention to detail.  One example:

Suggested Solution for the ASB: Perform quality assurance reviews on all sample opinion letter language and eliminate any inconsistencies in “base” language.

Use of the Term “Their” in Inclusive Method Reporting
(Originally Posted 5.25.2010)

For reasons that are beyond me, the “inclusive” reporting method guidance changes the term “its” to “their” when referring to the system within the scope paragraph, as shown below:

“their” system?

That sound you hear is legal bloggers posting article after article about the best reason subservice organizations should not agree to be included in the scope of an SSAE 16 examination.

Within the same scoping paragraph, the following statement is required when using the inclusive reporting method:

Contradictory?  I think so.

Suggested Solution for the ASB: Do not use the term “their” or imply that subservice organization’s have any responsibility for the service organizations system description or underlying controls.  Additionally, use wording similar to the following:

Typos
(Originally Posted 5.25.2010)

The sample paragraph on page 67 has deletions to the standard language that are not indicated. Given that deletions are indicated in other examples, they probably should be consistently marked here.

Suggested Solution for the ASB:  Consistently mark deviations from the standard language in all examples.

Yesterday, the PCI Standards Council posted a document highlighting some of the upcoming changes to the PCI DSS.  That document can be found here.

The document is a “teaser” to what is expected to be released in the October timeframe.  Two of the roughly four pages speak to the standards development process and feedback cycle.  In addition, the Council itself notes that the updates” are relatively straightforward and do not introduce significant changes.”

In terms of substantive previews, the following are worth noting:

• Additional guidance is expected for cardholder environment definition and data flow mapping (i.e. scoping)
• Guidance will be provided on virtualization (emanating from the Special Interest Group on this very topic) including how virtual components are defined “in-scope” as well as guidance on specific controls that were traditionally focused on physical hosts.
• Updates to requirement 3.6 for key management to provide additional flexibility for new technologies and evolving ways to manage encryption keys.
• Updates to vulnerability management and application security to reflect the evolving nature of those topics.

As with the last update, the devil will be in the details.  The new standards are expected to be published in October and will become effective for all assessments performed starting in January 2011.  At this time, there is no reason to believe that these changes will significantly impact the scope and cost of onsite assessments.

As always, if you have any questions, feel free to visit our PCI Resource Center or contact one of our QSAs .

Over the last few weeks, many service organizations have started to receive requests from customers wanting to know the “controls standard” used as the basis for their SAS 70 audits.  These requests initially seemed like a strange coincidence, but eventually it became clear that something was causing the inquiries.  A quick Google search was all it took to identify the source, a Gartner report issued in late June, entitled “SAS 70 Is Not Proof of Security, Continuity or Privacy Compliance,” written by analysts Jay Heiser and French Caldwell.  This one report has spawned more derivative articles on the topic of SAS 70 than any other in the standard’s history.  Unfortunately, the report includes a statement that is regularly misinterpreted by readers.  Now that the report has gone “viral,” service organizations must prepare to respond to questions about the basis of their SAS 70 report.

I first read the Gartner report when it was released.  Its contents were so innocuous that I hardly gave it a second thought.  Articles proclaiming that a SAS 70 audit is not a security assessment are a dime a dozen.  But this report is different.  Despite its title, the document does not deride the SAS 70 standard.  Instead, in less than 10 pages, the report provides the basic information regarding the SAS 70 audit and its proper use by service organizations, their clients, and CPA firms.

A major finding of the report is that the SAS 70 audit is not a security or compliance audit and does not result in a certification.  For anyone familiar with the SAS 70 standard, this finding is not newsworthy.  But toward the end of the report, the authors suggest alternative standards that may be “adopted” when proof of security, continuity or privacy compliance is required.  They describe ISO 27001/2, BITS Shared Assessments, SysTrust, WebTrust, and AT Section 101, all of which contain a prescriptive collection of security and compliance controls, with the exception of AT Section 101.

Unfortunately, the use of the word “adopted” is having unintended consequences.  Some readers are incorrectly interpreting this to mean “adopted as the basis for a SAS 70 audit” even though the report makes no such recommendation.  In fact, it uses the term “alternative assessment standards” when referring to the other standards.  While the report does not explicitly state this, it seems clear that the authors intended to identify the alternative standards as being complementary to the SAS 70 standard rather than as a suggested basis for a SAS 70 audit.  This is further evidenced when the report uses phrases such as “instead of the SAS 70” when discussing the use of the alternative standards.

Of course, Gartner reports always attract significant attention and these issues are being compounded by the hundreds of subsequent articles on the report, many of which further confuse the situation.  For example, Compliance Week has already published two articles on the Gartner report entitled “Study Faults SAS 70 Audits for False Sense of Security” and “SAS 70 Reports, in Harsh Spotlight Again.”

Forgive me, but this is misleading.  The study does not fault SAS 70 audits as the cause of any false sense of security.  Instead, it blames misuse and ignorance for causing the false sense of security.  The report essentially concludes that SAS 70 audits work for the purposes intended, and not so well when used otherwise.  I suppose someone could interpret that as criticism, but for most of us, it merely states the obvious.

The combination of these issues means that service organizations are now being forced to respond to illogical questions regarding the “basis” of their SAS 70 audit.  One organization responding to client inquiries generated by the Gartner report is Peak 10, a managed services company operating multiple data centers located throughout the United States.

According to David Kidd, Peak 10’s director of quality assurance and compliance, “Our response has been that our SAS 70 audit is designed to report on the controls we have implemented to meet the needs of our clients, versus the controls that are suggested by any number of well known risk management standards which may not appropriately specify controls suitable to our business or our customer’s needs.”  While Peak 10 does not use any alternative standards as the basis for its SAS 70 audit, Kidd stated “Our management incorporates ‘best practice’ guidance from outside standards, such as ISO 27002, COSO, PCI DSS, and others when designing and implementing controls, but we customize our controls based on what makes the most sense for us and our customers.”

Service organizations responding to inquiries on this topic should consider the following points when preparing a response to customers:

1. Statement on Auditing Standard (SAS) No. 70 is the only required basis for a SAS 70 audit.  It has a very specific purpose, and that purpose cannot be achieved through any alternative standard.
2. In the absence of technical citations to the contrary, there is absolutely no authoritative professional guidance that suggests that a SAS 70 audit should be based on any alternative standard.
3. The well known standards, such as ISO 27002 and COBIT, include codes of “best practice” controls that are normally specific to IT and compliance topics.  They are also largely common sense, which means that organizations instinctively implement some subset of the controls, and usually without any consideration of the standards.  For example, all organizations recognize the need to secure access points to their facilities and do so without ever studying ISO 27002 section 9.1.2 – “Physical entry controls”.  Many controls included in these alternative standards may appear in some form in a SAS 70 report, but the overlap is often just a coincidence.
4. Alternative standards each have a specific purpose.  It is possible that an organization could need both a SAS 70 audit and an assessment performed in accordance with an alternative standard depending on the reporting objectives of the service organizations and their customers’ needs.  However, it is incorrect to assert, for example, that ISO 27002 compliance can, or should be, assessed in the form of a SAS 70 audit.

In other words, service organizations should not allow a misunderstanding of the SAS 70 standard or process to persist.  SAS 70 audits are “based” on the needs of service organizations and their customers, and are not based on alternative standards promulgated by organizations unrelated to the practice of public accounting.  The flexibility of the SAS 70 standard over prescriptive standards is its strength.  It allows the standard to be applied to a wide variety of business and IT processes far beyond the limited purview of each alternative standard.  By understanding this distinction, service organizations can use the client inquiries generated by the Gartner report as an excellent opportunity to extol the virtues of their customized SAS 70 audit scope, as well as their organization’s awareness of these issues.

Cloud computing has seen some exciting announcements over the past few weeks so here is the Pragmatic Auditor’s perspective.

This week, OpenStack was launched.  OpenStack is a new collaborative effort to an open source cloud computing platform to the market.  Spearheaded by Rackspace and NASA, OpenStack provides open source distributions for file storage and provisioning.   So with open source alternatives to VMWare  on or Microsft’s Azure, as well as Amazon’s Web Services, the expectations are that OpenStack will provide a much lower barrier of entry to new cloud and service providers.

So I’ll leave the market / revolution discussion to my MBA classmate Dave Rosenberg (Editor at Software Interrupted CNET) and focus on the audit and compliance considerations:

• With an increased number of providers not to mention open source itself, the need for transparency of controls is even greater.
• A by-product of OpenStack will be the increase of service provider to sub-service provider relationships (e.g. a SaaS company hosts at an IaaS co-lo and has their systems maintained by a managed service provider.   The most important thing for cloud providers is to be able to map out all their customers’ control and compliance requirements ensuring there are no “gaps” where on provider thinks the other is doing (and vice versa).
• Service providers need to carefully evaluate what assurance and compliance tools suit their customers best.  This involves doing a requirements and cost-benefit analysis of SAS 70 / SSAE 16 audits and assessments, PCI DSS validation, SysTrust, ISO 27001 certification, or any combination of those and more.

I saw the earnings release yesteday and Amazon is not about to go out of business.    I believe we will see more providers players and the need for new players to substantiate that they can provide a service that is as reliable and secure as AWS.

CloudAudit Update

Along these lines, another recent update is that the Cloud Audit group recently published version 1.0 of the cloudaudit specification to the IETF.  I am active member in CloudAudit and this is very exciting.  The goal of CloudAudit is not to develop a new standard, but develop a new mechanisms for sharing control information.  Version 1 draft  laid out the framework for a directory structured that includes the ability for a service provider to post data that can be queried directly or through API for technologies like GRC.

I will provide more updates in the coming weeks.  We will soon be releasing the first set of “compliance paks”  which utilize the Cloud Security Alliance Cloud Controls Matrix as the initial set of controls that are leveraged in CloudAudit API.

That’s all for now.  Time to leave the cloud and head back down to earth for more IT audits!