The Three SOC 2 Evidence Failures That Auditors Flag Every Engagement

The American Institute of Certified Public Accountants (AICPA) Trust Services Criteria, which govern SOC 2 engagements, are not a security standard. They are an evidence standard. The difference matters enormously in practice: a company can have excellent information security controls and still fail a SOC 2 Type II audit because the documentation of those controls does not meet the evidentiary bar auditors are required to apply.

SOC 2 engagements are attestation engagements under AT-C Section 205 of the AICPA attestation standards. The CPA firm issuing the report is attesting that, based on procedures performed during the audit period, the controls described in management's assertion were suitably designed and operated effectively. "Based on procedures performed" is the operative phrase: auditors can only attest to what they can test, and they can only test what exists in documented, retrievable form.

The three evidence failures described here are not theoretical. They are the categories of finding that appear most frequently in SOC 2 management letters and qualified reports, identified consistently across published audit guidance from Big 4 firms, AICPA guidance documents, and practitioner accounts. Understanding what produces them — and why they persist even in organizations with mature security programs — is the first step toward documentation that survives the audit period.

Failure 1: Access Log Timestamps That Cannot Be Corroborated

Access control is typically the first Trust Services Criterion area auditors test — it covers user provisioning, access review, privileged access management, and logical access to systems storing in-scope data. The most common evidence failure in this category is not the absence of access logs. It is the presence of logs that cannot be corroborated as contemporaneous.

An access log is contemporaneous evidence when it can be demonstrated to have been generated at the time the access event occurred, from a system whose clock and log integrity are themselves verifiable. A log export run on the first day of audit fieldwork, sorted and formatted in a spreadsheet, is not contemporaneous evidence. An auditor sampling that log for access events during January has no way to verify that the January entries were written in January rather than written recently.

The AICPA's Guide to Service Organization Controls Reporting (the "SOC 2 Guide") is explicit about what constitutes sufficient evidence for access control testing: auditors are expected to look for evidence that controls "operated consistently over the specified period," which requires evidence of the control operating at multiple points in the period, not a single snapshot at period end. Organizations that rely on end-of-period exports rather than continuous, tamper-evident logging routinely fail this test.

Why Continuous Logging Matters More Than Point-in-Time Exports

The distinction between continuous logging and point-in-time evidence is not cosmetic. It reflects the core difference between a SOC 2 Type I report (which attests to control design at a point in time) and a Type II report (which attests to operating effectiveness over a period of time). A Type I engagement can be satisfied by a well-designed control that exists on the audit date. A Type II engagement requires evidence that the control worked throughout the period.

For access logs specifically, this means that the logging infrastructure itself must be in scope for the audit. If the company is producing access logs from a cloud identity provider, the auditor will expect to see that the logging was configured and active throughout the period — typically evidenced by configuration records with change history, or by a third-party attestation (such as a SOC 2 report from the identity provider itself) covering the audit period.

Organizations that implement continuous logging after their first audit finding in this category typically discover that the infrastructure requirement is the simple part. The harder problem is the retention policy: cloud platform logs default to retention periods that are often shorter than the SOC 2 audit period, and the organization that enables logging in October but exports and purges logs monthly will still not have January through September logs available when the auditor requests them in November.

Failure 2: Policy Documents Disconnected from Actual Procedures

The second major evidence failure is less technical and more organizational: policy documents that describe controls the organization does not actually perform as described. This failure category is broader than deliberate misrepresentation — most of the time, the policy accurately described the intended control when it was written, and the actual procedure evolved without the policy being updated.

Auditors test the connection between written policies and actual practice through a combination of direct observation, walkthroughs, and evidence sampling. A walkthrough involves an auditor sitting with the control owner and walking through how the control is actually performed — the specific system, the specific steps, the specific output. If the walkthrough does not match the written procedure, the auditor notes the discrepancy and expands testing to determine whether the discrepancy affects the operating effectiveness conclusion.

Common patterns that produce policy-procedure mismatches:

• Vendor or tool changes — the policy describes a control performed in Tool A, but the organization migrated to Tool B 18 months ago and updated the procedure without updating the policy
• Ownership changes — the policy assigns responsibility for a control to a role that no longer exists or has been restructured, and the actual control is performed by whoever is available
• Frequency mismatches — the policy requires quarterly access reviews, the organization performs them semi-annually because that is what is practical, and the policy has not been updated to reflect the approved exception
• Threshold inconsistencies — the policy states that changes above a certain risk score require two approvals, but the change management system enforces a different threshold or none at all

Writing Control Narratives That Auditors Believe

Control narratives — the written descriptions of how controls operate that appear in the SOC 2 system description — are the first thing auditors compare against actual evidence. A narrative written to accurately describe the control as practiced, rather than to describe the control as aspirationally designed, is a narrative that will survive the walkthrough.

Effective control narratives have several characteristics that distinguish them from narratives written primarily for marketing purposes:

• They name the specific system or tool in which the control operates, including version or configuration information when it affects the control
• They describe the frequency of the control precisely — "monthly, within the first five business days of the month" rather than "periodically"
• They identify the specific role responsible for performing and reviewing the control
• They describe what the output of the control looks like — what is generated, where it is stored, how long it is retained
• They acknowledge approved exceptions explicitly rather than describing the ideal case as the universal case

The AICPA's Illustrative Service Organization System Description, available as an exhibit to the SOC 2 Guide, provides examples of the level of specificity auditors expect. The illustrative descriptions are more operationally precise than most organizations' actual narratives — organizations that compare their narrative language to these examples often find significant gaps.

Failure 3: Change Management Without Approval Records

Change management is the third major evidence failure category, and it is the one most often underestimated by technology organizations that have mature internal change processes. The failure is not that changes go unapproved. The failure is that the approval records do not survive in a form auditors can test.

Change management evidence typically requires: a record that the change was proposed in a trackable system, a record of who approved it and when, a record that testing was completed before deployment, and a record that the change was reviewed post-deployment. Organizations that perform all four steps but document them inconsistently — some in Jira, some in Slack threads, some in spreadsheets, some in the ticketing system, some in email — produce evidence that cannot be sampled consistently.

Auditor sampling for change management typically works by pulling a list of all changes deployed during the period, selecting a random sample, and requesting the approval record for each. If the approval record for a sampled change exists in Slack rather than in the ticketing system, the auditor cannot verify it as a contemporaneous, unedited record in the same way they can verify a ticketing system entry. If the approval record does not exist at all because the change was treated as an emergency change and documented retroactively, that is a direct finding regardless of whether the change itself was appropriate.

Emergency change processes deserve specific mention because they are the category where documentation most consistently fails. Organizations that handle emergency changes well from a security perspective often handle them poorly from a documentation perspective — the urgency of the change leads to a decision to document after the fact, and the post-facto documentation lacks the timestamps and approval chain an auditor needs to verify the control operated as designed.

Type II vs. Type I: The Evidence Burden Is Categorically Different

Organizations preparing for their first SOC 2 Type II audit after a Type I engagement often underestimate how significantly the evidence requirements differ. A Type I report requires evidence of control design — that the organization has the controls it describes, and that those controls are appropriately designed to achieve the stated criteria. The evidence for design is point-in-time: the controls exist and work as of the audit date.

A Type II report requires evidence of operating effectiveness over a period — typically 6 or 12 months for an initial engagement. This requires evidence that each control operated as designed throughout that period, which means evidence from multiple points in the period rather than a single snapshot. The practical documentation requirement is continuous: organizations need to be capturing and retaining evidence ready for audit throughout the period, not assembling it at audit time.

The AICPA's attestation standards require auditors to apply a sampling methodology appropriate to the risk of the criterion being tested. For higher-risk areas (access control, change management, encryption), sample sizes are larger and the distribution of samples across the period is more rigorous. Organizations that produce high-quality evidence for the most recent quarter but thin evidence for earlier quarters will find that their sample results are uneven in ways that produce qualified opinions.