Methodology
This analysis uses the SOC 2 Quality Guild's Reliability Rubric v1.0 (February 2026), licensed under CC BY-SA 4.0. The rubric defines 11 signals across 3 pillars for evaluating SOC 2 report quality. We implemented each signal as a scoring function — some purely deterministic (regex and heuristics), others AI-assisted using Claude for semantic analysis.
Scoring Formula
Structure (25% of overall)
Report formatting and AICPA compliance
Checks that Section 1 contains all required AICPA paragraphs: Scope, Opinion, Service Organization's Responsibilities, Service Auditor's Responsibilities, Inherent Limitations, and Restricted Use.
Verifies Management's Assertion completeness: description accuracy, control design suitability, operating effectiveness (Type II), signature, and date.
Analyzes cross-section consistency: company names, placeholder text, scope boundaries, and subservice organization references across Sections 1, 3, and 4.
Substance (50% of overall)
Depth of audit testing and specificity
Assesses whether the System Description names specific technologies, tools, teams, and infrastructure — or uses generic boilerplate that could apply to any company.
Spot-checks whether controls logically address their assigned Trust Services Criteria. Flags illogical mappings like access controls mapped to communication criteria.
Applies the 5-question test to control descriptions: What is done? How? Who does it? When? Where? Also checks for contradictions between controls.
The strongest signal. Measures boilerplate test language rate, exception rate (100% 'No exceptions noted' is a red flag), sample sizes, meaningful test types, and pagination fingerprints.
Source (25% of overall)
Audit firm credibility and independence
Identifies the audit firm from license numbers and known firm names. Flags known problematic entities like PAC-FIRM-LIC-47383 (Delve/Accorp).
Estimates the ratio of reports issued to CPAs at the firm. A single license issuing 487+ reports is an extreme outlier.
Evaluates audit firm leadership credibility. Flags firms with shell company structures or untraceable registration.
Scans for GRC platform mentions (Vanta, Drata, Secureframe, Delve, etc.). The platform through which the audit was conducted can indicate quality.
Attribution
The scoring rubric is based on the SOC 2 Quality Guild Reliability Rubric v1.0 (February 2026), used under the CC BY-SA 4.0 license. This analysis and tool were built by Trenta. The rubric authors are not affiliated with this project and do not endorse these specific findings.