No test evidence: what the rows show and the clause behind it
The test evidence column is empty, or reads none, for a change that shipped.
Does every production change need recorded test evidence?
COBIT BAI07.05 has changes tested independently against a plan before migration to production, with formal sign-off, and BAI03.08 records results in a test log. ISO/IEC 27001 8.29 runs security testing through the life cycle and NIST SP 800-53 SA-11 asks the developer for evidence of testing. The checker reads the test evidence column when the export carries one and lists the shipped changes where it is empty or reads none.
The question the auditor is likely to ask
walkthrough packWhat testing was done before this change went to production, and where is the record?
The clause behind it
8 clauses in 6 frameworks| Framework | The clause behind it |
|---|---|
| SOX 404 / ICFR | SOX 404 ITGC IT General Controls (ITGC) - Access, Change, Operations |
| COBIT 2019 | COBIT BAI07.05 Perform acceptance tests · COBIT BAI03.08 Execute solution testing |
| SOC 2 (Trust Services Criteria) | SOC 2 CC8.1 Change management processes are in place |
| ISO/IEC 27001:2022 | ISO 27001 8.29 Security testing in development and acceptance |
| PCI DSS v4.0 | PCI DSS 6.2.3 Custom software reviewed prior to production |
| NIST SP 800-53 Rev 5 | SP 800-53 SA-11 Developer testing and evaluation · SP 800-53 CM-4 Impact analyses |
A result shows the frameworks you tick; SOX 404 brings COBIT and AS 2201 with it. PCI DSS clauses show on the cardholder systems only.
SOX 404 ITGC IT General Controls (ITGC) - Access, Change, OperationsThe clause text
ITGC including access management, change management, computer operations, program development, backup, recovery.
Source: SOX 404 / ICFR
COBIT BAI07.05 Perform acceptance testsThe clause text
Changes are tested independently according to the defined test plan before migration to the live environment: the categorised log of errors found by the development team is reviewed to verify remediation or formal acceptance; final acceptance is evaluated against the success criteria and results are presented understandably to process owners and IT for an informed decision; acceptance is approved with formal sign-off by process owners, third parties as appropriate and IT stakeholders before promotion; testing follows the plan and is designed and conducted by a test group independent from the development team; tests and expected outcomes follow the plan's success criteria; scripted test instructions are assessed and approved by the independent group; an appropriate balance of automated and interactive user testing is used; security tests measure weaknesses and consider security incidents since the plan was written; performance tests cover a range of metrics such as end-user response times and database update performance; fallback and rollback elements of the plan are addressed; and errors are identified, logged and classified with an audit trail of results and communication per the plan.
Where it usually falls short: Acceptance signed off with significant errors still open; Testing performed by the developers who built the change
Source: COBIT 2019
COBIT BAI03.08 Execute solution testingThe clause text
During development, testing including control testing is executed continually according to the test plan and development practices in the appropriate environment, engaging business process owners and end users in the test team: solutions and components are tested per the plan with testers independent from the solution team and representative process owners and end users, and results are recorded in a test log; clearly defined test instructions are used with an appropriate balance of automated scripted tests and interactive user testing; all tests cover the integration of business processes and IT components and non-functional requirements such as security, privacy, interoperability and performance; errors are identified, logged and classified (minor, significant, mission-critical) and tests repeated until all significant errors are resolved, with an audit trail of results; and outcomes are recorded and communicated to stakeholders per the plan.
Where it usually falls short: Testing done only by the developers; Significant errors accepted to meet a date
Source: COBIT 2019
SOC 2 CC8.1 Change management processes are in placeNamed, not quoted: the criteria text is not held in full here.
Where it usually falls short: Emergency changes used routinely, with retrospective approval that is never withheld; Approval and implementation performed by the same person, so the approval is not independent
Source: SOC 2 (Trust Services Criteria)
ISO 27001 8.29 Security testing in development and acceptanceThe clause text
Define and run security testing across the development life cycle.
Where it usually falls short: Testing only after release; Inconsistent test coverage across modules
Source: ISO/IEC 27001:2022
PCI DSS 6.2.3 Custom software reviewed prior to productionBespoke and custom software is reviewed before release to production by someone other than the developer who wrote it, using manual and automated methods.
A one-line statement; the standard's own text is not quoted here.
Where it usually falls short: Self-approval of pull requests; SAST not blocking on findings
Source: PCI DSS v4.0
SP 800-53 SA-11 Developer testing and evaluationThe clause text
Requires the developer, at all stages after design, to plan and perform ongoing security and privacy control assessment, to carry out defined testing types at a defined frequency, depth and coverage, to produce evidence of the assessment plan being executed and its results, to implement a verifiable flaw remediation process, and to correct the flaws that testing identifies.
Where it usually falls short: Testing evidence claimed but never provided to the organization for review; Depth and coverage undefined, so a single scan satisfies the requirement on paper
Source: NIST SP 800-53 Rev 5
SP 800-53 CM-4 Impact analysesThe clause text
Requires changes to the system to be analysed for their potential security and privacy impact before they are implemented, so that the consequences of a change are understood while it can still be stopped or modified.
Where it usually falls short: Analysis performed after deployment as part of a review rather than beforehand; Privacy impact never considered, only availability and security
Source: NIST SP 800-53 Rev 5