Skip to main content
Security templateReviewed

Penetration testing retest checklist: evidence for every finding, from the build to sign-off

A finding-by-finding checklist for the round after the fixes: what your team hands over and holds still, how whoever performs the retest repeats each finding and checks around it, the evidence each of the four outcomes needs and who may record it, and a close-out sheet both sides sign.

It starts where a remediation plan stops, with findings marked ready for retest.

Finding-by-finding retest evidence checklist

Copy this into a shared document. The handover steps belong to your team and are done once per round, before the retest is requested.

How to use it

Keep one copy of the "Each finding" and "Outcome check" steps for every finding ID: they belong to whoever performs the retest, a provider or, ideally, someone on your team who did not write the fix. The special cases settle sibling routes, findings that stop reproducing for unrelated reasons and new issues before anyone needs them. The close-out sheet is signed by the retester and the engagement owner. Every outcome describes the named build only, and a finding that was not retested stays open and is not shown as passed.

What a penetration testing retest checks

A penetration testing retest is a second look at specific findings, on a named build, once the fixes are deployed.

For each finding it repeats the original evidence, checks that the legitimate use of the feature still works, runs the regression checks agreed when the finding was written, and records one outcome: fixed, partially fixed, still reproducible or not retested. It is narrower than the original test by design. It answers whether these fixes hold on this build. It does not answer whether the application is free of other problems.

NIST SP 800-115, the US Technical Guide to Information Security Testing and Assessment, gives three ways to verify that mitigation actions were implemented: an audit of the system, retesting the system and its components, and holding personnel accountable through documentation. It adds two points this checklist is built around.

The test team will be able to verify an implementation only if a mirror copy of the original test is performed. And as technology evolves, additional vulnerabilities may be uncovered during follow-up security tests.

The checklist serves two readers. Your team, getting ready to ask for the retest, needs to know what to hand over and what to hold still. Whoever performs the retest, a provider or someone on your own team, needs a fixed way to repeat each finding, decide its outcome and show the evidence for it.

It starts where a remediation plan stops, with findings marked ready for retest, and ends with a close-out sheet both sides sign.

Before you request the retest: what your team hands over

Whether a finding can be repeated as it was first shown depends on four things your team controls.

Get them right before the request goes in, because a gap found during the retest can end up as a finding recorded as not retested.

  • One build. Deploy every fix in the round to one build of the same application, in the environment the original test used, and write its release number, commit or deployment id at the top of the record. If the retest runs on staging and production follows later, the outcomes describe staging until someone confirms production runs the same build.
  • A fix note for each finding. What changed (code, configuration, infrastructure, or a feature removed), where, whether the change targets the cause or only the reported route, which related routes it also covers, the regression test added, and any interim measure still switched on, such as an edge or firewall rule.
  • The same accounts and records. The account labels in the evidence, such as a customer in one organisation and a record owned by another, have to exist again with the same roles. Where one was deleted after the first test, recreate it and list the old and new labels side by side.
  • The same conditions. The tester's source addresses let through, or not, as they were for the original test; payments in test mode; and a signed authorisation that covers the retest dates, because a retest sends requests to the same targets.

A fix note is a claim, not a result

Write the fix note for the retester and for whoever reads the record later. It tells them what changed and where to look harder, such as a check added to one route handler when the finding listed three routes. It is not evidence that the fix works. SP 800-115 allows that an organisation may choose to verify mitigation through nontechnical means such as documentation, for example by asking the security personnel responsible for implementing it to sign a document describing all of the accomplished actions.

It says this may be appropriate and cost-effective, and is more cost-effective in the short term, but that there are risks posed by not technically verifying that changes have been implemented.

The checklist keeps the two apart: the note sits in the record beside the outcome, and only a request sent to the named build sets the outcome.

What to freeze, and what to leave out

Hold the retest build still from handover until the close-out sheet is signed: deployments, configuration and feature flags, and edge or firewall rules on the retest environment. Name one person who can approve an urgent change. That change gets a new build reference, and any finding already checked on the old build is run again or keeps the old build on its record.

SP 800-115 names the pull in the other direction during an assessment: administrators may want to take immediate steps to mitigate weaknesses and expect assessors to quickly re-assess the system, and it says assessors should communicate the importance of following the organisation's change management policies and procedures.

Then list the findings that will not be in the round, each with its reason and the name of whoever decided: a fix not ready, a risk accepted, a fix deferred to a named release, a feature due to be retired. They go into the record as not retested from the start, so nobody reads the round as covering them.

For a head start, have the fix owner try each finding's original steps against the handover build before the request goes in. A fix that fails there costs less to catch than one that fails mid-round.

Repeating the original test before changing anything

The first request for each finding is the original one, unchanged: the same role and account, route, method, parameters and payload, under the same conditions.

Nothing varies on that pass. If the request differs, a different response could come from the fix or from the difference, and the record cannot say which. That is the working meaning of the mirror copy SP 800-115 describes.

The first pass also tests the original report. The introduction to the OWASP Web Security Testing Guide (WSTG) v4.2 says the report should allow another security tester to reproduce the results. A retest is where that is put to work. If a step is missing, an account is unnamed or a payload was described rather than given, the retester notes the gap before going further, because a guessed request is not a repeat of the finding.

The same introduction says that once issues are reported, it is important to give the developer guidance on how to re-test and find the vulnerability, and that where a vulnerability can be found through a black-box penetration test, the report needs to say how to validate its exposure to the front end.

Retest criteria written into each finding before any fix are that guidance in a form both sides can check.

A repeat is still testing on a system people use, so the original conduct rules carry over. Destructive actions are off by default for real people's accounts and records that existed before the test, and reset, lockout and deletion checks run only on test accounts and test records where the rules of engagement allow them.

A finding first shown by locking an account needs a test account that can be locked again. Where the original finding was confirmed in code and code access is in the scope, the retester reads the change as well as sending the request.

The control request beside every attack request

Send the legitimate twin of each attack request: the owner opening their own record, the allowed role completing the action, valid input going through. A refused attack request means something only when the legitimate one still works. If both fail, the feature is broken or unavailable on this build, and that is a reason to stop and find out why, not a result.

The UK National Cyber Security Centre (NCSC) draws a similar line for an organisation's own functional testing of security controls. It says a functional testing plan should include positive tests, such as the logon box coming up every time you try to log in, and that negative tests, such as verifying you can't log in without the correct password, may be included where the skills are available in the organisation.

In a retest the control request is the positive test and the original attack request is the negative one, and neither means much without the other.

Evidence that a fix holds, and the checks around it

A refused request on one route is thin evidence on its own, because it says nothing about the routes beside it.

Each finding's regression checks reach outward from the original request: the other methods on the same route; the list, search, export and download versions of the same data; the API behind the screen; each role that should be refused; and other forms of the original input where the criteria call for them.

They come from the affected routes listed under the finding and from its retest criteria, written before any fix, which keeps the retest from drifting into a new test.

The WSTG v4.2 introduction describes the patch-and-penetrate model, which it says security professionals have come to realise is a fallacy: fixing a reported bug without proper investigation of the root cause. A retest shows that model as a fix added to the one route handler named in the evidence while its siblings answer as before.

NIST SP 800-115 says mitigation recommendations, including the outcome of the root cause analysis, should be developed for each finding. The cause line in the fix note is the first thing for the retester to read.

The fix does not have to be the one the report recommended. The NCSC says the solutions proposed by penetration testers may not be the only ones possible, gives alternatives such as uninstalling software that is not actually required or putting other controls in place to limit exposure, and says risk assessment and decisions on the application of fixes are the organisation's responsibility.

The retest judges the result against the criteria. It does not mark a finding down because the method differs from the recommendation.

Keeping the fix in the next build

A retest checks one build. To keep the fix in every build after it, have the fix owner turn each finding's retest criteria into an automated test in your own pipeline: the attack request expecting a refusal and the control request expecting success, run on every change to the code involved.

Put the test's name in the fix note, so the retester can see the regression check exists and a later reader can find it.

Four outcomes, the evidence each needs, and who may record it

Each finding leaves the round with exactly one outcome.

What the checklist adds to the four definitions is what each outcome needs on file and who is allowed to write it.

  • Fixed. On the named build: the original request refused or harmless, the control request working, every regression check in the criteria passing, and evidence references for all three. Written by the person who sent the requests.
  • Partially fixed. Evidence for the parts that now hold, as for fixed, plus the request and response for each listed route, role or part of the impact that still reproduces, named exactly. Written by the person who sent the requests. The finding stays open under its original ID.
  • Still reproducible. The original request working as written on the named build, its response, and the build confirmation, because a fix that did not deploy looks the same as a fix that failed. Written by the person who sent the requests, who passes the evidence back to the fix owner.
  • Not retested. The reason, and what would let the finding be retested. Written by the retester when the cause is on the testing side, or entered from the engagement owner's recorded decision when the business left the finding out. It stays open and is not shown as passed in any copy of the record.

Why the fix owner does not write the outcome

The person who changed the code writes the fix note. The engagement owner writes decisions to accept a risk or defer a fix. Only the person who sent the requests writes the outcome. If your own team performs the retest, choose someone other than the fix owner where you can, and record who ran each check and that they work on the system, so a reader can weigh the result.

SP 800-115 says it is important to continuously update the plan of action and milestones (POA&M) that drives remediation, to identify activities that have been accomplished, partially accomplished or are pending action by another individual or system. That tracks the work. The outcome tracks the evidence, and an accomplished activity is not a fixed finding until a request on the named build shows it.

Decisions sit beside outcomes, not in place of them

Accepting a risk and deferring a fix to a named release are business decisions. Each goes in its own column with the decision-maker's name and a review date. Neither moves the outcome: an accepted finding is still partially fixed, still reproducible or not retested, and the record shows both.

The NCSC says vulnerability risk assessment and mitigation is a business process and should not be wholly outsourced to the test team. Keeping the decision and the evidence in separate columns shows each half where a reader can see it.

Three cases to settle before the round starts

Once the handover is right, three situations still need a rule agreed in advance, because deciding them after the result is known invites an outcome that suits whoever is asking.

A fix that closes the reported route but leaves a sibling open

Take an invented example. 1 min read

A finding that stops reproducing for an unrelated reason

A request can fail for reasons that have nothing to do with the fix: the account in the evidence expired, the record it read was cleaned up, a feature flag is off on the retest environment, a new edge rule blocks the tester's addresses, or a third-party sandbox the step relies on is down. 1 min read

A new issue found during the retest

A retest repeats findings rather than exploring, but it still exercises the application, and something new can turn up, including a problem the fix itself introduced. 1 min read

Closing the round: two signatures and an updated record

A round closes when two people sign.

The retester signs the results: which build, which environment, between which times, the outcome and evidence references for each finding, the reasons for anything not retested, and the new findings raised. The engagement owner signs the decisions: for each finding not marked fixed, a further round, an accepted risk with an owner and a review date, or a deferral to a named release.

Then the record changes. The WSTG v4.2 reporting chapter suggests that for a re-test you might create a subsection summarising the findings of the previous test, the updated status of previously identified vulnerabilities and any cross-references with the current test. Keep the original IDs, give each outcome the round and build it came from, and list new findings after them with their cross-references.

The chapter's example disclaimer calls a test a point in time assessment, and a retest outcome has the same limit: it describes the build retested, on the dates given.

Last, tidy up. Disable the retest accounts, revoke any repository or environment access given for the round, list the records the retest created for removal, and replace any summary already shared with a customer or insurer with the updated one. If production runs a different build from the one retested, the summary says so.

Penetration test retest policy: questions to settle before you buy

The retest is part of what you are buying, so settle its terms before testing starts rather than after the report arrives.

Put these questions to any provider and have the answers written into the statement of work.

  • Which findings does the retest cover: every reported finding, or only some ratings?
  • When can it be requested, and what starts that window?
  • How many rounds are included, and how is a further round agreed?
  • What counts as the same application, and does the fix have to be in the environment that was first tested?
  • Is a fix made during the testing window confirmed at the time, or left for the retest?
  • Does the signed authorisation cover the retest dates, or will a variation be needed?
  • Which accounts, records and access have to be kept until the retest is complete?
  • How does the updated report show a finding that was not retested, and a new issue found during the round?
  • Can findings from another provider's report be retested, and against which evidence?

How the retest works with Clever Ops

Every Clever Ops test includes one consolidated retest. You request it within the window set in your statement of work, once your fixes are deployed to an identified build of the same application. It covers the reported findings within the agreed scope, checking each against the retest criteria written into its finding, and new features and unrelated changes are scoped as separate work.

Afterwards each finding carries one of the four outcomes, and a finding that was not retested is reported as not retested rather than as passed.

Further fix work and any additional retest are agreed separately. Findings from another provider's report can be retested as a separate piece of work, against the evidence in their report, and a security fix sprint can handle remediation under its own statement of work. If Clever Ops built or hosts the application, the work is a security review rather than an independent test, and the report says who built and who hosts it.

Where a tender, panel or contract requires a CREST-accredited provider, Clever Ops is not CREST accredited, so we refer that work to a provider that is.

Frequently asked questions

A second look at the findings from a penetration test once the fixes are deployed. It repeats each finding's original evidence on a named build, checks that the legitimate use of the feature still works, runs the regression checks agreed for that finding and records one outcome: fixed, partially fixed, still reproducible or not retested. It covers the reported findings, not the whole application.

The handover from your team (one build carrying every fix, a fix note per finding, the original accounts and records, unchanged test conditions and a freeze), the run for each finding (confirm the build, repeat the original steps exactly, send the control request, run the agreed regression checks, capture redacted evidence), the evidence each outcome needs, rules for sibling routes, unrelated failures and new issues, and a close-out sheet signed by the retester and the engagement owner.

The developer records what changed, in a fix note, and that note sits in the record beside the outcome. The outcome itself is written by the person who sent the requests on the named build. NIST SP 800-115 allows documentation as one way to verify mitigation, but says there are risks posed by not technically verifying that changes have been implemented.

For each finding: the build reference and environment, the original request sent again with its response, the control request showing the legitimate use still works, the result of each regression check in the retest criteria, evidence references with session values, passwords and personal details masked, the outcome, and who recorded it and when. For a finding not retested, the reason and what would let it be retested.

No. A retest repeats specific findings on a named build and checks the routes and criteria written into each one. It does not explore new features or test the application afresh, so a clean retest says nothing about code the original test did not cover. New features and other changes since the test need testing of their own.

The original finding IDs, the round number, the build and environment retested and the dates, then for each finding its outcome, its evidence references and any owner decision beside it, followed by new findings with their cross-references. The close-out sheet at the end of the checklist above holds those fields, and the finding ledger in a full report template can carry them from one round to the next.

Scope the right test for your application

Tell us what you have built, who uses it and when it needs to launch. You will get a plain-English recommendation: a website audit, a web application test or a referral to a specialist.