A Practical Workflow for AI-Found Vulnerability Reports
AI security tools can surface candidate vulnerabilities, but teams still need a reliable path from finding to fix. Build a workflow for triage, ownership and verification.
The GitHub Blog recently described using an open-source AI security agent to find Android vulnerabilities. For engineering leaders, the useful question is not whether one tool can discover flaws; it is how a team should handle candidate findings without flooding developers with noise or treating machine output as proof. A dependable workflow connects each report to a reproducible case, a named owner, a risk-based decision and a verified change. That is a software delivery problem as much as a security problem. This field note sets out practical controls for turning AI-assisted discovery into work that fits existing engineering processes.
Treat each result as a hypothesis, not a ticket verdict
An automated finding is a lead that needs evidence. Before creating a high-priority issue, ask a reviewer to identify the affected component, the relevant input or execution path, and the security consequence if the behavior is real. Capture the tool version, repository revision, configuration and available reproduction steps so another engineer can inspect the same result. Keep the original output attached, but distinguish observed behavior from the tool’s interpretation. This separation prevents confident-sounding explanations from becoming accepted facts and makes later reassessment possible when code or assumptions change.
Use a consistent intake template rather than letting each tool invent its own ticket format. Record the file or endpoint, a concise description, preconditions, expected versus observed behavior, and whether the report has been reproduced. Note missing context explicitly: for example, a suspected authorization issue may depend on a caller identity or deployment setting that is not visible in the repository. A reviewer can then classify the result as confirmed, needs investigation, duplicate, or not applicable. Preserve the reason for the decision. That creates useful engineering history without pretending every candidate is a confirmed vulnerability.
- Require a reproducible path or a documented reason reproduction is currently unavailable.
- Keep evidence and reviewer conclusions separate from generated explanation.
Route confirmed issues by risk, ownership and delivery impact
Once a finding is credible, send it to the team that owns the affected code, not simply to a central security queue. Assign an accountable engineer and a reviewer, then classify urgency using the reachable attack path, potential impact, affected users or data, and exposure of the component. A tool’s severity label can inform discussion but should not set the deadline by itself. A flaw in an unreachable test utility and a flaw in an externally accessible authorization boundary call for different responses. Make the rationale visible so product, security and engineering can agree on priorities.
Create a remediation issue that describes the desired security property, not just a suggested patch. State what must become impossible, which behavior must remain supported, and what tests will demonstrate both. Link the finding to the relevant code owner, change request and release decision. If immediate remediation is not practical, record the reason, compensating controls if any, decision owner and review date. This turns an exception into an explicit, time-bounded risk decision rather than an issue that silently disappears in a backlog. Avoid promising a universal fix deadline; urgency depends on the application and its deployment context.
- Define escalation and risk-acceptance ownership before the first urgent report arrives.
- Track duplicates and false positives to improve intake rules, not to suppress inconvenient findings.
Close the loop with tests and a measured pilot
A code change is not evidence of closure until the original behavior has been checked again. Add a focused regression test where practical, review the fix against the security property, and rerun the relevant analysis against the updated revision. Record whether the candidate is resolved, remains open, or needs a different interpretation. For complex cases, have someone other than the implementer inspect the reproduction and test. This modest separation reduces the risk that a patch satisfies the report’s wording while leaving the underlying weakness intact. The final record should let a future maintainer understand what changed and why.
Pilot the workflow on one repository or service before making it a mandatory organization-wide gate. Measure operational signals that help tune the process: time from report to human decision, proportion of reports reproduced, duplicate rate, time to assign an owner, and whether fixes gain regression coverage. These are internal process measures, not evidence that a tool finds every weakness or makes software secure. If the queue grows faster than teams can review it, narrow scope, improve deduplication or adjust the review cadence. Expand only when ownership, evidence quality and remediation capacity are working together.
- Review a sample of rejected reports periodically to detect mistaken dismissals.
- Keep human approval for severity, risk acceptance and closure decisions.
AI-assisted discovery becomes valuable when findings enter a disciplined engineering loop: preserve evidence, validate before prioritizing, assign work to code owners, specify the security outcome and verify the fix. The recent GitHub example is a timely signal, not a guarantee that automated reports are complete or correct. Teams can start with a small pilot and improve the handoffs before scaling. The practical measure of success is not the number of alerts produced; it is whether credible issues become owned, tested and traceable changes without overwhelming delivery teams.