Recently, we've been pushed to radically change our security review process with a large client whose LLM-powered development cycle has shrunk to roughly two weeks. Every two weeks they are pushing consensus-critical code to a major network, with the standard of "can't break." We would normally scope about 4 weeks to review the code this team pushes every two weeks, clearly a mismatch and impossible to keep up while maintaining review quality. So we've re-invented our audit workflow to adapt to these changes in developer workflows.
Yes, AI tools have sped up auditors, we've observed this across the team. But the gains to developer speed are not mapping one-to-one to auditor speed. We've observed that AI can speed up mapping out codebases, initial understanding of codebases, report generation, and many other aspects of review. But in order to find the ~20% of crits that LLMs are still not finding, the tier of bugs we view as our differentiator, we still need manual review, which takes up ~60% of audit time.
The standard audit cycle consists of code freeze, code review, bug report, bug fixes, and review of bug fixes, all in sequence. This model works quite naturally with code that is immutable or infrequently upgraded (smart contracts). It also works well with high-availability code with strong security guarantees. And it is the most efficient model per unit of reviewer effort: a frozen target means every hour of review lands on code that will actually ship. But we are getting to the point where teams can output code with high security guarantees at a faster pace due to AI-augmented dev team speed, and holding that code frozen for weeks is exactly the cost these teams can no longer pay. So what do we do?
Rolling commit reviews
Code freezes have always been at odds with developer speed, and there is a constant push and pull for last-minute changes vs. audit scope creep. We've started moving to rolling commit reviews to at least alleviate the code freeze requirement. This allows audits to be less disruptive to the development cycle and increases overall pace, at the cost of extra security review time due to the potential requirement to review multiple changes to the same code.
Rolling commit reviews remain as rigorous as freeze commit reviews. The engagement still opens with a baseline commit reviewed the same way a freeze commit would be, scoped changes are then reviewed incrementally as PRs land, and it closes with an integration review of a single final assessed commit. The report and the audit opinion apply to that final commit and the tracked scope that led to it, so the deliverable stays as concrete as a fixed-scope audit's.
Integrated security engineers
For the described client, rolling commit reviews alone were not enough. We started integrating security engineers directly into the project workflow. As PRs were raised and passed internal review, we reviewed the PRs themselves. When several PRs formed a coherent feature, we reviewed them as a batch. We left comments directly on PRs. We went back and forth with developers in Slack about these PRs. Security reviewers became a more integrated part of the team's engineering loop. The review process became parallel instead of sequential, concurrent across PRs, mirroring the development workflow.
However, this is not a free win. From a pure review-time perspective, it can be inefficient because you may review a function, review it again after it changes, and review it again before release. In the end though, only the final version matters.
The core trade-off is accepting the price of more reviewer time in exchange for reducing review wall-clock time. This review-time amplification is offset to some extent by gains in reviewer efficiency. Like a developer on their own codebase, security reviewers gain context from intermediate states and get deeper insight into reasoning behind changes. There is no starting from scratch; reviewers maintain a degree of familiarity constantly. Constant feedback and interaction naturally improves communication.
You could call this an experiment in "forward deployed" security engineering. LLMs have dropped the cost of code generation through the floor. But the best development teams are understanding code to completion. Reaching this level of understanding is likely the current bottleneck in the development cycle. It's the same bottleneck a security reviewer faces on every new engagement. Security engineers embedded as deeply as possible in the development cycle can minimize the additive time of security by reaching this understanding in parallel.