AI
Why Piecemeal Security Programs Fall Short
Application security requires coverage across the entire development lifecycle. Pen testing, patching, or threat modeling in isolation each address only part of the problem. OWASP SAMM maps the full scope, covering 15 practice areas. What it doesn’t give you is the order and priority. ASVS-driven development fills that gap: treat OWASP ASVS requirements as real sprint requirements and write automated tests to verify them.
ASVS Requirements as First-Class Development Work
ASVS-driven development adds ASVS on top of existing software engineering practices. Developers already specify requirements, implement them, and write unit and integration tests. ASVS requirements slot into that same flow: they become sprint tickets. Engineers implement and test each one. When the pipeline runs, any regression breaks the build. A junior developer can do this. The knowledge transfer happens as a side effect: adding a requirement that passwords must be at least 12 characters teaches the engineer why it matters, without a separate security course.
What the SAMM Benchmark Reveals About Testing
The SAMM Benchmark collects anonymized assessments from organizations that have adopted SAMM. The data shows security requirements as one of the highest-scoring SAMM activities. Requirements-driven testing ranks in the bottom five. Organizations write security requirements, then don’t verify them. That gap is the failure point. Hovsepyan’s argument: you need both. A requirement without a test leaves you assuming engineers implemented it correctly.
How Much of ASVS Is Automatable
Hovsepyan analyzed every ASVS v5 requirement to determine whether a unit, integration, or acceptance test could cover it. Result: 49% are fully testable with automated tests, 6% partially, putting over half of ASVS within reach of CI/CD. SAST tools cover an additional 40 of 278 requirements, though they need tuning, not out-of-the-box deployment. DAST adds 26 more, suited to running-application checks like security headers. SCA covers 3, all related to software supply chain and SBOM.
Testing ASVS Coverage in a Real Codebase
Codific applied ASVS-driven development to a Symfony PHP project that took four person-years to build. They selected 98 ASVS v4 requirements and put each into sprint planning as individual tickets. A junior engineer had 10 person-days total, 8 as primary implementer and 2 with Hovsepyan guiding interpretation and test reviews. The final score: 23% of the full ASVS completely covered by tests running in CI/CD, 9% partially. Failing tests surfaced real gaps: MFA replay attacks and MFA reset notifications were both absent.
Starting Small and Growing a Security Test Suite
Start with five requirements, not 90. Password checks are the easiest entry point: minimum length and complexity rules convert directly to unit tests. Add requirements from your own organizational policies on top of ASVS. These tests belong in CI/CD and should grow as the codebase grows. Security requirements then become the common language across the team: architects define them, engineers implement and test them, QA confirms.
Q&A
Did you use ASVS’s three levels to prioritize which requirements to implement first? Hovsepyan ignored the levels entirely during the study, though he agreed that starting with level one would be a significant win for most applications and could guide which requirements to pull in first. ▶ Watch (40:50)
How do you scale this across a microservices architecture? Start with a smaller subset of ASVS, perhaps 20 requirements scoped to a single repository, pick the easiest ones first, and build outward. Cross-service integration tests come later, after the per-service baseline is in place. ▶ Watch (42:27)
Is there a list showing which ASVS requirements map to SAST and DAST tools? Yes, a separate tab in Hovsepyan’s analysis spreadsheet covers SAST and DAST-testable requirements, with the caveat that coverage assumes ideal tool capabilities, not out-of-the-box behavior. ▶ Watch (43:51)
Notable Quotes
a time boox activity and we’ve done 30% Aram Hovsepyan · ▶ Watch (3:26)
asvs we’ve implemented 23% completely Aram Hovsepyan · ▶ Watch (37:06)
say start small don’t go 90 go five Aram Hovsepyan · ▶ Watch (38:32)
Key Takeaways
- ASVS-driven development converts security requirements into CI/CD-enforced tests, catching regressions on every commit.
- Over half of ASVS requirements can be verified with automated unit and integration tests.
- A junior engineer covered 23% of the full ASVS completely in 10 person-days with a time-boxed sprint.
- SAST tools need configuration tuning to catch ASVS requirements that automated tests miss.
- Start with five easy ASVS requirements, build passing tests into the pipeline, then expand.
About the Speaker
Aram Hovsepyan has 15 years of experience in application security as a researcher, industry expert, and core contributor to the OWASP SAMM project. He is the founder and CEO of Codific, a Belgian cybersecurity product firm.