AI
What a Security Self-Assessment Actually Contains
A security self-assessment is a structured document a project writes to reason about its own security and communicate it to others. It covers system components and whether failures in one area spill into others, the impact of different attack types, and whether the project follows basic security practices. Justin Cappos, who created the assessment process for TAG Security, notes that projects often spot their own issues while writing. The document also serves as the prerequisite for a joint assessment, where outside experts probe the same ground more deeply.
How Projects Discovered Their Own Gaps
k8gb entered the process with a 34-line document and exited with a 230-line document naming actors, actions, and components. K3s found that all security issues were routing to a SUSE-internal email address rather than a public channel. Kyverno identified that its global context entry feature, which fetches and caches data for faster lookup, lacked sufficient bounds limits. Orlin Vasilev noted K3s assumed it was handling security well but was not communicating its posture to the community. Each team described the process as revealing things they did not know to ask about.
Why Projects Started: Compliance That Became Conviction
All three projects began the assessment to satisfy a CNCF graduation or incubation requirement. Orlin Vasilev called it a checkbox that “actually transferred to something that we had to do long time ago.” Shuting Zhao noted Kyverno runs as an admission controller doing image verification, so security was already a focus, but the process found gaps that shipping features had masked. Bradley Andersen pointed out that k8gb handles traffic only at L3 and L4 and assumed it did not need deep security review. TAG Security re-runs the assessment check annually through its Security Slam program.
Practical Advice for Maintainers Starting an Assessment
Cappos’s first rule: submit before you think you are ready. He reviews every incoming PR and uses early feedback to test whether a team will engage or ignore comments, which predicts whether a joint assessment will go well. Shuting Zhao added that honesty about gaps matters because reviewers are there to help, not judge. Orlin Vasilev recommended reading other projects’ published self-assessments before writing your own, noting most people in the room had never done that. He also warned the document is not a weekend task.
Building on the Assessment: Joint Reviews and Open Issues
After completing a self-assessment, the logical next step is a joint assessment where TAG Security experts work through the document with the project and probe its reasoning. Cappos said projects that do their own assessment tend to pursue the joint review voluntarily. He also ran a graduate security class of more than 100 students at NYU, sending teams to produce starter assessments for projects that had not done their own. Those documents are checked in as baselines. Projects that find unsolvable problems during their assessment are encouraged to post open issues and contact the community, since security is one area where CNCF projects are not competing against each other.
Q&A
Does making the self-assessment mandatory increase uptake of joint assessments? Cappos confirmed that projects completing their own assessment frequently pursue a joint assessment immediately, and those that do report learning substantially more from the external feedback. ▶ 29:19
How do you raise security awareness inside organizations where people feel nothing has gone wrong yet? Cappos suggested reframing security work as compliance, noting EU fines can reach hundreds of millions of dollars, and recommended red-teaming your own systems and showing management what is possible. ▶ 24:12
Notable Quotes
we went from something like a 34line document that said yeah we do esponds we sign our commits uh to something like a 230 line document that says these are the actors these are the actions this is what everything does Bradley Andersen · ▶ 07:02
don’t wait until you think you’re ready. Just submit it and Justin and other reviewers will chime in and tell you, okay, you have to fix this and that and you’ll go through the process. Shuting Zhao · ▶ 20:14
there is no security system that is like unconditionally secure against every single possible attack, you know. So that is okay. That is expected. That is if you say that is not the case, then we are very worried that you don’t know what you’re talking about. Justin Cappos · ▶ 21:08
from checkbox it actually came to uh realization that we have to put a lot of effort because a lot of people are uh relying on our project Orlin Vasilev · ▶ 12:15
Key Takeaways
- k8gb’s security document grew from 34 lines to 230 lines through the assessment process.
- K3s found security reports routing to a private corporate email instead of a public channel.
- Kyverno removed cluster-wide secret view permissions as a direct result of least-privilege review.
- Completing a self-assessment most often motivates teams to pursue a joint assessment voluntarily.
- Reading published assessments from other CNCF projects is a practical starting point for new authors.
About the Speaker(s)
Justin Cappos is a professor at NYU with over 20 years working on software supply chain security. He created the TAG Security assessment process and is a maintainer and creator of TUF, Uptane, and in-toto, all Linux Foundation projects.
Shuting Zhao is a Kyverno maintainer and Staff Engineer at Nirmata. She has mentored contributors through LFX mentorship programs since March 2021 and participated directly in the TAG Security joint assessment process for Kyverno.
Bradley Andersen is community manager for k8gb, a global service load balancer for Kubernetes that operates over DNS.
Orlin Vasilev is an Open Source and Cloud Native Technology Advocate for Gardener and the NeoNephos Foundation at SAP. He serves as community manager for Harbor and K3s and is a second-term CNCF Ambassador.