From Code to Context: A New Lifecycle

▶ Watch (0:51)

Patrick Debois proposed treating context, prompts, instructions, and skills, as code with its own lifecycle. The loop has five stages: generate, test, distribute, observe, and adapt. Skills turn reusable code patterns (Python, Node.js packaging) back into context. agent.md files standardize instructions across coding agents, replacing Claude.md. The shift mirrors DevOps’ 2009 question: what if ops looked more like dev? Now the question is what if context is the code.

Testing Context Requires Evals and Error Budgets

▶ Watch (6:29)

Testing context needs new approaches beyond code tests. Debois showed three levels: linting validates context format (description length limits); Grammarly-style checks ask the LLM if the context is clear; LLM-as-judge tests verify generated code follows rules like prefixing every API endpoint with “awesome”. Because LLM outputs are non-deterministic, evals must run multiple times with error budgets. Four out of 5 passes might be acceptable.

Packaging Context as Skills with Registries and Dependencies

▶ Watch (14:01)

Debois compared distributing context to packaging code libraries. Skills bundle context, scripts, and documents into a format that coding agents support. A registry lets teams discover skills, though “99.9% of skills is crap.” Dependencies create version conflicts, repeating the pattern of package management. Security scanners like Snyk now scan context for credentials and third-party issues. AI SBOMs track which model built each skill.

Closing the Loop Through Observation and Feedback

▶ Watch (17:56)

Feedback on context quality comes from agent logs, PR reviews, and production errors. When an agent reports “missing this piece,” teams surface that signal and create new context for everyone. PR feedback that a generated change is wrong should improve the context, not just the PR. Debois showed production instrumentation that captures failing code and creates test cases automatically. Context filters block prompt injections in downloaded skills.

Q&A

Can architectural problem definitions and consistency checks serve as exotic forms of context for agents? Debois said writing evals rigorously takes more time than people expect. You must get every eval prompt right, not just the main prompt. ▶ Watch (26:00)

Notable Quotes

99.9, and I mean that in a very sincere way, of the skills is crap. Patrick Debois · ▶ Watch (15:09)

I’m sorry, but also with context we’re going to have dependency hell. Patrick Debois · ▶ Watch (16:23)

If you give the engine the wrong fuel, which is context, they’re not going to perform. Patrick Debois · ▶ Watch (23:50)

Key Takeaways

  • Context needs its own lifecycle: generate, test, distribute, observe, and adapt.
  • Testing context requires evals, LLM-as-judge checks, and error budgets.
  • Skills package context with registries, dependency hell, and security scanning.