Why Enterprise GenAI Keeps Failing

▶ Watch (06:30)

MIT reported a 95% enterprise GenAI failure rate. Johnson thinks the number is slightly high, but the pattern holds: money piles into proofs of concept and almost none reaches production. The root cause is a category error. Personal assistants like ChatGPT run on one powerful model with a three-page system prompt and a broad toolset. That works when a human watches and course-corrects. It fails when a business process needs to run reliably on its own. The parallel to early enterprise Java is direct: top-down vendor mandates drove a similar failure rate until a developer-led movement built things that worked.

Three Tactics Against Non-Determinism

▶ Watch (16:18)

Non-determinism in LLMs cannot be eliminated, but it can be minimized. The first move: break complex tasks into multiple steps, each with a smaller prompt and fewer tools. Johnson estimates this removes 80 to 90 percent of prompt engineering pain when done correctly. The second move: write the orchestration in code, not in the LLM. LangGraph uses a finite state machine; Crew AI sequences steps explicitly. Either way, the LLM stops choosing the path. The third move: replace natural-language I/O with typed structures. JSON and domain objects are testable in code. Natural-language strings are not.

Domain-Integrated Context Engineering

▶ Watch (23:50)

Context engineering improved on prompt engineering by shifting attention to what you send an LLM, not just how you phrase it. Johnson argues the field still misses something: the memory taxonomies people cite (short-term, long-term, episodic) describe human cognition, not enterprise systems. A bank customer service rep is not relying on personal memory. Every fact relevant to the call lives in structured systems of record. That state is already well-maintained. The most important context for an enterprise LLM call is the existing domain model. Push domain objects in, get domain objects back, typed on both ends.

Embabel: GOAP Planning on the JVM

▶ Watch (33:00)

Embabel is Johnson’s open-source JVM agent framework. Its key innovation is a deterministic planning layer built on GOAP (Goal-Oriented Action Planning), a non-LLM algorithm from game AI. An Embabel server holds typed actions and goals. Given a goal and the current world state, an A* planner chains actions in the optimal order, reassessing after each step. The system adapts when the initial plan fails without pre-wiring every transition. He walked through the Embabel action annotation code example (37:36) to show how typed method signatures replace magic strings and automatically express pre/post-conditions.

Multi-LLM Orchestration in Practice

▶ Watch (46:02)

Johnson built 90 percent of the talk’s slides with an Embabel agent. Input: a YAML file mapped to a Kotlin data class with notes, image hints, and GitHub repo links. GPT-4.1 identified research topics. GPT-4.1 mini, cheaper and armed with web search, ran the research in parallel. Claude 3.7 Sonnet (which outperformed GPT-4.1 on this task) assembled the final slides. A separate illustration pass searched for images in parallel using code, not an LLM. The Embabel slide-deck agent source on GitHub (49:23) is fully open source under the Embabel org.

Notable Quotes

Prompt engineering is more like alchemy. Rod Johnson · ▶ 10:25

What we’re doing is making simple HTTP Rod Johnson · ▶ 31:28

So 90% of this slide deck was built with Rod Johnson · ▶ 46:05

Key Takeaways

  • Enterprise GenAI fails because personal-assistant patterns, large prompts and unpredictable orchestration, cannot meet business reliability requirements.
  • Breaking workflows into small, orchestrated steps with typed I/O removes 80 to 90 percent of prompt engineering fragility.
  • Java’s strong typing and existing domain models make the JVM better suited than Python for production enterprise agents.