AI
Three Models for Domain Expertise
Lovejoy says winning in vertical AI is an organizational problem. He outlines three ways to embed domain expertise. The oracle directly improves AI outputs by tweaking prompts and adding documents. The evaluator defines metrics and builds systems to measure quality. The architect designs automated loops that learn from usage. The right model depends on whether performance is measurable and whether manual iteration is fast enough. Gartner reports 50% of generative AI projects were abandoned last year. Lovejoy attributes that to a lack of deep workflow understanding.
Case Study: Granoola’s Oracle
Granoola, a meeting notes AI company valued at over $1 billion, uses an oracle model. Joe, the first employee with a writing and journalism background, wrote all prompts. She acts as the primary gatekeeper of AI quality. She assesses outputs and makes improvements directly. This works because there is no objectively perfect meeting note. Taste matters. Even at scale, the core output is amenable to direct human review. Lovejoy argues that when taste is more important than metrics, an oracle is the right fit.
Case Study: Tandem and Anterior’s Evolution
Tandem’s medical AI scribe started with an oracle: doctor Roy reviewed notes and updated prompts. As scale grew, Tandem moved to a decentralized oracle with many doctors handling different specialties and countries. Anterior, a prior authorization startup, progressed from oracle to evaluator to architect. Lovejoy defined metrics, built a review dashboard, and hired clinicians to assess outputs. Manual iteration became too slow due to variation in policy interpretation. He then designed automated improvement systems that learn from usage at the edge.
Hiring and Organizing a Principal Domain Expert
Lovejoy recommends hiring a principal domain expert early. This person should have relevant domain expertise and breadth of adjacent skills like data science, prompting, or product management. Give them ownership, not an advisory role. Start them as an oracle, then evolve their role to evaluator or architect as the product scales. Avoid consensus by committee. A single accountable person speeds decisions. Pair them with complementary skills (e.g., a statistician) if needed. Lovejoy saw two senior clinicians leave a company because they lacked ownership and clear decision rights.
Notable Quotes
about 50% of all generative AI projects were abandoned last year. Chris Lovejoy · ▶ Watch (3:38)
winning in vertical AI is an organizational problem. Chris Lovejoy · ▶ Watch (2:33)
front-end models are good enough, but the gap is now how do organizations operationalize the expert judgment around them? Chris Lovejoy · ▶ Watch (4:07)
there’s no objectively perfect meeting note. Chris Lovejoy · ▶ Watch (11:48)
you want to give them ownership. Chris Lovejoy · ▶ Watch (20:26)
Key Takeaways
- Winning in vertical AI requires organizational design, not just model sophistication.
- Choose oracle, evaluator, or architect based on measurability and iteration speed.
- Hire a principal domain expert early, give them ownership, and evolve their role as the product scales.