AI
Why generalist AI breaks on enterprise work
Generalist AI works well enough for generic tasks. For core business work, it breaks on the specifics: a legal firm’s contract language, a drug company’s discovery workflow, a consulting firm’s proprietary methodology. The model hasn’t learned how work gets done inside that organization.
Traditional fine-tuning fixes this but costs months of data-scientist time and hands those scientists access to sensitive documents. Copilot Tuning is Microsoft’s approach to making fine-tuning self-serve, running entirely inside the M365 compliance boundary without Microsoft employees seeing the data.
The enterprise brain: what gets built when you fine-tune
Fine-tuning a model is only part of what Copilot Tuning produces. The output is what Ranveer Chandra calls the “enterprise brain”: a fine-tuned model plus task recipes, reward models encoding what “good” looks like for that organization, and ACLs inheriting the original data’s permissions. That last layer solves a standard fine-tuning risk: train on sensitive documents without access controls and anyone querying the model can effectively read them. Permissions flow from source data through to model output. The IT admin governs them like any other M365 resource.
Automated data preparation with Project Sargy
Extracting high-quality training samples from a million messy enterprise documents is the hard part of fine-tuning. Project Sargy (accepted at ICML) automates most of it. GPT-4 labels roughly 70% of the data correctly on a first pass. Sargy then trains a reward model on those samples and plots a quality distribution with an elbow (good data) and a knee (bad data). Human annotators label only a handful of points at those two boundaries, which flattens the slope and expands the usable training set. On McCarthy Tetrault’s corpus of hundreds of thousands of documents, the approach cut false positives 3x using only 1.25% annotation.
Task recipes: doc generation, expert Q&A, and summarization
Three recipes ship at Build, with roughly a dozen planned by year-end. Doc generation builds an adaptive outline first, then writes section by section so each paragraph is aware of all preceding content. Guardrails assert specific contract terms against the finished draft: a given interest rate appears consistently, or the balance of power favors the purchaser. Expert Q&A trains on synthetic data calibrated to a specific reasoning style and combines the tuned model with RAG for live data updates. Summarization learns preferred output format from examples, whether bullets, narrative, or a visual.
Live demo: configuring and deploying a fine-tuned agent
Khushboo Aggarwal walked through the full flow in the Copilot Studio fine-tuning demo (37:45) without writing code. The model maker selects a SharePoint corpus, grants permissions to security groups, picks a task recipe, and answers four plain-English questions about terminology and guardrails. Copilot Tuning analyzes ACLs and surfaces trade-offs: two groups cover 95% of the knowledge; one group gets 100%. After labeling a handful of AI-selected samples yes or no, training runs through Azure AI Foundry. The output: a five-page partner agreement in Word, matching the organization’s exact terminology.
Customer deployments and when fine-tuning is worth it
Five deployments are in pilot. McCarthy Tetrault drafts 50-to-500-page principal agreements on hundred-million-dollar deals. Ernst and Young chains an expert Q&A agent (trained on proprietary tax data) into a doc-generation agent that writes the client memo. Land O’Lakes preserves contract expertise that leaves with departing employees. Microsoft Legal uses fine-tuning rather than RAG because Poland and Lithuania regulations differ at a granularity retrieval hallucinates on. Ranveer Chandra named three conditions for when to fine-tune: existing Copilot falls short, the task is high-value, and you have good data.
“Fine-tuning is not cheap.” — Ranveer Chandra
The feature opens mid-June to enterprises with 5,000 or more Copilot seats.
Notable Quotes
Microsoft doesn’t get access to your data. Ranveer Chandra · ▶ 8:15
This is an enterprise product, period. Kalyan Ayloo · ▶ 33:52
Fine-tuning is not cheap. Ranveer Chandra · ▶ 58:34
Key Takeaways
- Copilot Tuning keeps all training data inside the M365 compliance boundary, with no Microsoft employee access.
- Project Sargy cuts annotation cost to 1.25% of the corpus by using a reward model to identify the highest-signal labeling points, cutting false positives 3x.
- Fine-tune only when existing Copilot is insufficient, the task is high-value, and you have good training data.