AI
Why LLM World Knowledge Is Not Enough for Enterprise Agents
Enterprises sit on more than 9 zettabytes of data, and 44% of it goes uncaptured. The other 68% that is collected largely goes unused. LLMs trained on public data cannot fix that gap on their own.
“LLMs are good at general knowledge.” — Rakesh Krishnan
That general knowledge does not answer questions about a company’s CRM, ERP tables, or internal HR records. Copilot Studio’s knowledge platform addresses this through RAG: it augments the LLM with enterprise data from sources like Dataverse, SharePoint, Salesforce, and OneDrive, then retrieves grounded answers instead of generic ones.
Pattern 1: Ingesting Unstructured Data into Dataverse
Pattern 1 handles unstructured content: Word documents, PDFs, and Excel files. Point Copilot Studio at an OneDrive folder and it syncs files into Dataverse, builds Azure AI Search vector embeddings, and keeps them current without reconfiguration. He walked through the OneDrive file picker and indexing demo (06:43) live.
For Excel and CSV, the platform detects when a question requires computation and invokes Code Interpreter automatically. An agent can calculate how long a candidate took to move from phone screen to offer date, running code against the uploaded file rather than retrieving text chunks.
Pattern 2: Querying Structured Data Without Moving It
Pattern 2 indexes only metadata: table names and column names. At query time the agent translates natural language into an API call, executes it against the source, and returns results. Existing RBAC policies in Snowflake, Databricks, Azure SQL, or Salesforce apply in full. No data leaves those systems.
Configuration also accepts synonyms and a glossary. During the Snowflake synonyms and glossary configuration demo (16:56), Rakesh mapped “PTO balance” to “vacation balance” and registered FML and PIP as company acronyms. Those entries let the agent build correct SQL before it ever touches the warehouse.
Pattern 3: Power Platform Connectors and Multi-Source Queries
Pattern 3 uses Power Platform’s 1,400 connectors for read operations, including Azure AI Search indexes already built in a customer’s subscription. One click adds the index as a knowledge source with no re-indexing.
The hiring agent combined multi-source demo (27:55) put all three patterns together. The agent searched OneDrive resumes, pulled a count from a SharePoint List, queried Snowflake for the max employee salary ($200,000), then found a high-priority Salesforce case, described it in plain language, and cited the Salesforce knowledge-base article with a direct link.
Customer Deployments: CSX, ESET, and Encino Energy
CSX, the freight railroad with 21,000 track miles across 23 states, built an agent called Chessie on Copilot Studio and Azure AI Foundry. Embedded in shipcsx.com (7,000 daily users), Chessie answers customer questions without a human in the loop. Business teams upload updated FAQs directly, no developer required.
ESET connected Azure AI Search as a knowledge source and cut support tickets handled by human agents. Encino Energy, Ohio’s largest oil producer, replaced complex hand-written SQL reports by connecting Snowflake as a knowledge source, so field operators now query production data in plain language.
Notable Quotes
moving data from one place to another is a big nightmare. Srikumar Nair · ▶ 1:47
LLMs are good at general knowledge. Rakesh Krishnan · ▶ 1:20
you can build exactly what you need for your business users Don Jackson · ▶ 37:47
Key Takeaways
- Three RAG patterns cover every enterprise scenario: ingest unstructured data, query structured data in-place, or read via Power Platform connector.
- Pattern 2 indexes only metadata, so existing RBAC and security policies in Snowflake, Salesforce, and SharePoint stay fully enforced at query time.
- Synonyms and glossary configuration let agents resolve company-specific acronyms like FML and PIP before translating natural language into SQL.