From LLM to Agents: The Evolution

▶ Watch (1:58)

LLMs only work on training data. They cannot access real-time enterprise data. RAG solved that by creating embeddings in a vector database. But RAG is not real-time. Agents go further. Agents make tool calls. MCP standardizes how agents talk to tools. Before MCP, each agent had its own runtime with embedded tools. MCP separates the tool server from the agent runtime. The MCP server runs in Kubernetes, not just a laptop.

Why Agents Are Different from Traditional Applications

▶ Watch (9:35)

A traditional loan application follows a fixed flow: submit, validate, credit check, income verify, decision. Agents work on intent and outcomes. They ask clarifying questions, run checks in parallel or sequence, and may recommend a roadmap instead of a hard rejection. Amar used a Lego analogy: traditional apps are pre-built sets. Agents get building blocks and solve the problem dynamically. That brings unpredictability.

Enterprise Challenges: Security, Observability, and Legacy

▶ Watch (12:06)

Enterprises face multiple LLM providers, AI tools, and gateways. Without a platform, teams build redundant solutions. Observability is critical because agent behavior is dynamic and unpredictable. Data poses security and privacy risks. Agents might call tools without proper consent. Legacy systems like AS/400 and mainframes still run. Integrating new agents with old applications is hard. Cost blow-ups from uncontrolled LLM calls and API key fraud are real threats.

The Platform Architecture: Four Control Planes

▶ Watch (18:30)

The proposed platform has four planes. First: security, governance, and observability. Without proper access controls and audit trails, nothing goes to production. Second: model plane. Swap models like GPT-4 for Anthropic without code changes. Third: runtime and orchestration. Agents run in isolation, coordinate with each other. Fourth: MCP server domain. This is where agents interact with enterprise data. Existing APIs and microservices are reused, not rewritten.

Q&A

Can agents ever achieve deterministic business outcomes for regulatory or network management? Human-in-the-loop is mandatory; multi-agent systems can improve consistency but not guarantee determinism. ▶ Watch (22:42)

Notable Quotes

agents are actually making the tool calls Neelabh Tripathi · ▶ Watch (5:20)

your MCP server doesn’t necessarily needs to be connected or be running in your laptops Neelabh Tripathi · ▶ Watch (7:56)

without observability, there is no way Amar Deep Singh · ▶ Watch (26:12)

data remains where it is expected to remain Amar Deep Singh · ▶ Watch (25:36)

Key Takeaways

  • MCP standardizes tool access for agents, separating server from runtime.
  • Enterprise deployment requires four control planes: security, model, runtime, and MCP server.
  • Observability and human-in-the-loop are mandatory for probabilistic agent workflows.

About the Speaker(s)

Amar Deep Singh is a distinguished software architect and author with extensive experience in microservices and cloud computing. He is the author of “Building and Delivering Microservices on AWS,” a comprehensive guide that explores software architecture patterns and the deployment…

Neelabh Tripathi is a seasoned IT professional with over 18 years of expertise in cloud computing, enterprise architecture, and microservices. He has worked with some of the world’s leading organizations, where he played pivotal roles in driving digital transformation and innov…