AI/ML Infra Meetup | LLM Agents and Implementation Challenges

AI/ML Infra Meetup | LLM Agents and Implementation Challenges

• 1.
  LLM Agents ANDImplementation Challenges Pritish Udgata Director of Products @ Adobe
• 2.
  Topics for today •Agentic AI • CoT vs RAG vs Agentic • Anatomy of an agent • Single Agent with MCP • Multi Agents with A2A • Implementation Challenges and Solutions
• 3.
  What is AgenticAI ? Agentic AI is an AI system that autonomously makes decisions and takes actions to achieve a goal without being told exactly what to do at every step.
• 4.
  Non-agentic vs AgenticAI START END Step 1 Step 2 Non-agentic AI “Developer” defined control path (static) All LLM applications follow some control path Step 1 Step 2 LLM Agentic AI “LLM” defined control path (dynamic) START END
• 5.
  CoT vs RAGvs Agentic Type Reason Plan Retrieve Proactive PROMPT : “Find me the best Italian restaurant nearby and book a table for 2 at 7 PM today” CoT ❌ ❌ ❌ “Bellagio is rated highly and is 2 miles away” RAG ❌ ❌ ❌ “Bellagio, rated 4.8, 2 miles away. Open until 10 PM.” Agentic “I booked a table for 2 at Bellagio at 7 PM. You’ll get a confirmation email shortly.” ● CoT - Chain of Thoughts ● RAG - Retrieval Augmented Generation
• 6.
  Anatomy of anagent In Agent / LLM Out Data Tools/APIs Memory Prompt Define persona, intent, constraint and goals Provide facts, contexts, signals and trends Connect to search, DB, and trigger action Store state, progress, history and patterns
• 7.
  Anatomy of an agent InAgent / LLM Out Data Tools/APIs Memory Prompt Define persona, intent, constraint and goals Provide facts, contexts, signals and trends Connect to search, DB, and trigger action Store state, progress, history and patterns Reason Plan Retrieve Proactive
• 8.
  Agent in anEnterprise Set-up Agent / LLM Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 Prompt Memory Multiple agents (N) must interact with multiple Data Sources/Tools (M). The classic “NxM problem” Example : N = 1, M = 6
• 9.
  MCP solves thisproblem Agent / LLM Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 Prompt Memory Model Context Protocol Standardizes how LLM applications communicate with data sources, tools
• 10.
  Single Agent withMCP Agent / LLM (MCP Host) Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 <MCP Protocol> MCP Server 1 MCP Server 2 MCP Server 3 MCP Client 1 MCP Client 2 MCP Client 3 Prompt Memory Maintain 1:1 connections with servers Expose specific capabilities through the standardized MCP The main application that uses LLM
• 11.
  Multi Agents withA2A Agent A (MCP Host) Data 1 Tools 1 Data 3 Tools 3 Data 2 Tools 2 <MCP Protocol> MCP Server 1 MCP Server 2 MCP Server 3 MCP Client 1 MCP Client 2 MCP Client 3 Agent B (MCP Host) Data 1 Tools 1 Data 2 Tools 2 <MCP Protocol> MCP Server 1 MCP Server 2 MCP Client 1 MCP Client 2 <A2A Protocol> Prompt Memory Prompt Memory A2A enables agent collaboration across frameworks and vendors Capability Discovery (Agent Cards) Secure Collaboration Task & State Management UX Negotiation A B C D
• 12.
  Implementation Challenges andSolutions Area Key Challenge(s) Solution(s) Tools / API Orchestration ● Picking the right tool ● Mapping LLM output to a tool ❏ Describe tools effectively ❏ Tool retrieval and omission ❏ Provide LLM right context (prompt structure) ❏ Schema/format matching (Tool 1 o/p > Too2 i/p) Memory Management ● Contextual memory (short-term) ● Persistent knowledge (long-term) ● Hallucination ❏ Context : In-memory storage (session context) ❏ Persistent : Vector (semantic), Graph (complex relationship), RDBMS (structured), Document (unstructured) ❏ Relevance filtering (READ what’s needed) ❏ Refresh and prune to avoid drift Security and Governance ● Unauthorized actions ● Compliance with org policies ❏ Role based access controls for tools/APIs ❏ Allow/deny list for sensitive actions ❏ Audit logging for decisions and actions Evaluation ● Agent performance ● Improve reason, retrieve, proactive ● Edge case failures ❏ Simulate before prod ❏ KPIs - task completion rate, accuracy, latency ❏ Implement human-in-the-loop (HITL)
• 13.
  Thank you foryour time !