Casey Chang

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

269

11 Comments

GPT-OSS introduces an architectural detail that I haven’t seen discussed much: trainable attention sinks. While attention sinks have been used before—particularly in sparse attention mechanisms—they were traditionally implemented by preserving the first few tokens in the sequence during KV-cache pruning. This is based on the observation that removing these tokens causes a measurable drop in model quality. In previous approaches, these “sink” tokens were not trainable parameters—they were simply the original sequence tokens retained for stability. GPT-OSS takes a different approach. The model uses trainable sink parameters that are incorporated only during the attention score computation: - The sink is added to the attention logits. - Softmax is computed over the augmented scores. - The sink is then removed before continuing the attention computation. This is not a common design choice. Sebastian Raschka, PhD, suggests that this might serve to stabilize attention in long-context scenarios (https://lnkd.in/dpHw4dDZ). Whether this still holds in the sliding-window attention used by GPT-OSS remains to be confirmed, but it’s a plausible explanation. You can experiment with GPT-OSS on Cerebras Systems hardware here, achieving up to 3000 tokens/sec throughput: 🔗 https://lnkd.in/dACuEQgs

90

3 Comments

Found an interesting paper today. AI agent LLMs need robust tool evaluation. Current benchmarks struggle with diverse MCP tools, complex parameter reasoning, varied API responses, and accounting for real-world tool success rates. MCPToolBench++ addresses this: A large-scale, multi-domain benchmark for AI agent MCP tool use. It leverages over 4,000 MCP servers from 40+ categories, featuring both single and challenging multi-step questions. Data generation uses an automated pipeline, including tool sampling, query generation with "Code Dictionaries" for specific inputs, and rigorous validation steps. Evaluation uses two key metrics: 1. Abstract Syntax Tree (AST) Score for static call accuracy, and 2. Pass@K Accuracy for actual tool execution success. A critical finding is that AST and Pass@K rankings often diverge. This means a model might correctly infer the tool and parameters (high AST) but fail during real-world execution (low Pass@K) due to factors like inconsistent tool success rates or parameter errors. Root cause analysis reveals common failures like "Parameter Errors," "API Error," and domain-specific issues (e.g., invalid map coordinates). MCPToolBench++ is crucial for developing more reliable AI agents. Arxiv link: https://lnkd.in/g6fEBM9D #AI #LLMs #AIAgents #MCP #Benchmark #ToolUse

89

3 Comments

Having worked on Agentic systems and open source multi agent deployments for last 2.5+ years (Starting with Langchain and moving to Langgraph, Autogen, Google ADK and many others) two common theme that has come across from deploying Agentic solutions and building using Agentic ecosystems in large scale production systems are - Theme 1 -> Agentic AI is shifting from experiment to enterprise backbone—but its ROI hinges on mastering cost containment and correctness at scale. In multi‑agent ecosystems, uncontrolled reasoning calls lead to spiraling spend with unpredictable accuracy (hence its very important to balance the choice of models, fault tolerant modes and accuracy metrics being defined both at Agent, Tool and Orchestrator levels). The winning CIO strategy is layered orchestration: lightweight models for routing, specialized LLMs for high‑value reasoning, and evaluator agents for correctness. This approach converts raw compute consumption into governed, auditable AI “throughput,” reshaping AI from an R&D cost center into a measurable productivity engine. Theme 2- > Standards like Google’s ADK and the Agent‑to‑Agent (A2A) protocol now provide the backbone for this transformation. By enabling a common language for agent communication, context‑sharing, and validation, they allow enterprises to modularize their AI stacks—aligning model choice with both business logic and budget. The value add: reduced infrastructure burn, accelerated time‑to‑market, and AI systems with built‑in accountability. For CIOs, this means a direct path to higher ROI curves where innovation can also take into account the true business value and cost of running these multi-agentic systems in production. #AgentAI #AgentOrchestration #CIOInsights #AIfrastructure #ADK #A2AProtocol

171

12 Comments

Memory is becoming the new infrastructure layer of AI. Every week, new frameworks emerge promising to fix the same fundamental problem: LLMs forget too quickly, and retrieval systems can’t keep up with the complexity of real work. Let’s break down a few of the most influential approaches — and where they still fall short: 🔹 REFRAG (Meta + NUS) – Instead of stuffing long documents directly into context, it compresses retrieval chunks into embeddings, reducing latency and context bloat. Smart for efficiency, but still struggles with knowledge integration: how do you connect ideas across multiple documents, not just compress them? 🔹 KG-RAG (Adobe Research) – Builds structured knowledge graphs out of enterprise docs. Great for reducing hallucination and noise, but graph construction is brittle and costly. Updating ontologies fast enough for dynamic teams remains unsolved. 🔹 Human-Inspired Frameworks (SALM, CAIM, PREMem) – Bring in concepts like episodic, semantic, and procedural memory. These are closer to how humans think, shifting some reasoning into memory itself. But they’re mostly research-grade, not production-ready: integration into daily team workflows is minimal. 🔹 System-Level OS Approaches (MemOS, MemoryOS) – Treat memory like an operating system, with lifecycle management, scheduling, and migration across “short-term,” “mid-term,” and “long-term” layers. Strong architectural vision, but too abstract for non-research teams who just want answers today, not a full OS overhaul. 🔹 Social & Group Memory (Social-RAG) – Retrieves not just facts but social context from group interactions. Crucial for collaboration use cases, but limited to niche scenarios right now. Across all of these, two unsolved challenges keep coming up: Integration vs. Efficiency – Compressing memory helps scale, but often destroys the relational context teams actually need. Lifecycle & Evolution – Few frameworks handle the reality that knowledge is constantly being updated, contradicted, and reshaped. 💡 Why Tanka takes a different path Instead of treating memory as just a retrieval hack, Tanka treats it as the native infrastructure of work. Every message, doc, and decision is captured once, then structured into living memory — automatically. Agents don’t just retrieve snippets; they act on evolving context, update it, and carry it across workflows. Memory isn’t abstracted away into embeddings or graphs nobody sees — it’s visible, searchable, and agent-actionable. For teams drowning in information every day — switching between Slack, Notion, Google Docs, and email — this difference is existential. Most frameworks solve part of the puzzle. Tanka solves the team problem: how do we stop losing context every time someone leaves, a tool changes, or a project shifts? The answer: memory-native collaboration. Because the future of work isn’t just about smarter AI — it’s about AI that never forgets.

62

7 Comments