Salman Mohammed

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

269

11 Comments

Most people evaluate LLMs by just benchmarks. But in production, the real question is- how well do they perform? When you’re running inference at scale, these are the 3 performance metrics that matter most: 1️⃣ Latency How fast does the model respond after receiving a prompt? There are two kinds to care about: → First-token latency: Time to start generating a response → End-to-end latency: Time to generate the full response Latency directly impacts UX for chat, speed for agentic workflows, and runtime cost for batch jobs. Even small delays add up fast at scale. 2️⃣ Context Window How much information can the model remember- both from the prompt and prior turns? This affects long-form summarization, RAG, and agent memory. Models range from: → GPT-3.5 / LLaMA 2: 4k–8k tokens → GPT-4 / Claude 2: 32k–200k tokens → GPT-OSS-120B: 131k tokens Larger context enables richer workflows but comes with tradeoffs: slower inference and higher compute cost. Use compression techniques like attention sink or sliding windows to get more out of your context window. 3️⃣ Throughput How many tokens or requests can the model handle per second? This is key when you’re serving thousands of requests or processing large document batches. Higher throughput = faster completion and lower cost. How to optimize based on your use case: → Real-time chat or tool use → prioritize low latency → Long documents or RAG → prioritize large context window → Agentic workflows → find a balance between latency and context → Async or high-volume processing → prioritize high throughput My 2 cents 🤌 → Choose in-region, lightweight models for lower latency → Use 32k+ context models only when necessary → Mix long-context models with fast first-token latency for agents → Optimize batch size and decoding strategy to maximize throughput Don’t just pick a model based on benchmarks. Pick the right tradeoffs for your workload. 〰️〰️〰️ Follow me (Aishwarya Srinivasan) for more AI insight and subscribe to my Substack to find more in-depth blogs and weekly updates in AI: https://lnkd.in/dpBNr6Jg

358

61 Comments

I’ve been focused on LLMs as retrieval and recommendation engines over the past year 🤖. Back in June, I had the chance to share some of my learnings at MLTO: Machine Learning Toronto in a talk called “Rethinking Retrieval with LLMs” — and now I’ve expanded those ideas into a two-part introductory blog post ✍️. 🔎 Part 1: walks through what retrieval is, gives a short review of traditional methods, and shows how LLMs are being used for retrieval and recommendation today (with examples from recent research). 💻 Part 2: is more hands-on: I show how to use vLLM to build a toy retrieval engine with an open-weight model, and I’ve shared the code so you can try it yourself. This is such an exciting area 🚀, and I wanted to provide a friendly introduction so more people can get familiar with the ideas. I’d love to hear your thoughts, and if you know someone who might find these blog posts useful, please feel free to share it with them! Part 1️⃣: https://lnkd.in/gYxgcPEV Part 2️⃣: https://lnkd.in/ge9iS3pE #recommendation_systems #ai #llms #vllm #retireval_engines #search_engine #genai #openai #claude #oracle #gemini #llama #machine_learning #perplexity

355

11 Comments

Pretty exciting new work from Kuaishou in the Generative Recommender domain. They replaced the entire video recommendation funnel (retrieval - ranking - value model) with a single encoder/decoder architecture trained on user action sequences with next-item prediction task, similar to how LLMs are pre-trained to predict the next token. The key to make this all work is their tokenizer, which is trained to map video content and metadata to semantic item IDs — so that similar videos share similar IDs, making generation feasible even at billion-scale vocabularies. On top of that, they apply reinforcement learning with user feedback to break through the ceiling of the traditional recommender stack. Based on their A/B tests this resulted in +0.5% App Stay time in the Kuaishou app. This finding shows just how much headroom there is left on top of traditional recommender systems! More about this in my blog: https://lnkd.in/gCr6k-MD

148

2 Comments

There’s a new RL algorithm in town… and it’s already shaking up the LLM research community. 🤯 Group Sequence Policy Optimization (GSPO) was created by the research team at Qwen and is purpose built for operating at the sequence level rather than the token level. This means it evaluates, clips, and optimizes entire response sequences by their overall likelihood. The important upshot is that it directly aligns the optimization process with how rewards are actually assigned in real world scenarios! Some key reasons to be excited! 1. Better stability. GSPO improves training stability and helps avoids model collapse that can happen with algorithms like GRPO. 2. Scalable RL for mixture of experts (MoE) models. GSPO inherently solves stability challenges teams face when scaling RL for MoE architectures. Large scale RL becomes more practically feasible! 3. Simplifies Infra. Sequence-level optimization is inherently less sensitive to small numerical differences between training and inference systems compared to token-level methods, which should lead to more robust RL pipelines! We can already see signs that this has paid off in big ways for the open-source community. Qwen3 models are doing really well on benchmarks, boasting competitive performance with top models like Gemini-2.5 Pro on GPQA, AIME25, and LiveCodeBench! https://lnkd.in/gCZMQvUj P.S. 👀 If you want to run RL on your own Qwen3 model with GRPO/GSPO, keep an eye on our open-source post-building repo 🧡 : https://lnkd.in/gXZ32qkm #AI #LLM #ReinforcementLearning

82

10 Comments

You're in a ML Inference Engineer interview at NVIDIA, and the interviewer asks: "Our team wants to switch from API to a self-hosted. Which serving framework and why?" Here's how you answer: Don't say: "vLLM is fastest" or "Ollama is easiest." Wrong framing. The real answer isn't about features - it's about matching serving philosophy to your constraints. Local prototype vs. production scale vs. complex workflows = completely different frameworks. NOTE - if you want to read this kinda content daily, consider subscribing my free newsletter - https://lnkd.in/g8ZJGsWj Most teams pick based on benchmarks, not bottlenecks Your serving choice matters less than your use case. PagedAttention isn't magic - it's just better memory management "Fastest" framework can be slowest for YOUR workload The infrastructure reality everyone misses: Single user prototype = Ollama/LLaMA.cpp wins Multi-user production = vLLM dominates AI agent workflows = SGLang is your only option Edge deployment = LLaMA.cpp or bust Architecture drives the decision, not features. "But what about performance?" Interviewer: "How do you handle 1000 concurrent users?" Performance without context is meaningless. > vLLM's PagedAttention gives you 24x throughput, but only if memory fragmentation was your bottleneck. > SGLang's RadixAttention only helps with cache reuse scenarios. By the way, just reading about inference frameworks won't help you land a job. I personally learnt a lot by doing stuff. If I were to do all over, I would rent a gpu from Hyperbolic for dirt cheap, and play with some open source model with different frameworks. The business framework that matters: > Rapid prototyping + Local dev = Ollama > Known high load + GPU budget = vLLM > Complex multi-step LLM workflows = SGLang > Constrained environments + Portability = LLaMA.cpp Match the tool to the business constraint. The evolution path most teams miss: > Start: Ollama (prototype fast) > Scale: vLLM (handle users) > Optimize: SGLang (complex workflows) > Edge: LLaMA.cpp (deploy anywhere) It's not replacement - it's graduation based on needs. The answer that gets you hired: "Framework choice isn't about better vs worse. It's about constraints vs capabilities. Ollama gives you speed-to-market. vLLM gives you user scale. SGLang gives you workflow complexity. Pick based on your biggest constraint, not your biggest benchmark." That's it for today. See ya tomorrow! And if you want to read more about these frameworks, checkout this blog: https://lnkd.in/eiH2sphA #llm #machinelearning #gpu #chatgpt #ai #agents

398

17 Comments

🚀 New blog post: How we're using multiple LLMs (Gemini, Claude, ChatGPT & more) to revolutionize software performance optimization at Fastcode ✨ Feeding academic papers + source code into AI models to uncover optimization opportunities that would take months to find manually 📈 Results: ~67% of AI recommendations lead to measurable performance gains 🔗 https://bit.ly/45WGw6u #AI #PerformanceEngineering #LLMs #SoftwareOptimization #Fastcode

65

3 Comments