Sergey Sokolov

What is Function calling & MCP for LLMs? (explained with visuals and code) Before MCPs became popular, AI workflows relied on traditional Function Calling for tool access. Now, MCP is standardizing it for Agents/LLMs. The visual below explains how Function Calling and MCP work under the hood. Today, let's learn: - Function calling by building custom tools for Agents. - How MCPs help by building a local MCP client with mcp-use and using tools from Browserbase MCP server. In Function Calling: - The LLM receives a prompt. - The LLM decides the tool. - The programmer implements a procedure to accept a tool call request from the LLM and prepare a function call. The tool call request is found in the LLM's response when you prompt it. - A backend service executes the tool. This Function Calling takes place within our stack: - We host the tool. - We implement a logic to determine the tool to invoke and its parameters. - We execute it. So Function Calling requires us to wire everything manually. MCP simplifies this! Instead of hard-wiring tools, MCP: - Standardizes defining, hosting, and exposing tools. - Makes it easy to discover tools, understand schemas, and use them. - Demands approval before invoking them. - Detaches implementation from consumption. For instance, whenever you integrate an MCP server, you never write a line of Python code to integrate the tools. Instead, you just integrate the MCP server and everything beyond this follows a standard protocol handled by the MCP client and the LLM: - They identify the MCP tool. - They prepare the input argument. - They invoke the tool. - They use the tool’s output to generate a response. Everything happens through a standard (but abstracted) protocol. So here’s the key point: MCP and Function Calling are not in conflict. They’re two sides of the same workflow. - Function Calling helps an LLM decide what it wants to do. - MCP ensures that tools are reliably available, discoverable, and executable, without you needing to custom-integrate everything. For example, an agent might say, “I need to search the web,” using function calling. That request can be routed through MCP to select from available web search tools, invoke the correct one, and return the result. Check the workflow in the diagram below. In this setup, to build a local MCP client, I used mcp-use because it lets us connect any LLM to MCP servers & build private MCP clients, unlike Claude/Cursor. - Compatible with Ollama & LangChain - Stream Agent output async - Built-in debugging mode, etc Find the mcp-use GitHub repo in the comments! ____ Find me → Avi Chawla Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

269

11 Comments

Pretty exciting new work from Kuaishou in the Generative Recommender domain. They replaced the entire video recommendation funnel (retrieval - ranking - value model) with a single encoder/decoder architecture trained on user action sequences with next-item prediction task, similar to how LLMs are pre-trained to predict the next token. The key to make this all work is their tokenizer, which is trained to map video content and metadata to semantic item IDs — so that similar videos share similar IDs, making generation feasible even at billion-scale vocabularies. On top of that, they apply reinforcement learning with user feedback to break through the ceiling of the traditional recommender stack. Based on their A/B tests this resulted in +0.5% App Stay time in the Kuaishou app. This finding shows just how much headroom there is left on top of traditional recommender systems! More about this in my blog: https://lnkd.in/gCr6k-MD

148

2 Comments

Did you know that in Foundry there is an end-to-end classical ML Ops toolchain? It spans from training in your favorite environment (hosted Jupyter, RStudio, VSCode), to experiments/evals, change management, release management, and managed deployment (batch and live inference). These can then be registered as functions (source of logic) into the Ontology. The models don’t have to be trained or even hosted in the platform. One modeling objective can have heterogeneous model architectures to test what works best, can compare against third-party models, or just import straight from HF. You can package up complex model runtimes into a Docker container and register and run that in the platform. The end goal is to bring your business logic into the Ontology. This is then available for AIP agentic flows to also utilize for Autonomy. Agents call models, then reason and plan out the output, and take action. The best of both worlds: deterministic models optimized for business functions paired with the reasoning power of LLMs, all securely orchestrated through AIP and Foundry.

189

14 Comments

We're introducing Stax, an experimental developer tool to streamline LLM evaluation with custom and pre-built autoraters. Stax is an AI evaluation platform that allows you to assess and benchmark generative AI models and applications against your specific needs and criteria. In contrast to general leaderboards, Stax helps you understand how an LLM or prompt will perform for your use cases. Testing AI is challenging due to the absence of a single "right" answer and it's non-deterministic nature. Stax provides a platform to measure the quality of AI-generated content, ensuring it aligns with your unique standards and criteria. With Stax, you can: ⚡ Experiment Rapidly: Use the "Quick Compare" feature to test different prompts across various models. 🌐 Evaluate at Scale: Utilize "Projects & Datasets" to run extensive evaluations on large sets of data. ✅ Define Your Own Criteria: Create custom evaluators tailored to your specific use case or use pre-built system evaluators for common criteria like fluency, groundedness, and safety. 📈 Analyze Performance: Visualize evaluation results and performance metrics on the Analytics page to gain insights into model behavior. Go try it now: https://lnkd.in/e_YvmrHC

54

2 Comments

I’ve been focused on LLMs as retrieval and recommendation engines over the past year 🤖. Back in June, I had the chance to share some of my learnings at MLTO: Machine Learning Toronto in a talk called “Rethinking Retrieval with LLMs” — and now I’ve expanded those ideas into a two-part introductory blog post ✍️. 🔎 Part 1: walks through what retrieval is, gives a short review of traditional methods, and shows how LLMs are being used for retrieval and recommendation today (with examples from recent research). 💻 Part 2: is more hands-on: I show how to use vLLM to build a toy retrieval engine with an open-weight model, and I’ve shared the code so you can try it yourself. This is such an exciting area 🚀, and I wanted to provide a friendly introduction so more people can get familiar with the ideas. I’d love to hear your thoughts, and if you know someone who might find these blog posts useful, please feel free to share it with them! Part 1️⃣: https://lnkd.in/gYxgcPEV Part 2️⃣: https://lnkd.in/ge9iS3pE #recommendation_systems #ai #llms #vllm #retireval_engines #search_engine #genai #openai #claude #oracle #gemini #llama #machine_learning #perplexity

355

11 Comments

🚀 New blog post: How we're using multiple LLMs (Gemini, Claude, ChatGPT & more) to revolutionize software performance optimization at Fastcode ✨ Feeding academic papers + source code into AI models to uncover optimization opportunities that would take months to find manually 📈 Results: ~67% of AI recommendations lead to measurable performance gains 🔗 https://bit.ly/45WGw6u #AI #PerformanceEngineering #LLMs #SoftwareOptimization #Fastcode

65

3 Comments

🔎 New Survey on Retrieval And Structuring Augmented Generation (RAS) with LLMs Large Language Models are powerful, but they still struggle with hallucinations, outdated knowledge, and limited domain expertise. This new survey from KDD’25 takes a deep dive into how Retrieval And Structuring (RAS) Augmented Generation can address those gaps. The paper explores: - Retrieval methods (sparse, dense, hybrid) - Structuring techniques (taxonomy, hierarchical classification, information extraction) - Integration with LLMs (prompt-based, reasoning frameworks, embeddings) - Open challenges (retrieval efficiency, structure quality, knowledge integration) - Future directions (multimodal retrieval, cross-lingual structures, interactive systems) If you’re working on RAG, knowledge graphs, or structured reasoning with LLMs, this survey is a must-read.

58

1 Comment

The Catalog team at Instacart just shared a behind-the-scenes look at the multi-modal attribute extraction platform that powers Instacart’s structured product data at scale. This system blends visual and textual signals, including LLM-powered understanding, to create product attributes. This is essential for helping customers find exactly what they need in a vast, ever-changing grocery catalog. It’s a great example of how thoughtful modeling choices and real-world constraints shape the ML systems we build to serve millions of households. Read the full breakdown from the team here: https://lnkd.in/g2WH5XuT

147

5 Comments

The complexity of PPO leads practitioners to avoid online RL in favor of RL-free or offline algorithms (e.g., DPO), but why not just use simpler versions of online RL? TL;DR: REINFORCE and RLOO have been shown to work well for training LLMs. And, they do not require a value model, which drastically reduces cost and complexity. Therefore, instead of completely avoiding online RL due to the overhead associated with PPO, we can just use algorithms that provide the benefits of online RL without the unnecessary complexity. Policy gradient: RL optimizers operate by computing the gradient of the RL objective (maximizing rewards) w.r.t. policy parameters—this is called the policy gradient—and performing gradient ascent. A basic policy gradient is easy to compute (see figure) but has high variance. So, we introduce some extra complexity to get stable gradients / training. Actor-critic: We can reduce variance by incorporating the advantage function (instead of the raw reward for a trajectory) into the policy gradient estimate. When we do this, the value function acts as a normalizer / baseline to the reward estimate for a state. PPO estimates advantage with an additional copy of the LLM—called a value model or critic—that is trained via an MSE loss to predict the final reward for every token in a sequence. The critic is on-policy (i.e., it depends on the current parameters of the policy during training) and is trained alongside the policy itself during RL. This is called an actor-critic framework. REINFORCE: Due to keeping and extra copy of the LLM in memory that is trained, actor-critic frameworks come with additional cost and complexity. As an alternative, we can use the much simpler REINFORCE algorithm, which estimates the value function via an average of rewards (either a moving average of all rewards or an average of rewards in a batch) throughout training. Estimating the value in this way is low cost and removes the need for a critic. REINFORCE does yield higher variance relative to actor-critic methods, but recent work has shown that this does not matter in the context of LLMs. Algorithms like PPO were developed in a prior generation of DeepRL research, where neural networks were primarily trained from scratch. In the LLM domain, we are finetuning pretrained models with a strong prior—unstable gradients are much less of a worry. For this reason, even simple online RL algorithms like REINFORCE work well for training LLMs despite their higher variance. RLOO: To reduce variance relative to REINFORCE, we can also use REINFORCE leave one out (RLOO). This algorithm reduces variance by sampling multiple completions for every prompt and estimating the value function as the average of rewards for other completions—excluding the completion for which the value is being estimated—to the same prompt.

217

6 Comments