MLflow

Join #MLflow at the upcoming Terminal Bench Meetup at Databricks HQ on November 6! 🎉 Learn how to build high-quality agents faster with MLflow from Danny Chiao and discover the new, exciting features from Terminal Bench 2.0. 👏 Plus, enjoy food, drinks, and great conversations with the community. 🗓 Thursday, Nov 6 🕕 6:00–8:00 PM PST 📍 Databricks, San Francisco 🔗 Secure your spot: https://luma.com/h9kb99vd #MLflow #AIagents #OpenSource

Building agentic applications is complex, but it doesn’t have to be slow. In this blog, Samraj Moorjani shows how to go from idea to measurable results in hours using the Claude Agent SDK with MLflow for observability and evaluation. What you’ll learn: 🔸 Rapid prototyping with the Claude Agent SDK to validate ideas quickly. 🔸 Automatic tracing with MLflow’s @mlflow.anthropic.autolog() for zero-instrumentation visibility into every agent action. 🔸 Objective iteration using mlflow.genai.evaluate() with custom scorers and judges to track quality improvements. 🔗 Dive in: https://lnkd.in/eAGpbs5b #MLflow #agentic #claudeagent #oss #opensource

📣 Join the next 𝗠𝗟𝗳𝗹𝗼𝘄 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝘆 𝗠𝗲𝗲𝘁𝘂𝗽 on November 12 to explore the latest in AI observability and agent tracing — featuring two exciting talks: 🔹 𝗖𝗹𝗮𝘂𝗱𝗲 𝗖𝗼𝗱𝗲 & 𝗔𝗴𝗲𝗻𝘁 𝗦𝗗𝗞 𝗧𝗿𝗮𝗰𝗶𝗻𝗴 (Samraj M.) Accelerate agentic app development with Claude Code and gain instant visibility into every agent step using MLflow’s built-in tracing. Debug, evaluate, and iterate faster with full trace data flowing into MLflow’s GenAI ecosystem. 🔹 𝗢𝗽𝗲𝗻𝗧𝗲𝗹𝗲𝗺𝗲𝘁𝗿𝘆 (𝗢𝗧𝗘𝗟) 𝗦𝘂𝗽𝗽𝗼𝗿𝘁 (Yuki Watanabe) MLflow 3.6 introduces native OTEL trace ingestion via the OTLP protocol—bringing unified observability to AI/ML systems across any language or framework. 🔗 RSVP: https://lnkd.in/e4gHMbTD #MLflow #opensource #AI #GenAI #MLOps #observability

How do you track your datasets in machine learning experiments? At our recent MLflow Community Meetup, Benjamin Wilson shared a deep dive into a major improvement — dataset tracking made simple. With a single unified interface, you can create, merge, and manage datasets directly in the MLflow tracking server. No more juggling multiple function calls or losing visibility into lineage. 🎥 Watch the full video: https://lnkd.in/gnAxi2ur #opensource #oss #linuxfoundation #mlflow

⏰ Don’t forget — MLflow Office Hours are this Wednesday, October 22 at 8AM PT on Zoom! Join the MLflow maintainers and contributors for a live, interactive session. Whether you’re debugging production issues or experimenting with LLMs and GenAI, this is your chance to ask questions directly and get expert feedback. Come for: 🔹 Real-time MLflow troubleshooting and guidance 🔹 Best practices for managing LLM & GenAI experiments 🔹 A look ahead at new MLflow features 🔗 Reserve your spot: https://lnkd.in/ezg-R8tc #opensource #oss #mlflow #genai #llm

Missed last week’s #MLflow Community Meetup? Check out this clip with Benjamin Wilson on 𝗮𝗴𝗲𝗻𝘁𝗶𝗰 𝗷𝘂𝗱𝗴𝗲𝘀! 🙌 “The judge no longer works as an LLM as a judge—it actually works as an agent as a judge.” In this mode, the trace metadata (the trace info object in MLflow) is passed in: the input to the call, the output, and basically the root span ID for that trace. With that metadata—and MLflow’s MCP features recently released—the judge can make tool calls to MLflow to do things like searching spans and querying different aspects of the trace. 🎥 Watch the full video to go deeper: https://lnkd.in/eDzZmd_E Have questions? Bring them to MLflow Office Hours next Wed, Oct 22 🔗 https://lnkd.in/ezg-R8tc #opensource #oss #mlflow #agenticjudges #llm #genai

In MLflow 3.4, the make_judge method introduces a declarative way to create MLflow Scorers, the core abstraction for automated evaluation. With simple instructions, you can build judges that understand your domain’s quality requirements and automatically align with feedback from human experts. This post shows how to: 🔹 Create custom scorers with make_judge using simple declarative instructions. 🔹 Build scorers that act as agents with built-in tools for trace introspection, handling complex evaluation without complicated prompts or complex span parsing logic. 🔹 Automatically align scorers with subject-matter expert preferences to improve scorer accuracy over time. 🔗 Dive in: https://lnkd.in/eHkcBvHN #opensource #oss #mlflow #LLM #genai #llmops

📣 MLflow Office Hours — Wednesday, October 22 at 8AM PT Connect directly with MLflow maintainers and contributors for live Q&A! Bring your production challenges or your latest #LLM and #GenAI experiments—this session is dedicated to hands-on technical discussion and feedback. You'll get: 🔹 Real-time support for both production and experimental MLflow setups 🔹 Expert guidance on LLM & GenAI model management 🔹 Early insight into future releases and new features 🔹 Best practices for tracking ML & LLM experiments Save your spot ➡️ https://lnkd.in/ezg-R8tc #opensource #oss #mlflow #genai #llm

MLflow 3.4 introduces an MCP server that enables conversational analysis of MLflow traces with assistants like Claude. Ask Claude—or any MCP‑compatible tool—to find failed traces, compare successful and unsuccessful runs, and identify patterns in retrieval steps. In this post, Daniel Liden walks through: 🔹 Setting up MLflow tracing with example data (including intentional failures) 🔹 Configuring the MCP server to work with Claude Desktop and Claude Code 🔹 Practical examples of using AI to analyze and debug AI application traces Setup takes about five minutes, and once configured, your trace data becomes something you can have a conversation with. Use AI to debug AI! ✅ 🔗 Read the full post: https://lnkd.in/eKhMt2kE #MLflow #MLOps #GenAI #AI #ClaudeCode

MLflow Ambassador Thor Steen Larsen will share insights on "How to use MLflow for Experiment Tracking and Deployment and how we use it in DSB" at Industrial Machine Learning in Action: From Models to Operations! 📅 Monday, November 3, 2025 ⏰ 09:00 – 13:30 📍IDA Conference Center, Copenhagen 🇩🇰 Explore practical ML deployment strategies and experiment tracking in real-world operations. ⬇️ 🔗 Registration link: https://lnkd.in/eJ4x9hDg #opensource #ml #MLflow #oss