Andrei Nita

Big news in the open-source AI world. The launch of gpt-oss by OpenAI — running natively on Databricks — feels like a pivotal moment in the GenAI space. This isn’t just about bigger models. It’s about better control, clearer governance, and real enterprise readiness. gpt-oss sets a new benchmark for what open language models can do — From advanced reasoning to seamless data integration. And when it runs on Databricks, it plays natively with your data, with full lineage and compliance baked in. Try both the 20B and 120B models today in the Mosaic AI Playground: https://lnkd.in/gaSQC3wH This is a big step toward making GenAI more transparent, secure, and customizable for every team — not just tech giants. Let me know what you build with it. Curious to see the ideas this unlocks. #OpenSourceAI #gptOSS #MosaicAI #Databricks #GenAI #EnterpriseAI #BuildersMindset #DataPattern

51

1 Comment

​Large language model (LLM) copilots and self-service dashboards often generate incorrect results because they query raw tables without understanding what the data represents. A semantic layer bridges this gap by translating technical schemas into clear business concepts like "active user" or "net revenue," and ensuring those definitions remain consistent and governed. With a semantic layer, both analysts and AI can ask a question once and get the same answer, no matter the tool or team. In this post, we’ll explore insights from Select Star CEO Shinji Kim’s appearance on the Data Engineering Podcast, covering where semantic layers add value and what happens when they’re missing. https://lnkd.in/gNvKDdKW

123

4 Comments

Production AI agents ≠ set it and forget it. At #DataAISummit, Eric Peter and Niall Turbitt from Databricks dive deep into what it actually takes to monitor and improve AI agent quality in the real world — beyond just latency and error rates. In this talk, they showcase how to apply software development lifecycle best practices (unit tests, QA testing, production telemetry) to GenAI application quality using MLflow 3  — a solution that gives technical teams full visibility into agent performance and quality. What they covered: ✅ Monitoring key operational metrics like cost, errors, and request volume ✅ Using AI judges to evaluate and improve quality (correctness, safety, guidelines) — no ground truth required ✅ Capturing human feedback + running offline evals to improve performance over time ✅ Building dashboards that help you drill into agent behavior by segment, topic, or timeframe This talk is based on a real-world example: Detecting and iteratively fixing quality issues in a customer support agent 🎥 Watch the session here → https://lnkd.in/eR5vQznS #AI #MLflow #GenAI #LLMOps #MLOps #Monitoring #OpenSource

36

1 Comment

What does it take to move agentic AI workflows from idea to enterprise production? At the 6th Annual MLOps World | GenAI Summit (Oct 8–9, Austin), Dr. Hemant Joshi (CTO, FloTorch) will lead "How to Build and Evaluate Agentic AI Workflows with FloTorch", one of many hands-on workshops at this year's gathering. This session dives into the challenges of deploying GenAI agents in real business environments. He'll be demonstrating how to connect any LLM through a single endpoint, add smart routing, and embed governance and observability so workflows can scale reliably. He’ll also cover how to evaluate cost, latency, and accuracy tradeoffs before deployment, giving teams a framework to accelerate adoption while maintaining trust and business value. Curated by AI expert volunteers, MLOps World is a high-impact 2 day experience designed for AI engineers, agent builders, solution architects, infra teams, and full-stack developers. Explore this year's tracks, speaker lineup, and secure your seat for Austin 👇 https://lnkd.in/gkT4yTtJ

54

Last week, we made OpenAI's new models gpt-oss-20B and gpt-oss-120B available natively on Databricks and the early adoption has been incredible! These open-weight models deliver chain-of-thought reasoning and tool use, best-in-class latency and cost efficiency through Mixture-of-Experts architecture, and a massive 131k context window for long documents and RAG. Build custom agents, automate tasks, or run real-time copilots securely next to your data with governance baked in → https://lnkd.in/g7AdqZnX

1,079

7 Comments

𝐀𝐈 𝐰𝐨𝐧’𝐭 𝐦𝐚𝐠𝐢𝐜𝐚𝐥𝐥𝐲 𝐟𝐢𝐱 𝐛𝐫𝐨𝐤𝐞𝐧 𝐝𝐚𝐭𝐚 𝐦𝐨𝐝𝐞𝐥𝐬. But if we get the foundations right, we can supercharge how LLMs help us model, document, and evolve our data ecosystems. Over the years, we’ve gone from Kimball and Inmon to dbt and the Modern Data Stack. But too often, we obsess over tools while neglecting fundamentals. As AI and agent-driven IDEs become mainstream, Alejandro Aboy highlights the real opportunity isn’t replacement, but augmentation. This is where 𝐕𝐢𝐛𝐞 𝐃𝐚𝐭𝐚 𝐌𝐨𝐝𝐞𝐥𝐥𝐢𝐧𝐠 comes in: an evolving vision for blending data modelling discipline with AI-native workflows. 📌 Key Takeaways ✅ Data Modelling still matters → It’s the semantic backbone for engineers, analysts, and governance. ✅ Two modes exist: Constructive (adding features, dependencies, docs) and Destructive (removing models, cleaning up). Both are complex, even before AI. ✅ Challenges for AI/Agents → Context limits, missing catalogs, outdated docs, and technical debt can cripple LLM-driven modelling. ✅ Readiness checklist → Solid foundations + well-documented schemas (Star Schema, OBT, or Normalisation depending on trade-offs). 💡 𝐓𝐡𝐞 𝐕𝐢𝐛𝐞 𝐕𝐢𝐬𝐢𝐨𝐧 Level 1: Extend your AI IDE Level 2: Add MCP Power Level 3: More Tools ⬇️ Access the detailed guide on Modern Data 101's latest substack in collaboration with Alejandro Aboy: https://lnkd.in/dTeEgkew The future of data modelling isn’t about AI replacing us but about AI-augmented modelling that reduces technical debt and accelerates business value. What’s your take: Are we ready for Vibe Data Modelling, or are we still stuck in tool-first thinking? #DataModelling #DataStrategy

57

1 Comment

Databricks is betting big on AI agents and next-gen databases with the launch of Agent Bricks and Lakebase, aiming to bridge the gap between polished demos and production-ready systems. Fresh off hitting a $4 billion revenue run-rate and raising a $1 billion Series K at a $100 billion-plus valuation, the company says these innovations are the missing pieces in scaling enterprise AI. “Most enterprises are flying blind when building agents,” said Nick Eayrs, VP of field engineering for APJ at Databricks, noting the lack of reliable ways to measure quality or balance costs. With Agent Bricks, teams can declare tasks in natural language while the system auto-optimises—generating evaluation suites, applying prompt engineering, and balancing quality with cost. “Agent Bricks delivers both higher quality and lower cost through many breakthroughs from our Mosaic AI research team,” Eayrs added. On the data side, Lakebase challenges decades-old OLTP systems by offering an AI-native operational database built on Postgres and powered by Neon technology. Unlike traditional databases, Lakebase integrates with the lakehouse architecture, enabling seamless autoscaling, real-time access, and governance via Unity Catalog. “Every AI application, agent, recommendation, and automated workflow needs fast, reliable data at the speed and scale of AI agents,” Eayrs emphasised. Together, Agent Bricks and Lakebase form what Databricks calls the “key pillars” of its strategy to unify data, analytics, and AI. As Eayrs summed it up, “At its core, Lakebase is built upon proven open source technologies. Unlike proprietary systems, it avoids vendor lock-in, promotes transparency, and enables community-driven innovation.”

40

1 Comment

🎈 Streamlit 1.47 has landed, and it's time to get creative!  🧑‍🎨 Introducing new theming configuration options to control font weights, chart colors (categorical and sequential), dataframe header background, heading fonts, and link underlines. ↔️ Plus, you now have expanded sizing control for many widgets, with additional width and height parameters. Check out the release notes for all the details 👇https://lnkd.in/gjxTnMKh #Python #OpenSource

259

5 Comments

Sometimes the fastest way forward is to stop trying to go faster. In a new article, Softwire’s Ted While explores how a slow PySpark data transformation highlighted the limits of incremental optimisation. Rather than keep tuning code, Ted stepped back and examined the data itself. By recognising repetition – and linking it to the way the business produced that data – he achieved orders-of-magnitude improvements. He draws on Gauss’s famous summation shortcut to make the point: the biggest gains often come not from “more compute” or micro-optimisations, but from reframing the problem and applying domain knowledge. Technical skill is essential, but true performance comes when engineers understand the business context behind the data. Read the full piece via the link in the bio 👇 #DataEngineering #BigData #PySpark #DataTransformation #DataOptimization #DataProcessing

31

1 Comment

Our cheatsheet on CME matching algorithms is now on our blog! If you're familiar with FIFO matching but fuzzy on top order, pro rata, LMM, or threshold-based algorithms—and which products use them—this is a great way to refresh your memory. Read it at the link below ⬇️ #cme #quanttrading #marketmicrostructure #futures #options

29

2 Comments