What is AI Query in Databricks? A Simple Explanation

Enterprise AI is moving past the hype and into practical deployment. At Snowflake, we're building the foundational tools that make AI a reality for everyday workflows. I sat down with Vaishnavi D. at The Economic Times to explore the strategies and challenges of this shift in AI. We discuss the critical role of data context, human oversight, and explainability in building scalable and trustworthy AI solutions.  Check out the article below to learn more about how approaches like open-source innovation and inference optimization are delivering measurable business value for AI today. https://lnkd.in/gYn-6Baj

🚀 𝐇𝐚𝐫𝐧𝐞𝐬𝐬𝐢𝐧𝐠 𝐋𝐢𝐪𝐮𝐢𝐝 𝐂𝐥𝐮𝐬𝐭𝐞𝐫𝐢𝐧𝐠 & 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐞𝐥𝐞𝐭𝐢𝐨𝐧 𝐢𝐧 𝐃𝐚𝐭𝐚𝐛𝐫𝐢𝐜𝐤𝐬 𝐟𝐨𝐫 𝐒𝐦𝐚𝐫𝐭𝐞𝐫 𝐃𝐚𝐭𝐚 𝐌𝐚𝐧𝐚𝐠𝐞𝐦𝐞𝐧𝐭🚀 In the fast-paced world of big data, leveraging advanced techniques like Liquid Clustering and Vector Deletion within platforms like Databricks can significantly enhance your data workflows and machine learning models. ✨ 𝐖𝐡𝐚𝐭 𝐢𝐬 𝐋𝐢𝐪𝐮𝐢𝐝 𝐂𝐥𝐮𝐬𝐭𝐞𝐫𝐢𝐧𝐠 𝐢𝐧 𝐃𝐚𝐭𝐚𝐛𝐫𝐢𝐜𝐤𝐬? A flexible, real-time clustering approach that adapts as new data streams into your Databricks environment. It allows clusters to evolve dynamically, reducing the need for costly re-clustering and enabling more accurate, up-to-date insights. Example: Continuously evolving customer segmentation models in Databricks as new user data arrives, ensuring personalized marketing strategies stay fresh. ✨ 𝐖𝐡𝐚𝐭 𝐢𝐬 𝐕𝐞𝐜𝐭𝐨𝐫 𝐃𝐞𝐥𝐞𝐭𝐢𝐨𝐧 𝐢𝐧 𝐃𝐚𝐭𝐚𝐛𝐫𝐢𝐜𝐤𝐬? The process of selectively removing certain vectors (features or data points) from your dataset or model within Databricks, streamlining your models for better performance and interpretability. Example: Pruning irrelevant feature vectors in a large-scale NLP model to improve inference speed and accuracy. 🔑 𝐖𝐡𝐲 𝐢𝐬 𝐭𝐡𝐢𝐬 𝐢𝐦𝐩𝐨𝐫𝐭𝐚𝐧𝐭? Integrating these techniques in Databricks allows data teams to build adaptable, efficient, and high-performing machine learning pipelines—unlocking faster insights and better business outcomes. Let’s innovate with Databricks and these advanced data techniques! 💡 #Databricks #DataScience #MachineLearning #BigData #AI #DataEngineering #TechInnovation

Performing sentiment analysis directly within Snowflake using our new AI SQL functions is easy. Analyze unstructured text data, such as customer feedback, to classify sentiments as positive, negative, or neutral without external tools. Discover how functions like SNOWFLAKE.CORTEX.SENTIMENT provide numerical sentiment scores and AI_SENTIMENT offer categorical labels. Enrich your datasets and visualize results in tools like Streamlit or Power BI for enhanced business insights. https://lnkd.in/ee8Kcp6C #snowflake #cortex #sentiment #aisql

Big data and AI are only as valuable as the insights people can see. Today’s Tableau turns complex pipelines into clear, shared understanding—combining AI, governance, and speed so everyone can act. The latest capabilities meet teams where they work. Pulse and natural-language data stories surface trends and explain drivers in plain English. Live queries to Snowflake and BigQuery, plus the Hyper engine, accelerate big data at scale. Accelerators jump-start dashboards, while governance—catalog, lineage, and virtual connections—keeps self-service trusted. And embedded analytics and Slack alerts deliver insights in the flow of work. Leading a data program or building dashboards? This guide shows what’s new in Tableau, where AI fits, and how to turn big data into action. Read more: https://lnkd.in/eWBX5wgJ #Tableau #DataVisualization #Analytics #AI #BigData #DataGovernance

Why wait for data scientists to answer every predictive question? SQL users can do AI too! Too many teams are stuck in data science backlogs, waiting weeks for answers that could drive business forward. The challenge: empowering analysts to deliver predictive insights without bottlenecks. In this session, Jarry Chen will show how Snowflake’s built-in ML functions and AISQL make advanced analytics accessible to anyone who knows SQL. At Data Saturday Melbourne discover how your team can become a predictive powerhouse and free up your data scientists for the next big challenge. Speaker: Jarry Chen, expert in AI/ML and Snowflake Register free: https://lnkd.in/gVNJjd7W #DataSaturday #Snowflake #AI #MachineLearning #DataAnalytics #Melbourne

🔍 Databricks + AI + Analytics = The New Era of Data 🚀 In today’s data-first world, businesses are demanding one unified platform that can handle analytics, AI, and data management. That’s where Databricks and the Lakehouse paradigm come in. In our new blog, we explore: ✅ What a “Lakehouse” is and why it matters ✅ How Databricks integrates analytics and AI workflows ✅ Real-world benefits: efficiency, insight, and agility 👉 Dive into the full story: https://lnkd.in/gTRnFSGQ I’d love to hear how your team is handling your data stack—what tools are driving your analytics & AI push? #DataEngineering #AI #Analytics #Lakehouse #Databricks #ExilonTechnology

Conceptual illustration of Databricks Lakehouse architecture showing integration of data lakes and data warehouses into one platform that powers analytics and AI #DataEngineering #Databricks #AI #Analytics #ExilonTechnology

"But AI isn't useful for Data" is a thing I hear a lot. I get it, I've had that experience. Ask it to build a web page? Magic. Ask it to clean up messy data? Suddenly I'm doing the heavy lifting. Last week I threw survey data at Claude Code. Its first instinct? Jump straight to counting—technically correct, completely useless. It generated metrics without context, like a chef cooking without checking what's in the fridge. What it didn't ask: - What story are you trying to tell? - What's hiding in those open-ended responses? - How does this connect to your actual business problem? When it came to the survey data, the speed improvement from AI was still there, but more than time saved, my analysis was better by thoughtfully using the LLM to pay attention to every response and take notes. What I achieved was better than keywords and sentiment analysis, even if I didn’t save a huge amount of time. To get there, I had to bring the strategy. I had to be the detective. My challenge: Next time you hand data to AI, don't start with "analyze this." Start with "help me understand this" and begin to break down the task. Full blog post: https://lnkd.in/gXAMCbtZ #AI #DataAnalysis #DataScience #ArtificialIntelligence #BusinessIntelligence #Analytics

Everyone's watching OpenAI's $100M Databricks deal, but nobody's talking about the real story: How to get 𝗳𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗔𝗜 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗮𝘁 𝟭/𝟵𝟬𝘁𝗵 𝘁𝗵𝗲 𝗰𝗼𝘀𝘁. Here's what just changed the game for enterprise AI 👇 𝗧𝗛𝗘 𝗛𝗘𝗔𝗗𝗟𝗜𝗡𝗘: Databricks drops $100M to make GPT-5 natively available (yes, callable from SQL). But that's not the disruption... 𝗧𝗛𝗘 𝗗𝗜𝗦𝗥𝗨𝗣𝗧𝗜𝗢𝗡: Their new GEPA (Generative Evolutionary Prompt Adaptation) technique makes AI critique and rewrite its own prompts until performance soars. Results? → 4-7 point improvements across finance, legal, and healthcare tasks → Matching and often exceeding fine-tuned models WITHOUT the training cost → 20% immediate cost savings on serving 𝗧𝗛𝗘 𝗠𝗔𝗧𝗛 𝗧𝗛𝗔𝗧 𝗠𝗔𝗧𝗧𝗘𝗥𝗦: At 100k requests, an optimized open source model delivers premium quality at 𝟵𝟬× 𝗹𝗼𝘄𝗲𝗿 𝗰𝗼𝘀𝘁 than frontier models. <- 𝗥𝗲𝗮𝗱 𝘁𝗵𝗮𝘁 𝗮𝗴𝗮𝗶𝗻. 𝗪𝗛𝗔𝗧 𝗧𝗛𝗜𝗦 𝗠𝗘𝗔𝗡𝗦 𝗙𝗢𝗥 𝗬𝗢𝗨: With GPT-5, Claude, and Gemini all native on one platform, you can now: • A/B test prompts across ALL models • Find the cheapest model that hits your metrics • Scale without bleeding budget 𝗬𝗢𝗨𝗥 𝟯-𝗦𝗧𝗘𝗣 𝗣𝗟𝗔𝗬𝗕𝗢𝗢𝗞: 1️⃣ Build evals first (measure everything) 2️⃣ Run GEPA optimization on your top workloads 3️⃣ Default to small/OSS models and only escalate when metrics demand it The new reality: 𝗙𝗿𝗼𝗻𝘁𝗶𝗲𝗿 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗶𝘀 𝗮 𝗽𝗿𝗼𝗰𝗲𝘀𝘀, 𝗻𝗼𝘁 𝗮 𝗽𝘂𝗿𝗰𝗵𝗮𝘀𝗲. #AI #LLM #CostOptimization   Booz Allen Hamilton

Bringing AI to Data Solutions: Snowflake Generative AI Professional Certified! As a Data Solutions Engineer, I’ve always been the bridge between data and business, answering queries and delivering insights. One recurring challenge has been the turnaround time—even small queries often required code changes, testing, and moving updates to production. This is exactly where AI can transform the way we work. This course was a hands-on deep dive into how AI can enhance data solutions engineering, especially through Text-to-SQL generative apps, which allow natural language querying of structured data. Key learnings from the program include: Building applications for AI tasks like summarization, translation, sentiment analysis, and text classification Performing prompt engineering and inference with foundation model families like Llama, Mistral, and Anthropic Fine-tuning foundation models for desired behaviors or distilling capabilities from larger models Asking questions of structured data using natural language (Text-to-SQL) Building and evaluating RAG applications to extract insights from unstructured data Through lab exercises and applied projects, I gained practical experience in model fine-tuning, batch analysis of unstructured text, text classification, and implementing retrieval-augmented generation applications. This is just the beginning of how AI can revolutionize data solutions engineering, making insights faster, more accessible, and actionable. Excited to bring these skills into real-world projects and continue exploring the future of AI in data! #GenerativeAI #Snowflake #TextToSQL #DataEngineering #Analytics #AIinData #RAG #ProfessionalCertificate