Maxime Havez

Exciting news for the open data community: Variant, the native data type for semi-structured data, is now ratified in the Apache Parquet™ community — with support across Delta Lake, Apache Iceberg™, and Apache Spark™. Variant brings a unified, open standard to how the lakehouse stores and queries flexible data—making it faster, simpler, and more consistent across formats and engines. https://lnkd.in/gESP97Nm

1 073

23 commentaires

Our colleague Fabrice Deseyn will be speaking at Dataminds Connect on October 8th, 2025 at 14:50. 📢 Session: Securing Data at Scale on Databricks: Beyond the Limits of Secret Scopes 🔐 Secret Scopes are often the go-to for handling sensitive values in Databricks. But when organizations scale, these mechanisms quickly show their limits. Fabrice will dive into: ✔️ Why Secret Scopes fall short for credential storage & encryption key management ✔️ How these limitations impact data access, auditability, lifecycle management & key segregation ✔️ Real-world use cases to highlight enterprise challenges ✔️ What a scalable, production-grade secrets management approach should look like Don’t miss this session if you want to learn how to secure your Databricks environment beyond the basics! 👉 Read more: https://lnkd.in/gPwyd_j #ShapingDigitalTogether #AI #DataMindsConnect #Databricks

45

Monitoring streaming data pipelines can be challenging due to their asynchronous nature. Datadog Data Streams Monitoring gives you an end-to-end view across data streaming technologies like Kafka by automatically mapping interdependencies and surfacing key health metrics. ✅ Detect slowdowns such as consumer lag ✅ Pinpoint bottlenecks and failures ✅ Remediate floods of backed-up messages Learn how unified visibility helps teams avoid downtime and build resilient event-driven applications ➡️ https://bit.ly/47MbFfj

174

2 commentaires

Got a messy data pile that’s driving you nuts? Alex Merced’s new MEAP, Architecting an Apache Iceberg Lakehouse, might help you get it all in order. He breaks down how to build a solid data platform with Iceberg — from storage all the way to governance. If you want to check it out, there’s a 50% off code mercedML through August 13th: https://hubs.la/Q03znKgL0

81

2 commentaires

Smarter Insights, Faster Decisions Snowflake Semantic Views + Cortex Analyst open the door to a new way of working with data — where teams can query in plain language, trust the answers, and access governed, real-time insights all within Snowflake. In our latest #TechnicalTuesday blog, Daksh Trehan explores how this approach standardizes KPIs, streamlines analytics, and empowers everyone to go from question to insight in seconds. 📖 Read here: https://lnkd.in/dTXAz_jS #WNSKipiAI #SemanticModeling #Snowflake #TechnicalTuesday

25

Scale time series forecasting across any DataFrame format! Most time series libraries lock you into a single DataFrame framework, forcing painful migrations when scaling up. Switching frameworks usually means rewriting data loading, preprocessing, and model interfaces with tons of boilerplate code. TimeGPT eliminates DataFrame boundaries with universal compatibility. The method signatures of forecast are the same across all frameworks. Works seamlessly with: pandas, Polars, Dask, Spark, Ray. #TimeSeries #TimeGPT #DataScience

56

4 commentaires

Communications Service Providers are under growing pressure to deploy AI at scale, but success depends on data readiness, trust, and governance. TM Forum Member Databricks brings world-class capabilities in secure data intelligence, unified governance, and democratized AI – exactly what's needed to accelerate this transformation. That’s why we’re delighted to announce that Nevash Pillay, Global Head of the Communications Industry at Databricks has joined TM Forum’s AI & Data Innovation Mission Board, alongside industry leaders from Jio Platforms Limited (JPL), Verizon Global Services, PT. Indosat Tbk, AT&T and Accenture. Led by TM Forum’s Guy Lupo, the AI & Data Innovation Mission's aim is to accelerate the adoption of AI into production for CSPs in every market through the development of foundational concepts, best practice guidance, ontologies and models, Open APIs, and shared code and assets. Welcome aboard Nevash, we’re excited to help shape the future together! Find out more about TM Forum’s AI & Data Innovation Mission here: https://lnkd.in/ehy5TVPw

72

9 commentaires

Bring reliability and trust to your Gen AI apps! AI Observability in Snowflake Cortex AI is now generally available, empowering developers to evaluate and optimize. 👩‍💻 Observability is deeply integrated across Cortex features like Search, Analyst, and Document AI. Discover how Snowflake builds foundational capabilities to manage, improve, and trust your AI apps end-to-end. ❄️ Learn more: 👉 https://bit.ly/3UcrpAv

180

3 commentaires

Data + AI are one system, and teams should monitor them that way. Agent Observability makes it easy. With Agent Observability, teams can confidently deliver reliable, production-grade AI by monitoring data pipelines and agents – including prompts, context, completions, errors, metrics, and everything in between – all in one place. This means teams can: ✅ Tie evaluation data and outputs back to the broader data landscape for better traceability and faster root cause analysis ✅ Use one consolidated tool for better visibility and transparency across data + AI teams ✅ Store telemetry data directly within your data warehouse or lakehouse No more fragmented workflows. No more unreliable AI outputs. Learn more here: https://lnkd.in/e-gZsyTQ #agent #observability #agentobservability #dataquality #AIquality

20

1 commentaire