Duhit Choudhary

Stop wasting hours writing DataWeave scripts. AI + DataWeave now makes transformations effortless. Generative Transformation in DataWeave lets you: 🚀 Auto-generate scripts from metadata and sample data 🚀 Streamline mappings and calculations 🚀 Get AI-powered guidance to reduce errors and speed up development 🚀 Get started: https://sforce.co/3JVUHkK

293

2 Comments

Hear three key risks every IT leader must consider when governing AI for their organization. data security, unintended actions from non‑deterministic agents, and attacker manipulation via prompt injection/data poisoning. You’ll get a clear, actionable framework from Box CTO Ben Kus and Senior Product Marketing Manger Meena Ganesh, plus guardrails to make agents operationally safe: secure access (authorization/permissions, least privilege, auditing), restrict dangerous tools, and require human‑in‑the‑loop for high‑risk actions. Watch the full conversation at the link in the comments

50

1 Comment

Snowflake is the all-in-one cloud data platform designed to store, share, and secure your data while boosting performance. Explore this cheatsheet for a quick guide to its top features! - Data Protection: Secure your data with features like Replication (sync data), Time Travel (restore historical data), Fail Safe (disaster recovery), and Cloning (instant, zero-copy data copies). - Data Security: Maintain data privacy with Secure Views (encrypted access), Object Tagging (easy tracking), Column-Level and Row-Level Security (control data access at fine granularity). - Data Sharing: Share data seamlessly via Private Exchange (internal sharing), Data Marketplace (external datasets), Data Exchange, and Data Shares (cross-account sharing). - Apps & Extensibility: Extend Snowflake’s functionality with APIs, Streamlit, Client Connectors, and RESTful APIs for building custom apps. - Ingestion: Load data efficiently using Kafka, Snowpipe, COPY Command, and Snowpipe Streaming for real-time ingestion. - Security Features: Secure your platform with Encryption, MFA, Federated Auth, and Network Policies. - Editions: Choose from Standard, Enterprise, Virtual Private, and Business Critical editions based on security and performance needs. - Transformation: Transform data with UDFs, Stored Procedures, Dynamic Tables, and Streams & Tasks for automation. - Cloud Services: Leverage Snowflake's Query Optimizer, Metadata Manager, Access Control, and Authentication for efficient cloud operations. - Storage: Store data with Internal/External Stage, Micropartitions, and Data Clustering to optimize query performance. - Tools & Interfaces: Use Snowpark, SnowSQL, Snowsight, and VS Code Extension for seamless data interaction and development. - Gen AI & LLMs: Utilize AI-driven features like Copilot, Cortex, and Document AI for advanced insights. - Performance: Boost performance with Caching, Query Acceleration, Search Optimization, and Query Profiles. - Compute: Manage workloads with Auto Scale, Virtual Warehouse, Auto Suspend, and Multi-Cluster Warehouses for optimal compute resource management. This cheatsheet provides a quick overview of Snowflake’s key capabilities for data storage, security, sharing, and performance optimization. credit:-Ashish Joshi follow:-Tech In Nutshell

38

11 Comments

Airflow is the backbone of many data engineering teams, powering pipelines that transform raw data into valuable products. But how do you ensure it scales with your needs? In this episode, Savin Goyal at Outerbounds shares how data engineers use Airflow to build reliable workflows and why orchestration plays a critical role in managing large-scale data operations. We discuss best practices for optimizing Airflow, integrating it with ML workflows and maintaining efficiency as data demands grow. Listen to the conversation at the link in the comments. #AI #Automation #Airflow #MachineLearning

27

1 Comment

🔄Redshift to Databricks Migration: Best Practices When it comes to Redshift vs. Databricks, it’s not about one being “better” than the other - the right choice will always depend on your business needs. But if the decision is made to undertake a Redshift to Databricks migration, it’s critical to understand the process and nuances to achieve success. 👉 So in his latest article, Sebastian Diaz SVP of Data & AI at Qubika, goes into detail about a structured, phased, 6-step approach to what senior data leaders should focus on - starting from migration discovery and assessment, through to the final step of re-establishing connections for all downstream consumers. He also shares a case study of how Qubika helped one of the world’s largest entertainment and gaming leaders successfully transition to Databricks, while at the same time instilling data engineering best practices. https://lnkd.in/eW9ygGi2

21