PinnedPublished inData Engineer ThingsFrom S3 to Schema: Deploying Unity Catalog with Terraform on DatabricksIn this post, I’ll walk you through the process of creating a new catalog in Databricks Unity Catalog, using Databricks on AWS with…Jun 24Jun 24
PinnedPublished inDataDarvishDatabricks Cost Optimization: Practical Tips for Performance and SavingsIn a recent Databricks cost optimization project I led, I achieved significant results: reducing unnecessary compute spend, improving…Mar 28A response icon1Mar 28A response icon1
PinnedPublished inDataDarvishUnit Testing in Data Engineering: Python, PySpark, and GitHub CI WorkflowLearn how to implement unit tests for Python and PySpark, automate testing with CI, and boost data pipeline reliability.Mar 5A response icon5Mar 5A response icon5
Published inData Engineer ThingsSigma + Databricks: Setup, Authentication, and Writeback ExplainedBringing Sigma and Databricks together empowers organizations to unlock governed, self-service analytics at scale. Business users get the…Sep 4Sep 4
Published inData Engineer ThingsBuilding a Data App in Databricks: Automating Invoice Retrieval and PDF DeliveryIn this post, I’ll walk you through how to build a simple data app using Databricks Apps. The goal of the app is to automate the manual…Aug 3A response icon1Aug 3A response icon1
Published inData Engineer ThingsSetting Up a Google Cloud Service Account with JSON Key for AuthenticationLearn how to create a Google Cloud service account, generate a secure JSON key, and manage access permissions for authentication in…Jul 31Jul 31
Published inData Engineer ThingsShhh… Secrets in Databricks: A Hands-On CLI GuideThe Databricks CLI is a powerful tool that lets you manage your Databricks workspace directly from the command line.Jul 9Jul 9
Published inData Engineer ThingsStep-by-Step Guide to Creating a Catalog in Databricks Unity Catalog on AWSIn this post, I’ll walk you through the process of creating a new catalog in Databricks Unity Catalog, using the Databricks on AWS UI as an…Jun 23A response icon2Jun 23A response icon2
Published inData Engineer ThingsHow to Generate a PAT for a Databricks Service Principal (Step-by-Step Guide)Working with service principals in Databricks is essential for secure, automated, and scalable integrations — especially when connecting…May 30A response icon2May 30A response icon2
How I Saved Money, Time, and Stress by Optimizing Databricks the Right WayI recently wrapped up a Databricks cost optimization project where we slashed unnecessary spend, improved workload performance, and freed…Mar 28Mar 28