If you’re running Apache Airflow in production, this guide shows how common orchestration patterns map to Lakeflow Jobs, Databricks’ built-in orchestrator. See how control flow, triggers, parameters, and dynamic execution work when orchestration is integrated with the lakehouse,
@databricks
-
coSTAR Automates Agent Testing and Refinement Loop
By
–
With coSTAR, we replaced manual agent iteration with an automated test and refine loop. Agents run against defined scenarios. MLflow captures execution traces, and LLM judges score the results. A coding assistant then updates the agent until it passes the tests. This cuts
-
Serverless Spark Jobs: Simplified Data Processing Without Cluster Management
By
–
Running Scala and Java Spark jobs often requires managing clusters and handling runtime upgrades. Serverless JARs remove that overhead. Jobs start in seconds and run on the latest supported runtime without cluster management. With Databricks Connect, you can build and test from
-
Genie Code: Autonomous AI Partner for Data Pipeline Management
By
–
Genie Code is your autonomous AI partner for data work.
— Databricks (@databricks) 5 avril 2026
✔️ Builds pipelines
✔️ Debugs issues
✔️ Maintains production systems
All while proactively monitoring pipelines and models in the background.
Through Unity Catalog and Lakehouse Federation, Genie Code understands your… pic.twitter.com/CUvtvkyiosGenie Code is your autonomous AI partner for data work. Builds pipelines Debugs issues Maintains production systems All while proactively monitoring pipelines and models in the background. Through Unity Catalog and Lakehouse Federation, Genie Code understands your
-
Databricks Releases New Foundation Models and Platform Updates
By
–
Nick Karpov and Holly Smith walk through some of the latest Databricks features – and how they work together under a single architecture. Our R&D teams have been busy. Recent updates include:
– Three new foundation models in Databricks Foundation Model API support
– Stateless -
Building AI Systems That Improve From Expert Feedback
By
–
What does it take to build an AI system that actually improves with expert feedback? This guide shows how our engineering team built an American football defensive coordinator assistant that answers situational questions using governed play-by-play and roster data, and then gets
-
Lakebase Autoscaling Adjusts Compute Resources Dynamically
By
–
Lakebase autoscaling adjusts compute dynamically based on CPU load, memory usage, and working set size, within developer-defined limits. Scaling happens automatically without database restarts, so connections stay open while resources adjust to demand. Paired with scale to zero,
-
Unity Catalog Discover: Finding, Understanding, and Trusting Data
By
–
As data estates grow, teams struggle to find, understand, and trust the right data.
— Databricks (@databricks) 2 avril 2026
The new Discover experience in Unity Catalog brings business context, trust signals, and access into one place.
With domains, intelligent curation, and governed access, users can discover data,… pic.twitter.com/8vB54yL3iFAs data estates grow, teams struggle to find, understand, and trust the right data. The new Discover experience in Unity Catalog brings business context, trust signals, and access into one place. With domains, intelligent curation, and governed access, users can discover data,
-
Data Warehouse Migration: Beyond Cost and SQL Conversion
By
–
Data warehouse migrations are often slowed down by the wrong assumptions. Focusing only on cost, treating it as SQL code conversion, or migrating all legacy objects can increase cost, extend timelines, and carry forward unnecessary technical debt. The reality is different.
-
DevRel Track: Multi-Day Agentic Data Apps Hackathon
By
–
The DevRel track at Data + AI Summit 2026 is designed for builders. A multi-day hackathon where teams build agentic data apps for social impact using Databricks Apps, Lakebase, and Agent Bricks DevHub Expo with interactive demos, technical showcases, and community