Releases: apache/airflow
Airflow 3.0.3
What's Changed
📦 PyPI: https://pypi.org/project/apache-airflow/3.0.3/
📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.0.3/
🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.0.3/release_notes.html
🪶 Sources: https://airflow.apache.org/docs/apache-airflow/3.0.3/installation/installing-from-sources.html
🐳 Docker Image: "docker pull apache/airflow:3.0.3"
🚏 Constraints: https://github.com/apache/airflow/tree/constraints-3.0.3
Bug Fixes
- Fix task execution failures with large data by improving internal communication protocol (#51924, #53194)
- Fix reschedule sensors failing after multiple re-queue attempts over long periods (#52706)
- Improve
xcom_pull
to cover different scenarios for mapped tasks (#51568) - Fix connection retrieval failures in triggerer when schema field is used (#52691)
- Add back user impersonation (
run_as_user
) support for task execution (#51780) - Fix DAG version not updating when bundle name changes without DAG structure changes (#51939)
- Add back
exception
to context for task callbacks (#52066) - Fix task log retrieval for retry attempts showing incorrect logs (#51592)
- Fix data interval handling for DAGs created before AIP-39 during serialization (#51913)
- Fix lingering task supervisors when
EOF
is missed (#51180) (#51970) - Persist
EventsTimetable
's description during serialization (#51926) - Delete import error when a dag bundle becomes inactive (#51921)
- Cleanup import errors during DB migration (#51919)
- Fix
EOF
detection of subprocesses in Dag Processor (#51895) - Stop streaming task logs if end of log mark is missing (#51482)
- Allow more empty loops before stopping log streaming (#52624)
- Fix Jinja2 Template deep copy error with
dag.test
(#51673) - Explicitly close log file descriptor in the supervise function (#51654)
- Improve structured logging format and layout (#51567) (#51626)
- Use Connection Hook Names for Dropdown instead of connection IDs (#51613)
- Add back config setting to control exposing stacktrace (#51617)
- Fix task level alias resolution in structure endpoint (#51579)
- Fix backfill creation to include DAG run configuration from form (#51584)
- Fix structure edges in API responses (#51489)
- Make
dag.test
consistent withairflow dags test
CLI command (#51476) - Fix downstream asset attachment at task level in structure endpoint (#51425)
- Fix Task Instance
No Status
Filter (#52154) - UI: Fix backfill creation to respect run backwards setting from form (#52168)
- UI: Set downstream option to default on task instance clear (#52246)
- UI: Enable iframe script execution (#52568)
- UI: Fix DAG tags filter not showing all tags in UI when tags are greater than 50 (#52714)
- UI: Add real-time clock updates to timezone selector (#52414)
- Improve Grid view performance and responsiveness with optimized data loading (#52718,#52822,#52919)
- Fix editing connection with sensitive extra field (#52445)
- Fix archival for cascading deletes by archiving dependent tables first (#51952)
- Fix whitespace handling in DAG owners parsing for multiple owners (#52221)
- Fix SQLite migration from 2.7.0 to 3.0.0 (#51431)
- Fix http exception when ti not found for extra links API (#51465)
- Fix Starting from Trigger when using
MappedOperator
(#52681) - Add ti information to re-queue logs (#49995)
- Task SDK: Fix
AssetEventOperations.get
to usealias_name
when specified (#52324) - Ensure trigger kwargs are properly deserialized during trigger execution (#52721)
- Fixing bad cadwyn migration for upstream map indexes (#52797)
- Run trigger expansion logic only when
start_from_trigger
is True (#52873) - Fix example dag
example_external_task_parent_deferrable.py
imports (#52957) - Fixes pagination in DAG run lists (#52989)
- Fix db downgrade check condition (#53005)
- Fix log viewing for skipped task (#53028,#53101)
- Fixes Grid view refresh after user actions (#53086)
- Fix
no_status
andduration
for grid summaries (#53092) - Fix
ti.log_url
not in Task Context (#50376) - Fix XCom data deserialization when using
XCom.get_all()
method (#53102)
Miscellaneous
- Update
connections_test
CLI to use Connection instead of BaseHook (#51834) (#51917) - Fix table pagination when DAG filtering changes (#51795)
- UI: Move asset events to its own tab (#51655)
- Exclude
libcst
1.8.1 for Python 3.9 (#51609) - UI: Implement navigation on bar click (#50416)
- Reduce unnecessary logging when retrieving connections and variables (#51826)
Doc Only Changes
- Add note about payload size considerations in API docs (#51768)
- Enhance ENV vars and conns visibility docs (#52026)
- Add http-only warning when running behind proxy in documentation (#52699)
- Publish separate docs for Task SDK (#52682)
- Streamline Taskflow examples and link to core tutorial (#52709)
- Refresh Public Interface & align how-to guides for Airflow 3.0+ (#53011)
Full Changelog: 3.0.2...3.0.3
Apache Airflow Helm Chart 1.18.0
Significant Changes
No significant changes.
Improvements
- Allow ConfigMap and Secret references in
apiServer.env
(#51191) - Add custom annotations to JWT Secret (#52166)
- Allow
valuesFrom
ingitSync.env
(#50228)
Bug Fixes
- Fix JWT secret name (#52268)
- Use
api-server
instead ofwebserver
in NOTES.txt for Airflow 3.0+ (#52194) - Change default executor in pod template to support executor parameter in task (#49433)
- Use
merged
to render airflow.cfg and include computed defaults (#51828) - Use
[api] secret_key
for Airflow 3.0+ instead of[webserver] secret_key
(#52269) - Fix for
fernetkey
and add test of its value (#52977)
Doc only changes
Apache Airflow Helm Chart 1.17.0
Significant Changes
Default Airflow image is updated to 3.0.2
(#51594)
The default Airflow image that is used with the Chart is now 3.0.2
, previously it was 2.10.5
.
New Features
Improvements
- Unify k8s labels & add some missing k8s labels (#49522)
Bug Fixes
- Fix missing api server ingress (#49727)
- Replace break function in
pod-launcher-rolebinding
template (#49219) - Add
webserver_config.py
file to api-server (#50108) - Declare missing API server properties (#51012)
- Add missing api server replicas parameter (#50814)
- Fix FAB
enable_proxy_fix
default for Airflow 3 (#50056) - Add the dag processor ServiceAccount to SecurityContextConstraints role binding (#51080)
- Generate JWT secret during HELM install (#49923)
- Always deploy JWT secret (#51799)
- Add missing
workers.kerberosInitContainer
configuration in values (#51405) - Truncate the executor label length (#51817)
- Fix execution_api_server_url when base_url has a subpath (#51454)
Doc only changes
- Bump minimum helm version in docs (#48700)
- Clarify which worker fields apply to Celery and Kubernetes worker pods (#50458)
- Capitalize the term airflow (#49450)
- Add EdgeExecutor to readme (#51017)
- Add 3.X/2.X clarification for CeleryKubernetesExecutor (#49916)
Misc
Airflow 3.0.2
What's Changed
📦 PyPI: https://pypi.org/project/apache-airflow/3.0.2/
📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.0.2/
🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.0.2/release_notes.html
🪶 Sources: https://airflow.apache.org/docs/apache-airflow/3.0.2/installation/installing-from-sources.html
Bug Fixes
- Fix memory leak in dag-processor (#50558)
- Add back invalid inlet and outlet check before running tasks (#50773)
- Implement slice on LazyXComSequence to allow filtering items from a mapped task(#50117)
- Fix execution API server URL handling for relative paths in KE (#51183)
- Add log lookup exception for Empty operator subtypes (#50325)
- Increase the max zoom on the graph view to make it easier to see small dags on big monitor screens (#50772)
- Fix timezone selection and dashboard layout (#50463)
- Creating backfill for a dag is affecting other dags (#50577)
- Fix next asset schedule and dag card UX (#50271)
- Add bundle path to
sys.path
in task runner (#51318) - Add bundle path to
sys.path
in dag processor (#50385) - Prevent CPU spike in task supervisor when heartbeat timeout exceeded (#51023)
- Fix Airflow Connection Form widget error (#51168)
- Add backwards compatibility shim and deprecation warning for EmailOperator (#51004)
- Handle
SIGSEGV
signals during DAG file imports (#51171) - Fix deferred task resumption in
dag.test()
(#51182) - Fix get dags query to not have join explosion (#50984)
- Ensure Logical date is populated correctly in Context vars (#50898)
- Mask variable values in task logs only if the variable key is sensitive (#50775)
- Mask secrets when retrieving variables from secrets backend (#50895)
- Deserialize should work while retrieving variables with secrets backend (#50889)
- Fix XCom deserialization for mapped tasks with custom backend (#50687)
- Support macros defined via plugins in Airflow 3 (#50642)
- Fix Pydantic
ForwardRef
error by reordering discriminated union definitions (#50688) - Adding backwards compatibility shim for
BaseNotifier
(#50340) - Use latest bundle version when clearing / re-running dag (#50040)
- Handle
upstream_mapped_index
when xcom access is needed (#50641) - Remove unnecessary breaking flag in config command (#50781)
- Do not flood worker logs with secrets backend loading logs (#50581)
- Persist table sorting preferences across sessions using local storage (#50720)
- Fixed patch_task_instance API endpoint to support task instance summaries and task groups (#50550)
- Fixed bulk API schemas to improve OpenAPI compatibility and client generation (#50852)
- Fixed variable API endpoints to support keys containing slashes (#50841)
- Restored backward compatibility for the
/run
API endpoint for older Task SDK clients - Fixed dropdown overflow and error text styling in
FlexibleForm
component (#50845) - Corrected DAG tag rendering to display
+1 more
when tags exceed the display limit by one (#50669) - Fix permission check on the ui config endpoint (#50608)
- Fix
default_args
handling in operator.partial()
to preventTypeError
when unused keys are present (#50525) - DAG Processor: Fix index to sort by last parsing duration (#50388)
- UI: Fix border overlap issue in the Events page (#50453)
- Fix
airflow tasks clear
command (#49631) - Restored support for
--local
flag indag list
anddag list-import-errors
CLI commands (#49380) - CLI: Exclude example dags when a bundle is passed (#50401)
- Fix CLI export to handle stdout without file descriptors (#50328)
- Fix
DagProcessor
stats log to show the correct parse duration (#50316) - Fix OpenAPI schema for
get_log
API (#50547) - Remove
logical_date
check when validating inlets and outlets (#51464) - Guard
ti
update state and set task to fail if exception encountered (#51295)
Miscellaneous
- UI: Implement navigation on bar click (#50416)
- UI: Always Show Trends count in Dag Overview (#50183)
- UI: Add basic json check to variable value
- Remove filtering by last dag run state in patch dags endpoint (#51347)
- Ensure that both public and ui dags endpoints map to DagService (#51226)
- Refresh Dag details page on new run (#51173)
- Log fallback to None when no XCom value is found (#51285)
- Move
example_dags
in standard provider toexample_dags
in sources (#51275) - Bring back "standard" example dags to the
airflow-core
package (#51192) - Faster note on grid endpoint (#51247)
- Port
task.test
to Task SDK (#50827) - Port
dag.test
to Task SDK (#50300,#50419) - Port
ti.run
to Task SDK execution path (#50141) - Support running
airflow dags test
from local files (#50420) - Move macros to task SDK
execution_time
module (#50940) - Add a link to the Airflow logo in Nav (#50304)
- UI: Bump minor and patch package json dependencies (#50298)
- Added a direct link to the latest DAG run in the DAG header (#51119,#51148)
- Fetch only the most recent
dagrun
value for list display (#50834) - Move
secret_key
config toapi
section (#50839) - Move various
webserver
configs tofab
provider (#50774,#50269,#50208,#50896) - Make
dag_run
nullable in Details page (#50719) - Rename Operation IDs for task instance endpoints to include map indexes (#49608)
- Update default sort for connections and dags (#50600)
- Raise exception if downgrade can't proceed due to no
ab_user
table (#50343) - Enable JSON serialization for variables created via the bulk API (#51057)
- Always display the backfill option in the UI; enable it only for DAGs with a defined schedule (#50969)
- Optimized DAG header to fetch only the most recent DAG run for improved performance (#50767)
- Add
owner_links
field toDAGDetailsResponse
for enhanced owner metadata in the API (#50557) - UI: Move map index column to be in line with other columns when viewing a summary mapped tasks (#50302)
- Separate configurations for colorized and json logs in Task SDK / Celery Executor (#51082)
- Enhanced task log viewer with virtualized rendering for improved performance on large logs (#50746)
Doc Only Changes
- Add dates for Limited Maintenance & EOL for Airflow 2.x (#50794)
- Add Apache Airflow setup instructions for Apple Silicon (#50179)
- Update recommendation for upgrade path to airflow 3 (#50318)
- Add "disappearing DAGs" section on FAQ doc (#49987)
- Update Airflow 3 migration guide with step about custom operators (#50871) (#50948)
- Use
AssetAlias
for alias in AssetMetadata
example (#50768) - Do not use outdated
schedule_interval
in tutorial dags (#50947) - Add Airflow Version in Page Title (#50358)
- Fix callbacks docs (#50377)
- Updating operator extra links doc (#50197)
- Prune old Airflow versions from release notes (#50860)
- Fix types in config templates reference (#50792)
- Fix wrong import for
PythonOperator
in tutorial dag (#50962) - Better structure of extras documentation (#50495)
Full Changelog: 3.0.1...3.0.2
Airflow 2.11.0
Significant Changes
DeltaTriggerTimetable
for trigger-based scheduling (#47074)
This change introduces DeltaTriggerTimetable, a new built-in timetable that complements the existing suite of
Airflow timetables by supporting delta-based trigger schedules without relying on data intervals.
Airflow currently has two major types of timetables:
- Data interval-based (e.g.,
CronDataIntervalTimetable
,DeltaDataIntervalTimetable
) - Trigger-based (e.g.,
CronTriggerTimetable
)
However, there was no equivalent trigger-based option for delta intervals like timedelta(days=1)
.
As a result, even simple schedules like schedule=timedelta(days=1)
were interpreted through a data interval
lens—adding unnecessary complexity for users who don't care about upstream/downstream data dependencies.
This feature is backported to Airflow 2.11.0 to help users begin transitioning before upgrading to Airflow 3.0.
- In Airflow 2.11,
schedule=timedelta(...)
still defaults toDeltaDataIntervalTimetable
. - A new config option
[scheduler] create_delta_data_intervals
(default:True
) allows opting in toDeltaTriggerTimetable
. - In Airflow 3.0, this config defaults to
False
, meaningDeltaTriggerTimetable
becomes the default for timedelta schedules.
By flipping this config in 2.11, users can preview and adopt the new scheduling behavior in advance — minimizing surprises during upgrade.
Consistent timing metrics across all backends (#39908, #43966)
Previously, Airflow reported timing metrics in milliseconds for StatsD
but in seconds for other backends
such as OpenTelemetry
and Datadog
. This inconsistency made it difficult to interpret or compare
timing metrics across systems.
Airflow 2.11 introduces a new config option:
[metrics] timer_unit_consistency
(default:False
in 2.11,True
and dropped in Airflow 3.0).
When enabled, all timing metrics are consistently reported in milliseconds, regardless of the backend.
This setting has become mandatory and always True
in Airflow 3.0 (the config will be removed), so
enabling it in 2.11 allows users to migrate early and avoid surprises during upgrade.
Ease migration to Airflow 3
This release introduces several changes to help users prepare for upgrading to Airflow 3:
- All models using
execution_date
now also include alogical_date
field. Airflow 3 dropsexecution_date
entirely in favor oflogical_date
(#44283) - Added
airflow config lint
andairflow config update
commands in 2.11 to help audit and migrate configs for Airflow 3.0. (#45736, #50353, #46757)
Python 3.8 support removed
Support for Python 3.8 has been removed, as it has reached end-of-life.
Airflow 2.11 requires Python 3.9, 3.10, 3.11, or 3.12.
New Features
- Introduce
DeltaTriggerTimetable
(#47074) - Backport
airflow config update
andairflow config lint
changes to ease migration to Airflow 3 (#45736, #50353) - Add link to show task in a DAG in DAG Dependencies view (#47721)
- Align timers and timing metrics (ms) across all metrics loggers (#39908, #43966)
Bug Fixes
- Don't resolve path for DAGs folder (#46877)
- Fix
ti.log_url
timestamp format from"%Y-%m-%dT%H:%M:%S%z"
to"%Y-%m-%dT%H:%M:%S.%f%z"
(#50306) - Ensure that the generated
airflow.cfg
contains a randomfernet_key
andsecret_key
(#47755) - Fixed setting
rendered_map_index
via internal api (#49057) - Store rendered_map_index from
TaskInstancePydantic
intoTaskInstance
(#48571) - Allow using
log_url
property onTaskInstancePydantic
(Internal API) (#50560) - Fix Trigger Form with Empty Object Default (#46872)
- Fix
TypeError
when deserializing task withexecution_timeout
set toNone
(#46822) - Always populate mapped tasks (#46790)
- Ensure
check_query_exists
returns a bool (#46707) - UI:
/xcom/list
got exception when applying filter on thevalue
column (#46053) - Allow to set note field via the experimental internal api (#47769)
Miscellaneous
- Add
logical_date
to models usingexecution_date
(#44283) - Drop support for Python 3.8 (#49980, #50015)
- Emit warning for deprecated
BaseOperatorLink.get_link
signature (#46448)
Doc Only Changes
Airflow 3.0.1
📦 PyPI: https://pypi.org/project/apache-airflow/3.0.1/
📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.0.1/index.html
🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.0.1/release_notes.html
🪶 Sources: https://airflow.apache.org/docs/apache-airflow/3.0.1/installation/installing-from-sources.html
Bug Fixes
- Improves the handling of value masking when setting Airflow variables for enhanced security (#43123)
- Make entire task box clickable to select the task (#49299)
- Vertically align task log header components in full screen mode (#49569)
- Remove
dag_code
records with no serialized dag (#49478) - Clear out the
dag_code
andserialized_dag
tables on 3.0 upgrade (#49563) - Remove extra slash so that the runs tab is selected (#49600)
- Null out the
scheduler_interval
field on downgrade (#49583) - Logout functionality should respect
base_url
in api server (#49545) - Fix bug with showing invalid credentials on Login UI (#49556)
- Fix Dag Code text selection when dark mode is enabled (#49649)
- Bugfix:
max_active_tis_per_dag
is not respected by dynamically mapped tasks (#49708) - Fix infinite redirect caused by mistakenly setting token cookie as secure (#49721)
- Better handle safe url redirects in login form for
SimpleAuthManager
(#49697)(#49866) - API: Add missing
bundle_version
to DagRun response (#49726) - Display bundle version in Dag details tab (#49787)
- Fix gcp remote log module import in airflow local settings (#49788)
- Bugfix: Grid view stops loading when there is a pending task to be expanded (#49772)
- Treat single
task_ids
inxcom_pull
the same as multiple when provided as part of a list (#49692) - UI: Auto refresh Home page stats (#49830)
- UI: Error alert overflows out of the alert box (#49880)
- Show backfill banner after creating a new backfill (#49666)
- Mark
DAGModel
stale and associate bundle on import errors to aid migration from 2.10.5 (#49769) - Improve detection and handling of timed out DAG processor processes (#49868)
- Fix editing port for connections (#50002)
- Improve & Fix grid endpoint response time (#49969)
- Update time duration format (#49914)
- Fix Dashboard overflow and handle no status tasks (#49964)
- Fix timezone setting for logical date input on Trigger Run form (#49662)
- Help
pip
with avoiding resolution too deep issues in Python 3.12 (#49853) - Bugfix: backfill dry run does not use same timezone as create backfill (#49911)
- Fix Edit Connection when connection is imported (#49989)
- Bugfix: Filtering items from a mapped task is broken (#50011)
- Fix Dashboard for queued DagRuns (#49961)
- Fix backwards-compat import path for
BashSensor
(#49935) - Apply task group sorting based on webserver config in grid structure response (#49418)
- Render custom
map_index_template
on task completion (#49809) - Fix
ContinuousTimetable
false triggering when last run ends in future (#45175) - Make Trigger Dag form warning more obvious (#49981)
- Restore task hover and selection indicators in the Grid view (#50050)
- Fix datetime validation for backfills (#50116)
- Fix duration charts (#50094)
- Fix DAG node selections (#50095)
- UI: Fix date range field alignment (#50086)
- Add auto-refresh for
Stats
(#50088) - UI: Fixes validation error and adds error indicator for Params form (#50127)
- fix: wrap overflowing texts of asset events (#50173)
- Add audit log extra to table and improve UX (#50100)
- Handle map indexes for Mapped
TaskGroup
(#49996) - Do not use introspection in migration to fix offline SQL generation (#49873)
- Fix operator extra links for mapped tasks (#50238)
- Fix backfill form (#50249)(#50243)
- UI: Fix operator overflow in graph (#50252)
- UI: Pass
mapIndex
to clear the relevant task instances. (#50256) - Fix markdown rendering on dag docs (#50142)
Miscellaneous
- Add
STRAIGHT_JOIN
prefix for MySQL query optimization inget_sorted_triggers
(#46303) - Ensure
sqlalchemy[asyncio]
extra is in core deps (#49452) - Remove unused constant
HANDLER_SUPPORTS_TRIGGERER
(#49370) - Remove sort indicators on XCom table to avoid confusion (#49547)
- Remove
gitpython
as a core dependency (#49537) - Bump
@babel/runtime
from7.26.0
to7.27.0
(#49479) - Add backwards compatibility shim for
get_current_context
(#49630) - AIP-38: enhance layout for
RunBackfillForm
(#49609) - AIP-38: merge Backfill and Trigger Dag Run (#49490)
- Add count to Stats Cards in Dashboard (#49519)
- Add auto-refresh to health section for live updates. (#49645)
- Tweak Execution API OpenAPI spec to improve code Generation (#49700)
- Stricter validation for
backfill_id
(#49691)(#49716) - Add
SimpleAllAdminMiddleware
to allow api usage without auth header in request (#49599) - Bump
react-router
andreact-router-dom
from 7.4.0 to 7.5.2 (#49742) - Remove reference to
root_dag_id
in dagbag and restore logic (#49668) - Fix a few SqlAlchemy deprecation warnings (#49477)
- Make default execution server URL be relative to API Base URL (#49747)(#49782)
- Common
airflow.cfg
files across all containers in defaultdocker-compose.yaml
(#49681) - Add redirects for old operators location to standard provider (#49776)
- Bump packaging from 24.2 to 25.0 in
/airflow-core
(#49512) - Move some non-core dependencies to the
apache-airflow
meta package (#49846) - Add more lower-bind limits to address resolution too deep (#49860)
- UI: Add counts to pool bar (#49894)
- Add type hints for
@task.kuberenetes_cmd
(#46913) - Bump
vite
from5.4.17
to5.4.19
for Airflow UI (#49162)(#50074) - Add
map_index
filter option toGetTICount
andGetTaskStates
(#49818) - Add
stats
ui endpoint (#49985) - Add link to tag to filter dags associated with the tag (#49680)
- Add keyboard shortcut for full screen and wrap in logs. (#50008)
- Update graph node styling to decrease border width on tasks in UI (#50047) (#50073)
- Allow non-string valid JSON values in Variable import. (#49844)
- Bump min versions of crucial providers (#50076)
- Add
state
attribute toRuntimeTaskInstance
for easierti.state
access in Task Context (#50031) - Move SQS message queue to Amazon provider (#50057)
- Execution API: Improve task instance logging with structlog context (#50120)
- Add
dag_run_conf
toRunBackfillForm
(#49763) - Refactor Dashboard to enhance layout (#50026)
- Add the download button on the assets page (#50045)
- Add
dateInterval
validation and error handling (#50072) - Add
Task Instances [{map_index}]
tab to mapped task details (#50085) - Add focus view on grid and graph on second click (#50125)
- Add formatted extra to asset events (#50124)
- Move webserver expose config to api section (#50209)
Doc Only Changes
- Remove flask application configuration from docs for AF3 (#49393)
- Docker compose: airflow-cli to depend on airflow common services (#49318)
- Better upgrade docs about flask/fab plugins in Airflow 3 (#49632)(#49614)(#49628)
- Various Airflow 3.0 Release notes & Updating guide docs updates (#49623)(#49401)(#49654)(#49663)(#49988)(#49954)(#49840)(#50195)(#50264)
- Add using the rest api by referring to
security/api.rst
(#49675) - Add correct redirects for rest api and upgrade docs (#49764)
- Update
max_consecutive_failed_dag_runs
default value to zero in TaskSDK dag (#49795) (#49803) - Fix spacing issues in params example dag (
example_params_ui_tutorial
) (#49905) - Doc: Fix Kubernetes duplicated version in maintenance policy (#50030)
- Fix links to source examples in Airflow docs (#50082)
- Update ruff instructions for migration checks (#50232)
- Fix example of backfill command (#50222)
- Update docs for running behind proxy for Content-Security-Policy (#50236)
Full Changelog: 3.0.0...3.0.1
Airflow 3.0.0
📣 We are proud to announce the General Availability of Apache Airflow® 3.0, the most significant release in the project’s history.
Airflow 3.0 builds on the foundation of Airflow 2 and introduces a new service-oriented architecture, a modern React-based UI, enhanced security, and a host of long-requested features such as DAG versioning, improved backfills, event-driven scheduling, and support for remote execution.
You can read more about what 3.0 brings in https://airflow.apache.org/blog/airflow-three-point-oh-is-here/.
📦 PyPI: https://pypi.org/project/apache-airflow/3.0.0/
📚 Docs: https://airflow.apache.org/docs/apache-airflow/3.0.0
🛠️ Release Notes: https://airflow.apache.org/docs/apache-airflow/3.0.0/release_notes.html
🪶 Sources: https://airflow.apache.org/docs/apache-airflow/3.0.0/installation/installing-from-sources.html
This is the result of 300+ developers within the Airflow community working together tirelessly for many months! A huge thank you to all of them for their contributions.
Resources
Apache Airflow Helm Chart 1.16.0
Significant Changes
Default git-sync image is updated to 4.3.0
(#41411)
The default git-sync image that is used with the Chart is now 4.3.0
, previously it was 4.1.0
.
Default Airflow image is updated to 2.10.5
(#46624)
The default Airflow image that is used with the Chart is now 2.10.5
, previously it was 2.9.3
.
Default PgBouncer image is updated to 1.23.1
(#47416)
The default PgBouncer image that is used with the chart is now airflow-pgbouncer-2025.03.05-1.23.1
, previously it was airflow-pgbouncer-2024.01.19-1.21.0
.
Default PgBouncer Exporter image is updated to v0.18.0
(#47416)
The default PgBouncer Exporter image that is used with the chart is now airflow-pgbouncer-exporter-2025.03.05-0.18.0
, previously it was airflow-pgbouncer-exporter-2024.06.18-0.17.0
.
Default StatsD exporter image is updated to v0.28.0
(#43393)
The default StatsD exporter image that is used with the chart is now v0.28.0
, previously it was v0.26.1
.
New Features
- Allow passing custom env to log groomer sidecar containers (#46003)
- Allow using existing persistence claim in Redis StatefulSet (#41619)
- Add
hostAliases
support in Triggerer (#41725) - Enable HPA for Airflow Webserver (#41955)
- Add env support for database migration job (#42345)
- Support NodePort on Redis Service (#41811)
- Add heartbeat metric for DAG processor (#42398)
- Option to enable ipv6 ipaddress resolve support for StatsD host (#42625)
- Allow customizing
podManagementPolicy
in worker (#42673) - Support multiple executors in chart (#43606, #44424)
- Swap internal RPC server for API server in the helm chart (#44463)
- Add OpenSearch remote logging options (#45082)
- Add
startupProbe
to flower deployment (#45012) - Add PgBouncer and StatsD ingress (#41759)
- Add environment variable controlling the log grooming frequency (#46237)
Improvements
- Update metrics names to allow multiple executors to report metrics (#40778)
- Add a specific internal IP address for the ClusterIP service (#40912)
- Remove scheduler automate ServiceAccount token (#44173)
- More controls for PgBouncer secrets configuration (#45248)
- Add
ti.running
metric export (#47773) - Add optional configuration for
startupProbe
initialDelaySeconds
(#47094) - Introduce
worker.extraPorts
to expose additional ports to worker container (#46679)
Bug Fixes
- Enable
AIRFLOW__CELERY__BROKER_URL_CMD
whenpasswordSecretName
is true (#40270) - Properly implement termination grace period seconds (#41374)
- Add kerberos env to base container env, add webserver-config volume (#41645)
- Fix
volumeClaimTemplates
missingapiVersion
andkind
(#41771) - Render global volumes and volume mounts into cleanup job (#40191) (#42268)
- Fix flower ingress service reference (#41179)
- Fix
volumeClaimTemplate
for scheduler in local and persistent mode (#42946) - Fix role binding for multiple executors (#44424)
- Set container name to
envSourceContainerName
in KEDA ScaledObject (#44963) - Update scheduler-deployment to cope with multiple executors (#46039)
- Replace disallowed characters in metadata label (#46811)
- Grant Airflow API Server Permission to Read Pod Logs (#47212)
- Fix scheduler ServiceAccount auto-mount for multi-executor (#46486)
Doc only changes
- Reflect in docs that
extraInitContainers
is supported for jobs (#41674) - Add guide how to PgBouncer with Kubernetes Secret (#42460)
- Update descriptions private registry params (#43721)
- Change description for kerberos
reinitFrequency
(#45343) - Update Helm eviction configuration guide to reflect
workers.safeToEvict
default value (#44852) - Add info that
storageClassName
can be templated (#45176) - Fix broker-url secret name in production guide (#45863)
- Replace DAGs with dags in docs (#47959)
- Enhance
airflowLocalSettings
value description (#47855) - Be consistent with denoting templated params (#46481)
Misc
- Support templated hostname in NOTES (#41423)
- Default airflow version to 2.10.5 (#46624)
- Changing triggerer config option
default_capacity
tocapacity
(#48032) - AIP-84 Move public api under /api/v2 (#47760)
- Default to the FabAuthManager in the chart (#47976)
- Update PgBouncer to
1.23.1
and PgBouncer exporter to0.18.0
(#47416) - Move api-server to port 8080 (#47310)
- Start the api-server in Airflow 3, webserver in Airflow 2 (#47085)
- Move
fastapi-api
command toapi-server
(#47076) - Move execution_api_server_url config to the core section (#46969)
- Use standalone dag processor for Airflow 3 (#45659)
- Update
quay.io/prometheus/statsd-exporter
fromv0.26.1
tov0.28.0
(#43393)
Apache Airflow 2.10.5
Significant Changes
Ensure teardown tasks are executed when DAG run is set to failed (#45530)
Previously when a DAG run was manually set to "failed" or to "success" state the terminal state was set to all tasks.
But this was a gap for cases when setup- and teardown tasks were defined: If teardown was used to clean-up infrastructure
or other resources, they were also skipped and thus resources could stay allocated.
As of now when setup tasks had been executed before and the DAG is manually set to "failed" or "success" then teardown
tasks are executed. Teardown tasks are skipped if the setup was also skipped.
As a side effect this means if the DAG contains teardown tasks, then the manual marking of DAG as "failed" or "success"
will need to keep the DAG in running state to ensure that teardown tasks will be scheduled. They would not be scheduled
if the DAG is directly set to "failed" or "success".
Bug Fixes
- Prevent using
trigger_rule=TriggerRule.ALWAYS
in a task-generated mapping within bare tasks (#44751) - Fix ShortCircuitOperator mapped tasks (#44912)
- Fix premature evaluation of tasks with certain trigger rules (e.g.
ONE_DONE
) in a mapped task group (#44937) - Fix task_id validation in BaseOperator (#44938) (#44938)
- Allow fetching XCom with forward slash from the API and escape it in the UI (#45134)
- Fix
FileTaskHandler
only read from default executor (#46000) - Fix empty task instance for log (#45702) (#45703)
- Remove
skip_if
andrun_if
decorators before TaskFlow virtualenv tasks are run (#41832) (#45680) - Fix request body for json requests in event log (#45546) (#45560)
- Ensure teardown tasks are executed when DAG run is set to failed (#45530) (#45581)
- Do not update DR on TI update after task execution (#45348)
- Fix object and array DAG params that have a None default (#45313) (#45315)
- Fix endless sensor rescheduling (#45224) (#45250)
- Evaluate None in SQLAlchemy's extended JSON type decorator (#45119) (#45120)
- Allow dynamic tasks to be filtered by
rendered_map_index
(#45109) (#45122) - Handle relative paths when sanitizing URLs (#41995) (#45080)
- Set Autocomplete Off on Login Form (#44929) (#44940)
- Add Webserver parameters
max_form_parts
,max_form_memory_size
(#46243) (#45749) - Fixed accessing thread local variable in BaseOperators
execute
safeguard mechanism (#44646) (#46280) - Add map_index parameter to extra links API (#46337)
Miscellaneous
Apache Airflow 2.10.4
Significant Changes
TaskInstance priority_weight
is capped in 32-bit signed integer ranges (#43611)
Some database engines are limited to 32-bit integer values. As some users reported errors in
weight rolled-over to negative values, we decided to cap the value to the 32-bit integer. Even
if internally in python smaller or larger values to 64 bit are supported, priority_weight
is
capped and only storing values from -2147483648 to 2147483647.
Bug Fixes
- Fix stats of dynamic mapped tasks after automatic retries of failed tasks (#44300)
- Fix wrong display of multi-line messages in the log after filtering (#44457)
- Allow "/" in metrics validator (#42934) (#44515)
- Fix gantt flickering (#44488) (#44517)
- Fix problem with inability to remove fields from Connection form (#40421) (#44442)
- Check pool_slots on partial task import instead of execution (#39724) (#42693)
- Avoid grouping task instance stats by try_number for dynamic mapped tasks (#44300) (#44319)
- Re-queue task when they are stuck in queued (#43520) (#44158)
- Suppress the warnings where we check for sensitive values (#44148) (#44167)
- Fix get_task_instance_try_details to return appropriate schema (#43830) (#44133)
- Log message source details are grouped (#43681) (#44070)
- Fix duplication of Task tries in the UI (#43891) (#43950)
- Add correct mime-type in OpenAPI spec (#43879) (#43901)
- Disable extra links button if link is null or empty (#43844) (#43851)
- Disable XCom list ordering by execution_date (#43680) (#43696)
- Fix venv numpy example which needs to be 1.26 at least to be working in Python 3.12 (#43659)
- Fix Try Selector in Mapped Tasks also on Index 0 (#43590) (#43591)
- Prevent using
trigger_rule="always"
in a dynamic mapped task (#43810) - Prevent using
trigger_rule=TriggerRule.ALWAYS
in a task-generated mapping within bare tasks (#44751)
Doc Only Changes
Miscellaneous
- Raise deprecation warning when accessing inlet or outlet events through str (#43922)