backfill

Elena Kostyuchenko

Definition

Backfill is a sub-command in the dag command group. It allows you to run subsections of a DAG for a specified date range.

If reset_dag_run option is used, the command will first prompt whether Airflow should clear all the previous DAG runs and task instances within the provided date range.

If rerun_failed_tasks is used, Airflow will re-run the previous failed task instances within the provided date range.

Usage

$ airflow dags backfill [-h] [-c CONF] [--continue-on-failures]
                      [--delay-on-limit DELAY_ON_LIMIT] [--disable-retry] [-x]
                      [-n] [-e END_DATE] [-i] [-I] [-l] [-m] [--pool POOL]
                      [--rerun-failed-tasks] [--reset-dagruns] [-B]
                      [-s START_DATE] [-S SUBDIR] [-t TASK_REGEX]
                      [--treat-dag-as-regex] [--treat-dag-id-as-regex] [-v]
                      [-y]
                      dag_id

Arguments

Parameter Description

dag_id

The ID of the DAG to backfill

-s, --start-date

The start date of the backfill period in YYYY-MM-DD format

-e, --end-date

The end date of the backfill period in YYYY-MM-DD format

-m, --mark-success

Mark jobs as succeeded without running them

-i, --ignore-dependencies

Skip upstream tasks. Run only the tasks matching the regex

-t, --task-regex

Regex pattern to filter specific task IDs to backfill

-I, --ignore-first-depends-on-past

Ignore depends_on_past dependencies for the first set of tasks only

-l, --local

Run the task using the LocalExecutor

-x, --donot-pickle

Do not pickle the DAG object to send to workers. Workers will run their version of the code

-n, --dry-run

Perform a dry run

--pool

Resource pool to use

--rerun-failed-tasks

Re-run previously failed task instances within the date range

--reset-dagruns

Delete existing backfill-related DAG runs and start fresh

-B, --run-backwards

Run tasks from the most recent date first. Incompatible with depends_on_past

--delay-on-limit

Time (in seconds) to wait before retrying when max active DAG runs limit is reached

--disable-retry

Mark failed tasks as failed without retrying

-c, --conf

JSON string to be stored in the DAG run’s conf attribute

-y

Skip confirmation prompt when resetting DAG runs

-S, --subdir

Path to the DAG files directory. Defaults to [AIRFLOW_HOME]/dags

--treat-dag-as-regex

Interpret dag_id argument as a regex pattern to match multiple DAGs

--treat-dag-id-as-regex

Alias for --treat-dag-as-regex

-h, --help

Print out the help manual for this command

-v, --verbose

Make logging output more verbose

Examples

To run a single selected task from the specified DAG, run:

$ airflow dags backfill example_complex -s 2025-04-03 -e 2025-04-04 -t get_entry_group
Found a mistake? Seleсt text and press Ctrl+Enter to report it