Arenadata Orchestrator
Arenadata Orchestrator (ADO) is a platform for setting up and operating data pipelines in a production environment. The central component of the platform is Apache Airflow, an open source tool used to programmatically create, schedule, and monitor process and task sequences (DAGs).
TOP-10 popular articles
An overview of Airflow concepts (DAG, task, operator) and Airflow architectural components (Web server, Metadata database, Scheduler, Executor, Worker).
Arenadata Orchestrator software requirements for proper cluster installation.
The table with ADO service ports required for successful Arenadata Orchestrator installation.
The article describes the Airflow web interfaces.
The article describes the DBT service — SQL-first transformation tool for building ELT pipelines.
The guide that explains how to configure integration with HashiCorp Vault to store secrets.
The article shows how to create and run your first DAG to process CSV files.
An overview of available performance tuning approaches for Airflow optimization in ADO with examples.
Airflow CLI: DAGs, tasks, connections and other commands.
Overview of existing approaches to delivering the necessary clients to Airflow workers with examples.