-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[dagster-airlift] Flesh out airlift landing page
- Loading branch information
Showing
1 changed file
with
47 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1 +1,47 @@ | ||
This is the main page. | ||
# Airlift | ||
|
||
Airlift is a toolkit for integrating Dagster and Airflow. | ||
|
||
- Observe Airflow instances from within Dagster | ||
- Accelerate the migration of Airflow DAGs to Dagster assets with opinionated tooling. | ||
|
||
--- | ||
|
||
## Compatibility | ||
|
||
### REST API Availability | ||
|
||
Airlift depends on the availability of Airflow’s REST API. Airflow’s REST API was made stable in its 2.0 release (Dec 2020) and was introduced experimentally in 1.10 in August 2018. Currently Airflow requires the availability of the REST API. | ||
|
||
- **OSS:** Stable as of 2.00 | ||
- **MWAA** | ||
- Note: only available in Airflow 2.4.3 or later on MWAA. | ||
- **Cloud Composer:** No limitations as far as we know. | ||
- **Astronomer:** No limitations as far as we know. | ||
|
||
--- | ||
|
||
## Airflow Migration Tutorial | ||
|
||
In this tutorial, we'll use `dagster-airlift` to migrate an Airflow DAG to Dagster assets. | ||
|
||
By the end of the tutorial, you'll understand how to use `dagster-airlift` to enable a migration process that | ||
|
||
- Can be done task-by-task in any order with minimal coordination | ||
- Has task-by-task rollback to reduce risk | ||
- Retains Airflow DAG structure and execution history during the migration | ||
|
||
[Click here to get started](/integrations/airlift/tutorial). | ||
|
||
## References | ||
|
||
<ArticleList> | ||
<ArticleListItem | ||
title="Airlift API reference" | ||
href="/\_apidocs/libraries/dagster-airlift" | ||
></ArticleListItem> | ||
<ArticleListItem | ||
title="Additional Airlift Functionality" | ||
href="/integrations/airlift/reference" | ||
></ArticleListItem> | ||
</ArticleList> |