Kyle Hamlin
01/19/2022, 5:24 PMmake_dagster_repo_from_airflow_dags_path
trying to convert my airflow DAGs to dagster pipelines. Getting the following import exception, seem like maybe the dagster-airflow only supports an older version of airflow. Either way looks to be broken, any advice to resolve?
Stack Trace:
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/grpc/server.py", line 216, in __init__
self._repository_symbols_and_code_pointers.load()
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/grpc/server.py", line 92, in load
self._loadable_repository_symbols = load_loadable_repository_symbols(
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/grpc/server.py", line 110, in load_loadable_repository_symbols
loadable_targets = get_loadable_targets(
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/grpc/utils.py", line 27, in get_loadable_targets
else loadable_targets_from_python_file(python_file, working_directory)
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/core/workspace/autodiscovery.py", line 17, in loadable_targets_from_python_file
loaded_module = load_python_file(python_file, working_directory)
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 93, in load_python_file
raise DagsterImportError(
The above exception was caused by the following exception:
ImportError: cannot import name 'BaseOperator' from 'airflow.operators' (/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/airflow/operators/__init__.py)
Stack Trace:
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 79, in load_python_file
return import_module_from_path(module_name, python_file)
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster/seven/__init__.py", line 50, in import_module_from_path
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 783, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "data_eng/jobs/make_dagster_pipeline.py", line 1, in <module>
from dagster_airflow.dagster_pipeline_factory import make_dagster_repo_from_airflow_dags_path
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster_airflow/__init__.py", line 10, in <module>
from .factory import make_airflow_dag, make_airflow_dag_containerized, make_airflow_dag_for_operator
File "/Users/khamlin/.pyenv/versions/3.8.8/lib/python3.8/site-packages/dagster_airflow/factory.py", line 7, in <module>
from airflow.operators import BaseOperator