Benedikt Buchert
01/07/2023, 5:56 PMStack Trace:
File "/usr/local/lib/python3.7/site-packages/dagster/_daemon/run_coordinator/queued_run_coordinator_daemon.py", line 283, in _dequeue_run_guarded
self._dequeue_run(workspace_process_context, run)
File "/usr/local/lib/python3.7/site-packages/dagster/_daemon/run_coordinator/queued_run_coordinator_daemon.py", line 323, in _dequeue_run
instance.launch_run(run.run_id, workspace_process_context.create_request_context())
File "/usr/local/lib/python3.7/site-packages/dagster/_core/instance/__init__.py", line 1839, in launch_run
self.run_launcher.launch_run(LaunchRunContext(pipeline_run=run, workspace=workspace))
File "/usr/local/lib/python3.7/site-packages/dagster_docker/docker_run_launcher.py", line 153, in launch_run
self._launch_container_with_command(run, docker_image, command)
File "/usr/local/lib/python3.7/site-packages/dagster_docker/docker_run_launcher.py", line 95, in _launch_container_with_command
docker_env = dict([parse_env_var(env_var) for env_var in container_context.env_vars])
File "/usr/local/lib/python3.7/site-packages/dagster_docker/docker_run_launcher.py", line 95, in <listcomp>
docker_env = dict([parse_env_var(env_var) for env_var in container_context.env_vars])
File "/usr/local/lib/python3.7/site-packages/dagster/_core/utils.py", line 107, in parse_env_var
raise Exception(f"Tried to load environment variable {env_var_str}, but it was not set")
I set the variable inside my dagster.yaml like so:
run_launcher:
module: dagster_docker
class: DockerRunLauncher
config:
env_vars:
- DAGSTER_POSTGRES_USER
- DAGSTER_POSTGRES_PASSWORD
- DAGSTER_POSTGRES_DB
- FIVETRAN_API_KEY
- FIVETRAN_API_SECRET
daniel
01/07/2023, 7:08 PMBenedikt Buchert
01/07/2023, 7:47 PM