https://dagster.io/ logo
Title
m

Melle Minderhoud

07/06/2021, 9:24 AM
Hi All, I am running into problems when launching a pipeline from dagit that is deployed on ECS. Dagit & Daemon seem to start a task based on the
dagster-run
task definition containing a container
run
, but it is killed due to the next error:
Error: Got unexpected extra arguments ("ExecuteRunArgs", "instance_ref": {"__class__": "InstanceRef", "compute_logs_data": {"__class__": "ConfigurableClassData", "class_name": "LocalComputeLogManager", "config_yaml": "base_dir: /opt/dagster/dagster_home/storage\n", "module_name": "dagster.core.storage.local_compute_log_manager"}, "custom_instance_class_data": null, "event_storage_data": {"__class__": "ConfigurableClassData", "class_name": "PostgresEventLogStorage", "config_yaml": "postgres_db:\n db_name:\n env: DAGSTER_POSTGRES_DB\n hostname:\n env: DAGSTER_POSTGRES_HOSTNAME\n password:\n env: DAGSTER_POSTGRES_PASSWORD\n port: 5432\n username:\n env: DAGSTER_POSTGRES_USER\n", "module_name": "dagster_postgres.event_log"}, "local_artifact_storage_data": {"__class__": "ConfigurableClassData", "class_name": "LocalArtifactStorage", "config_yaml": "base_dir: /opt/dagster/dagster_home/\n", "module_name": "dagster.core.storage.root"}, "run_coordinator_data": {"__class__": "ConfigurableClassData", "class_name": "QueuedRunCoordinator", "config_yaml": "{}\n", "module_name": "dagster.core.run_coordinator"}, "run_launcher_data": {"__class__": "ConfigurableClassData", "class_name": "EcsRunLauncher", "config_yaml": "{}\n", "module_name": "dagster_aws.ecs"}, "run_storage_data": {"__class__": "ConfigurableClassData", "class_name": "PostgresRunStorage", "config_yaml": "postgres_db:\n db_name:\n env: DAGSTER_POSTGRES_DB\n hostname:\n env: DAGSTER_POSTGRES_HOSTNAME\n password:\n env: DAGSTER_POSTGRES_PASSWORD\n port: 5432\n username:\n env: DAGSTER_POSTGRES_USER\n", "module_name": "dagster_postgres.run_storage"}, "schedule_storage_data": {"__class__": "ConfigurableClassData", "class_name": "PostgresScheduleStorage", "config_yaml": "postgres_db:\n db_name:\n env: DAGSTER_POSTGRES_DB\n hostname:\n env: DAGSTER_POSTGRES_HOSTNAME\n password:\n env: DAGSTER_POSTGRES_PASSWORD\n port: 5432\n username:\n env: DAGSTER_POSTGRES_USER\n", "module_name": "dagster_postgres.schedule_storage"}, "scheduler_data": {"__class__": "ConfigurableClassData", "class_name": "DagsterDaemonScheduler", "config_yaml": "{}\n", "module_name": "dagster.core.scheduler"}, "settings": {"telemetry": null}}, "pipeline_origin": {"__class__": "PipelinePythonOrigin", "pipeline_name": "[removed]", "repository_origin": {"__class__": "RepositoryPythonOrigin", "code_pointer": {"__class__": "FileCodePointer", "fn_name": "my_repository", "python_file": "repo.py", "working_directory": "[removed]"}, "container_image": "[removed]", "executable_path": "/opt/pysetup/.venv/bin/python"}}, "pipeline_run_id": "018515a6-02f2-4441-a23e-25cedb785ee0"})
ECS injects the next command in this
run
container: