https://dagster.io/ logo
#deployment-ecs
Title
# deployment-ecs
r

Ruben Partouche

02/14/2024, 2:43 PM
Hello, I'm trying to use the EcsRunLauncher with our dagster setup. We're running open source dagster. Currently, we're running the code server, the daemons and the webserver in separate containers on ECS, all of them using the same image. The code server is on one task definition, and the daemon and the webserver are in another task definition that run both containers. To try and run the jobs with the run launcher, using the same task definition that's running the daemons/webserver I simply added in my dagster.yaml :
Copy code
run_launcher:
  module: dagster_aws.ecs
  class: EcsRunLauncher
When I deploy the new configuration, the jobs are failing with this stack trace :
Copy code
dagster._check.ParameterCheckError: Param "image" is not a str. Got None which is type <class 'NoneType'>.
  File "/usr/local/lib/python3.11/site-packages/dagster/_daemon/run_coordinator/queued_run_coordinator_daemon.py", line 351, in _dequeue_run
    instance.run_launcher.launch_run(LaunchRunContext(dagster_run=run, workspace=workspace))
  File "/usr/local/lib/python3.11/site-packages/dagster_aws/ecs/launcher.py", line 387, in launch_run
    run_task_kwargs = self._run_task_kwargs(run, image, container_context)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster_aws/ecs/launcher.py", line 633, in _run_task_kwargs
    task_definition_config = DagsterEcsTaskDefinitionConfig.from_task_definition_dict(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster_aws/ecs/tasks.py", line 154, in from_task_definition_dict
    return DagsterEcsTaskDefinitionConfig(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster_aws/ecs/tasks.py", line 63, in __new__
    check.str_param(image, "image"),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/dagster/_check/__init__.py", line 1305, in str_param
    raise _param_type_mismatch_exception(obj, str, param_name, additional_message)
After investigating a little, I found by looking at the runs that the origin of the issue is that the repository origin doesn't contain the "container_image" value :
Copy code
RepositoryPythonOrigin(executable_path='/usr/local/bin/python', code_pointer=ModuleCodePointer(module='orchestrator', fn_name='defs', working_directory='/home/app'), container_image=None, entry_point=['dagster'], container_context={})
I tried setting this value by every means I could think of, mainly using the DAGSTER_CURRENT_IMAGE env var in the webserver container, and using tags on jobs (like dagster/image:<my_image_url>), but it doesn't work... Does anyone has an idea on this ?
Resolution
After fighting with this for 1 day, found the solution 1 hour after posting this... Setting the DAGSTER_CURRENT_IMAGE was indeed the solution, but since the "origin" of the runs were not the webserver but the code server, I had to set the variable in the code server environment, and not in the webserver.
20 Views