https://dagster.io/ logo
Powered by
#ask-community
Title
# ask-community
m

Mike Atlas

08/24/2022, 5:49 PM
Copy code
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid type for parameter containerDefinitions[0].image, value: None, type: <class 'NoneType'>, valid types: <class 'str'>
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/instance/__init__.py", line 1747, in launch_run
    self._run_launcher.launch_run(LaunchRunContext(pipeline_run=run, workspace=workspace))
  File "/usr/local/lib/python3.10/site-packages/dagster_aws/ecs/launcher.py", line 201, in launch_run
hi friends, looking for some insight on an ECS run launcher deployment I'm trying to stand up. This error above is logged by the daemon process. 🧵
is this something I need to fix in the daemon dagster.yaml config?
Copy code
run_launcher:
  module: dagster_aws.ecs
  class: EcsRunLauncher
This is all I have set here at the moment
my workspace.yaml in the daemon has:
Copy code
load_from:
  - grpc_server:
      host: pipeline1.dagster.local
      port: 8001
      location_name: "pipeline1"
@prha I am studying the 
ecs_tests/launcher_tests/test_launching.py
codebase for answers and saw that you're an author along with DanG who happens to be out on leave rn. Any thoughts on this to get me over this hurdle?
my pipeline1 repo.py code is just stubs, and the daemon can reach it.
Copy code
from dagster import graph, op, repository

@op
def hello_one():
  return 1

@op
def goodbye(foo):
  if foo != 1:
    raise Exception("Bad io manager")
  return foo * 2

@graph
def my_graph():
  goodbye(hello_one())

my_job = my_graph.to_job(name="my_job")

@repository
def deploy_docker_repository():
    return [my_job]
p

prha

08/24/2022, 6:07 PM
Do you have a built container image with your user code?
👌 1
I think the error is indicating that the container image is missing from the repository origin
👍 1
m

Mike Atlas

08/24/2022, 6:14 PM
Yes, the built container image is running as another ecs service in the same cluster, deployed similarly in 
taskDefinition.containerDefinition[0].image = blablabla/pipeline1:latest
where is 
repository_origin
supposed to be configured? workspace.yaml in same image as user code? or dagster.yaml in same image as user code? (i think the latter)
p

prha

08/24/2022, 6:48 PM
I think (not as familiar with this part of deployment) you can specify the image using the 
DAGSTER_CURRENT_IMAGE
env var. https://docs.dagster.io/concepts/repositories-workspaces/workspaces#specifying-a-docker-image-in-your-grpc-server
m

Mike Atlas

08/24/2022, 6:49 PM
ha! was just testing that out now. will get back to you
Copy code
"containerDefinitions": [
  {
    "environment": [
      {
        "name": "DAGSTER_CURRENT_IMAGE",
        "value": "<http://xyz.dkr.ecr.us-east-1.amazonaws.com/user_code:pipeline1-78|xyz.dkr.ecr.us-east-1.amazonaws.com/user_code:pipeline1-78>"
      }
    ],
    "image": "<http://xyz.dkr.ecr.us-east-1.amazonaws.com/user_code:pipeline1-78|xyz.dkr.ecr.us-east-1.amazonaws.com/user_code:pipeline1-78>",
    "name": "user_code-pipeline1-task"
  }
],
yeah i do believe that got me a step farther
Copy code
botocore.errorfactory.ClientException: An error occurred (ClientException) when calling the RegisterTaskDefinition operation: The secret name must be unique and not shared with any new or existing environment variables set on the container, such as 'DAGSTER_RDBMS_HOST'.
  File "/usr/local/lib/python3.10/site-packages/dagster/_core/instance/__init__.py", line 1747, in launch_run
    self._run_launcher.launch_run(LaunchRunContext(pipeline_run=run, workspace=workspace))
  File "/usr/local/lib/python3.10/site-packages/dagster_aws/ecs/launcher.py", line 201, in launch_run
but I popped a new error lol
seems like the attempt to merge the new task def with existing one or something
that's an ECS error actually
I think it's extra env vars set on our RunLauncher config actually, 🥺
hey @prha thank you box I'm now able to kick off my task!!!!!! just need to fix up some errors for permissions on storing compute logs
🎉 1
I can solve that myself.
thank you for the reply.
dagsir 2
14 Views
Powered by