https://dagster.io/ logo
Join Slack
Powered by
Hi everyone! I’m trying to enable ECS run launcher...
# deployment-ecs
s
Hi everyone! I’m trying to enable ECS run launcher for our company and I tried to follow @daniel’s example here and came up with the following config in 
dagster.yaml
but I’m getting an error back from AWS saying: 
botocore.exceptions.ParamValidationError: Parameter validation failed: Invalid type for parameter networkConfiguration.awsvpcConfiguration.assignPublicIp, value: False, type: <class 'bool'>, valid types: <class 'str'>
config in `dagster.yaml`:
Copy code
run_launcher:
  module: "dagster_aws.ecs"
  class: "EcsRunLauncher"
  config:
    include_sidecars: true
    run_task_kwargs:
      networkConfiguration:
        awsvpcConfiguration:
          assignPublicIp: false
          securityGroups:
            - sg-*****
          subnets:
            - subnet-****
            - subnet-****
A few notes: • I have checked with our DEVOPS and the values of security groups and subnets are correct. • I have tried many variations of this but was never able to get it to work. I’ll list some of those tries in a thread under this.
dagster bot responded by community 1
Error when trying it without any `run_task_kwargs`: 
botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: Network Configuration is not valid for the given networkMode of this task definition.
Error when I tried this with 
assignPublicIp: DISABLED
was: 
botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: Network Configuration is not valid for the given networkMode of this task definition.
m
have you tried 
assignPublicIp: 'false'
? looks like it is asking for a string not a bool
d
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ecs.html - I think it might be "DISABLED"? If you search the page for "assignPublicIp" that's what shows up
s
@Mike Atlas: Not yet (since in Daniel’s example it was set to just true and not string 
true
) but I’ll try that too @daniel: I tried that one before (see 2nd paragraph of first comment)
m
what about just omitting 
assignPublicIp
s
Btw, just for context, we’re on 1.1.2 (not sure if that matters)
Will try that one too.
m
Think you might need to specify 
networkMode
to 
awsvpc
as well
s
That’s specified in the task def already
👍 1
Looking at AWS console, seems like 
networkMode
is part of task def and 
networkConfiguration
is part of service def.
m
fwiw I have 
networkMode
set to 
awspvc
for both task defs (job runs) and service defs for our ecs deployment
s
Ok. I’ll try that one too. Is it by any chance possible to have your 
run_launcher
config?
m
ours is simply
Copy code
run_launcher:
  module: dagster_aws.ecs
  class: EcsRunLauncher
  config:
    include_sidecars: true
    container_name: "runlauncher"
s
Ok. thanks anyway.
👍 1
Just for posterity and in case someone had a similar issue in the future, the minimal configuration that ended up working for me under our awsvpc & fargate setup was: In daemon and dagit, add following to `dagster.yaml`:
Copy code
run_launcher:
  module: "dagster_aws.ecs"
  class: "EcsRunLauncher"
  config:
    include_sidecars: true
    run_task_kwargs:
      launchType: "FARGATE"
      networkConfiguration:
        awsvpcConfiguration:
          securityGroups:
            - sg-****
          subnets:
            - subnet-****
            - subnet-****
(of course security group and subnet have to come in as env vars and be injected) In the code repo export the 
DAGSTER_CONTAINER_CONTEXT
env var as follows before 
grpc
server is run:
Copy code
export DAGSTER_CONTAINER_CONTEXT='{"ecs":{"task_definition_arn":"'$TASK_DEF_ARN'","container_name":"'$CONTAINER_NAME'"}}'
(of course 
TASK_DEF_ARN
should be extracted from aws and 
CONTAINER_NAME
should refer to whatever your container is called)
🌈 1
daggy 3d 1
blob hearteyes 1
249 Views
Open in Slack
PreviousNext
Powered by