Shahab Tasharrofi
01/06/2023, 7:25 PMdagster.yaml
but I’m getting an error back from AWS saying: botocore.exceptions.ParamValidationError: Parameter validation failed: Invalid type for parameter networkConfiguration.awsvpcConfiguration.assignPublicIp, value: False, type: <class 'bool'>, valid types: <class 'str'>
config in `dagster.yaml`:
run_launcher:
module: "dagster_aws.ecs"
class: "EcsRunLauncher"
config:
include_sidecars: true
run_task_kwargs:
networkConfiguration:
awsvpcConfiguration:
assignPublicIp: false
securityGroups:
- sg-*****
subnets:
- subnet-****
- subnet-****
A few notes:
• I have checked with our DEVOPS and the values of security groups and subnets are correct.
• I have tried many variations of this but was never able to get it to work. I’ll list some of those tries in a thread under this.botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: Network Configuration is not valid for the given networkMode of this task definition.
Error when I tried this with assignPublicIp: DISABLED
was: botocore.errorfactory.InvalidParameterException: An error occurred (InvalidParameterException) when calling the RunTask operation: Network Configuration is not valid for the given networkMode of this task definition.
Mike Atlas
01/06/2023, 7:27 PMassignPublicIp: 'false'
? looks like it is asking for a string not a booldaniel
01/06/2023, 7:28 PMShahab Tasharrofi
01/06/2023, 7:29 PMtrue
) but I’ll try that too
@daniel: I tried that one before (see 2nd paragraph of first comment)Mike Atlas
01/06/2023, 7:30 PMassignPublicIp
Shahab Tasharrofi
01/06/2023, 7:31 PMMike Atlas
01/06/2023, 7:33 PMnetworkMode
to awsvpc
as wellShahab Tasharrofi
01/06/2023, 7:33 PMnetworkMode
is part of task def and networkConfiguration
is part of service def.Mike Atlas
01/06/2023, 7:38 PMnetworkMode
set to awspvc
for both task defs (job runs) and service defs for our ecs deploymentShahab Tasharrofi
01/06/2023, 7:39 PMrun_launcher
config?Mike Atlas
01/06/2023, 7:40 PMrun_launcher:
module: dagster_aws.ecs
class: EcsRunLauncher
config:
include_sidecars: true
container_name: "runlauncher"
Shahab Tasharrofi
01/06/2023, 7:40 PMrun_launcher:
module: "dagster_aws.ecs"
class: "EcsRunLauncher"
config:
include_sidecars: true
run_task_kwargs:
launchType: "FARGATE"
networkConfiguration:
awsvpcConfiguration:
securityGroups:
- sg-****
subnets:
- subnet-****
- subnet-****
(of course security group and subnet have to come in as env vars and be injected)
In the code repo export the DAGSTER_CONTAINER_CONTEXT
env var as follows before grpc
server is run:
export DAGSTER_CONTAINER_CONTEXT='{"ecs":{"task_definition_arn":"'$TASK_DEF_ARN'","container_name":"'$CONTAINER_NAME'"}}'
(of course TASK_DEF_ARN
should be extracted from aws and CONTAINER_NAME
should refer to whatever your container is called)