Chris Zubak-Skees
02/03/2023, 12:35 AMawsvpc
requires enabling task ENI, which would require modifying our Ubuntu AMI in a way I probably can't, as well as ENI trunking, which is also an issue. I see comparatively few EC2 launch type configs in here, mostly Fargate. Are we certain awsvpc
is needed? Does anyone know why?jordan
02/03/2023, 2:44 PMAre we certainOriginally we only supported Fargate.is needed?awsvpc
awsvpc
is the only network mode that AWS supports for Fargate.
I bet as our ECS support evolved to also support EC2, we never relaxed that constraint. It looks like you’ve maybe found a workaround for now by launching in Fargate instead, but if you open an issue in GitHub, we can look into relaxing the constraint.
I think we’d more or less need to thread this through DagsterEcsTaskDefinitionConfig
instead of hardcoding it. And then give you a way to configure that on the launcher:
https://github.com/dagster-io/dagster/blob/74eac38af3cb7c9f65306f6c151c4dad07a2d427/python_modules/libraries/dagster-aws/dagster_aws/ecs/tasks.py#L71Chris Zubak-Skees
02/03/2023, 5:55 PMAndrew Grigorev
02/08/2023, 8:41 AMuse_current_ecs_task_config: false
task_definition: <arn-of-a-manually-created-task-definition-without-awsvpc>
run_task_kwargs:
cluster: <cluster-name>
in EcsRunLauncher config could be a workaround for this issue?