Hi, we are deploying dagster cloud hybrid to an ex...
# dagster-plus
e
Hi, we are deploying dagster cloud hybrid to an existing cluster and vpc. Our agent is running, and has the correct VPC, subnets, and security group IDs. However, we’re in the process of setting up our github actions, which is calling the [Dagster Hybrid Deploy](https://github.com/dagster-io/dagster-cloud-action/blob/main/actions/hybrid_prod_deploy/action.yml) action. When the service is created, our action eventually fails with
Copy code
2023-04-27 15:35:56 +0000 - dagster_cloud.user_code_launcher - ERROR - Error while waiting for server for prod:dagster_user_code to be ready: dagster_cloud.workspace.ecs.client.EcsServiceError: ECS service failed because task arn:aws:ecs:us-east-1:***:task/dagster-prod/abcdef1234 failed: ResourceInitializationError: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post "<https://api.ecr.us-east-1.amazonaws.com/>": dial tcp x.x.x i/o timeout. Please check your task network configuration.
When comparing the service for our user code in the ECS console with the service for our dagster agent, we noticed that the user code service does not have an assigned security group. How can we make the github action aware of the security group it needs to use when it is creating the service?
1
d
Hi Eric - you can configure which security group IDs your agent uses with the "security_group_ids" field here: https://docs.dagster.io/dagster-cloud/deployment/agents/amazon-ecs/configuration-reference#per-deployment-configuration Altering this for your agent may require editing the cloudformation template so that it includes that field in the dagster.yaml file that it passed into the startup command for the agent task
(which security group it uses for any services it creates, I should say)
e
thanks. we’ve been translating the cloud formation to terraform, shouldve included that detail 😅
so our terraform config does have the security group set for the agent
d
Yeah, the agent won't neccesarily know that it should use that same security group for other ECS resources that it creates - I believe it will use the default one if none is configured on that dagster.yaml file
e
ah! so we need to update the yaml that is used in the command
got it
d
That's right - what you expected might be a better default behavior, but it doesn't pull it from its own task currently
🙌 1
e
thanks! that helps a lot
D 1