https://dagster.io/ logo
Join Slack
Powered by
Hi all, I am still having an extremely hard time g...
# deployment-ecs
l
Hi all, I am still having an extremely hard time getting this deployed on ecs. Any help would be greatly appreciated The cloud formation fails to create resources for postgres and user_code services and then deletes everything. with the following error: 
UsercodeService TaskFailedToStart: Resourceinitializationerror: unable to pull secrets or registry auth: execution resource retrieval failed: unable to retrieve ecr registry auth: service call has been retried 3 time(s): RequestError: send request failed caused by: Post "<https://api.ecr.us-east-1.amazonaws.com/>":
I have also created vpc endpoints with sg and subnets mentioned in run_launcher config for: com.amazonaws.us-east-1.ecr.api com.amazonaws.us-east-1.ecr.dkr com.amazonaws.us-east-1.secretsmananger the default vpc (which is what is set here) allows traffic from igw
Copy code
run_launcher:
  module: "dagster_aws.ecs"
  class: "EcsRunLauncher"
  config:
    run_task_kwargs:
      networkConfiguration:
        awsvpcConfiguration:
          subnets:
            - subnet-xxxxxx
            - subnet-xxxx
          assignPublicIp: true
          securityGroups:
            - vpc-xxxxx
m
my only guess is that your aws account where the cloudformation is trying to run is being constrained by network boundaries like egress firewalls. seems like the CF job is trying to reach out to the ECR API and failing - chalk this up to your environment and not the dagster set up
Seems like CF job is also failing to retrieve auth secrets from Secrets Manager, again, probably your environment and not Dagster
Possibly the CF job role is constrained ^
l
maybe. i will check out that role. do you know if there is a way to see more logs about what is happening?
m
cloudwatch logs maybe?
btw, why do you need vpc endpoints for services like 
com.amazonaws.us-east-1.ecr.api
l
m
> "In my case granting the IAM privilege 
secretsmanager:GetSecretValue
, along with opening up network access, especially as the ECR I'm trying to reach is in another account, were the keys to solving the issue"
Comment on that same answer is more along the lines of what I'm guessing is going on
But I don't know for sure
l
yes, i'll check that, too. I think that privilege is granted. but worth checking again
113 Views
Open in Slack
PreviousNext
Powered by