https://dagster.io/ logo
#deployment-ecs
Title
# deployment-ecs
t

Timothy Elder

04/03/2023, 8:37 PM
Is there any documentation on using AWS ECR to store container images for Dagster jobs? or any documentation on what is required for Dagster to communicate with and use AWS ECR?
m

Mike Atlas

04/03/2023, 8:47 PM
when you specify an image in an ECS task definition, you can point to a private ECR registry. ability to pull the image is controlled by granting permissions to the user-code service task executor role with an IAM role that has privs to pull the image from ECR.
on my team, we deploy new user code by building an ECR image in CI/CD, then mutate the ECS task definition to replace the latest label/tag of the image, then apply the new task definition
t

Timothy Elder

04/03/2023, 8:49 PM
Good to know. Thanks!
m

Mike Atlas

04/03/2023, 8:50 PM
Dagster daemon will notice the new code and refresh your workspace/user code after the updated ECS task finishes rolling out
in practice that takes a couple minutes
t

Timothy Elder

04/03/2023, 8:51 PM
Wouldn't happen to have an example of the ECS task or config do you?
m

Mike Atlas

04/03/2023, 9:01 PM
Sure, here's the template we use to create the task definition via terraform https://gist.github.com/mikeatlas/5336f323297391a0bb1b47b0b620e2cc
And, after being applied, here's the task definition in ECS (with some annotations added by ECS) https://gist.github.com/mikeatlas/bd1be372d12d00ccb20f063e7cf4bef4
pardon some of the mix of underscores and dashes in the second link, they may not be consistent... was redacting internal values quickly
t

Timothy Elder

04/03/2023, 9:19 PM
Not a problem
m

Mike Atlas

04/04/2023, 3:02 PM
just to elaborate, we grant the task executor role permission to pull the ECR image
21 Views