Hello, has anyone correctly set up an ECS deployme...
# deployment-ecs
m
Hello, has anyone correctly set up an ECS deployment with a specific image tag for a pipelines task/service? The task that is created for the dagster-run seems to ignore the image tag and just default to "latest". For example, I have the
DAGSTER_CURRENT_IMAGE
variable set to
"$REGISTRY_URL/<my-ecr-directory>/pipelines:$DAGSTER_IMAGE_TAG"
with an image tag tied to a commit hash, which loads correctly for the pipelines task/service, but is ignored for the dagster-run. Thanks!
j
Are you managing your own deployment of ECS or using the docker compose example in the docs? https://github.com/dagster-io/dagster/tree/master/examples/deploy_ecs Can you double check that you’re indeed setting
DAGSTER_CURRENT_IMAGE
in the pipelines service? In the example it defaults to latest https://github.com/dagster-io/dagster/blob/7ae9dffc389044690742a4bea19fbd6c820e7835/examples/deploy_ecs/docker-compose.yml#L103
The run launcher is sourcing the image from the repository origin: https://github.com/dagster-io/dagster/blob/7ae9dffc389044690742a4bea19fbd6c820e7835/python_modules/libraries/dagster-aws/dagster_aws/ecs/launcher.py#L94 Which is in turn sourcing the image from that envvar.
m
I modified the docker compose example. It works perfectly, except the run launched uses the latest tag
Copy code
# This service runs a gRPC server that serves information about your
# repository. By setting DAGSTER_CURRENT_IMAGE to its own image, we tell the
# run launcher to use this same image when launching runs in a new container.
# Multiple containers like this can be deployed separately - each needs to
# run on its own port and have its own entry in the workspace.yaml file.
pipelines:
  build:
    context: .
    dockerfile: ./Dockerfile
    target: pipelines
  image: "$REGISTRY_URL/<my-ecr-registry>/pipelines:$DAGSTER_IMAGE_TAG"
  container_name: pipelines
  command: "dagster api grpc -h 0.0.0.0 -p 4000 -f dagster_repository.py"
  environment:
    DAGSTER_PG_DB: "$DAGSTER_POSTGRES_DB"
    DAGSTER_PG_HOST: "$DAGSTER_POSTGRES_HOSTNAME"
    DAGSTER_PG_PASSWORD: "$DAGSTER_POSTGRES_PASSWORD"
    DAGSTER_PG_USERNAME: "$DAGSTER_POSTGRES_USER"
    DAGSTER_CURRENT_IMAGE: "$REGISTRY_URL/<my-ecr-repository>/pipelines:$DAGSTER_IMAGE_TAG"
this is the pipelines section I have set up
In the task definition on ECS, I can see that the
DAGSTER_CURRENT_IMAGE
was set correctly to:
Copy code
{
  "name": "DAGSTER_CURRENT_IMAGE",
  "value": "<REGISTRY_URL>/<my-ecr-repository>/pipelines:b4ea218de98329ab95d322190861d5fc2c82deeb"
}
so the tag made it there, but then the dagster-run task has no tag associated with it when I try to launch a run from the dagit UI
spinthinking 1
You know what, let me try rebuilding one more time. I'm wondering if there's an image somewhere with an old value of that environment variable.
j
I’m trying to reproduce now but yeah, that’d be helpful.
👍 1
m
It appears that was it, or CloudFormation just took too long to update. So sorry to waste your time!
j
No worries! Glad to hear it’s working.
m
You all rock 👍