Really excited to start using the hosted version! Just stuck on custom environment variables import. Using Hybrid w/ ECS and haven’t figured out where to import custom env variables (i’ve tried the AWS Secrets approach with dagster tag, as well as inserting via Github Action secrets in the build file, as well as manually revising the task definition to add env variables but those get overwritten with a fresh task build). Any docs I’ve found re: ECS seem to refer to the Open Source version, which I figure is slightly different (as i deployed via the hybrid cloudformation stack)

hey @Ben Latz AWS secretsmanager secrets can be configured in Hybrid w/ ECS using the 'container_context' field described here: https://docs.dagster.cloud/guides/adding-code#environment-specific-config For example, you could configure your code as follows:

container_context:
  ecs:
    secrets:
      - name: "MY_API_TOKEN"
        valueFrom: "arn:aws:secretsmanager:us-east-1:123456789012:secret:FOO-AbCdEf:token::"
      - name: "MY_PASSWORD"
        valueFrom: "arn:aws:secretsmanager:us-east-1:123456789012:secret:FOO-AbCdEf:password::"

ahh ok got it - saw that but didn’t connect it all the way, thanks! Is this going to GET from secrets only on a deploy or every time the job runs? Secrets gets really expensive for high frequency stuff, so we have typically opted towards using Parameter Store instead

The agent will fetch secrets for every run. We should be able to add support for parameter store though.

cool, yeah the ideal target in my mind is being able to make the env variables part of the container definition so that way it’s just a normal environment variable and doesn’t make get requests every time to AWS’ services

thanks for the feedback! I don't think theres any major reason why we don't support SSM just haven't got around to it yet.

@Ben Latz it's not quite environment variables but I use SSM parameters quite heavily for configuration and wrote a little SSMSource config field that allows you to reference a SSM param in your run config, which gets looked up and automatically replaced with the SSM parameter value when you launch the run - https://gist.github.com/zyd14/1280e424e32d81e5bfb74eb81913706a