https://dagster.io/ logo
#dagster-plus
Title
# dagster-plus
r

rectalogic

03/30/2023, 8:56 PM
is there a way to add a "wildcard" code location that applies to any docker image? Using ECS hybrid, I want my agent container to register a code location (using the CLI
dagster-cloud workspace add-location ...
) that applies to all images - so I can specify the
container_context
ecs secrets, env vars,
task_role_arn
etc. Otherwise users need to keep track of all these AWS ARNs each time they add a code location. When I provision my agent via AWS CDK, I have access to all of those so would like to register that info once, for all code locations. https://docs.dagster.io/dagster-cloud/deployment/agents/amazon-ecs/configuration-reference#per-location-configuration
d

daniel

03/30/2023, 9:07 PM
You can set many of these fields at the deployment level, which will make them apply to all code locations: https://docs.dagster.io/dagster-cloud/deployment/agents/amazon-ecs/configuration-reference#per-deployment-configuration (the downside is that this configuration has to go in the agent task definition)
r

rectalogic

03/30/2023, 9:14 PM
can that accept
env_vars
and
secrets
like the locations yaml?
When I provision the agent I know all the AWS Secret ARNs and so I want to make them available to the tasks it runs, without users needing to configure them via the UI
d

daniel

03/30/2023, 9:19 PM
secrets and env_vars are options, yeah - it looks like those are missing from that reference for some reason, we'll get that fixed
r

rectalogic

03/30/2023, 9:19 PM
oh great, thanks!
r

rectalogic

04/03/2023, 5:15 PM
hmm, it looks like
agent_token
is plaintext in the cloudformation config and
dagster.yaml
. Is there a way to have dagster retrieve this from an AWS secret? Or can it be left out of the yaml and retrieved from an environment variable?
d

daniel

04/03/2023, 5:16 PM
Both of those are possible, yeah - the only detail is you'll then need to make sure that that env var or secret is included in any tasks that the agent spins up, by setting that same secret or env_var in the secrets or env_vars options mentioned earlier
changing it to
Copy code
agent_token:
  env: YOUR_ENV_VAR_NAME_HERE
will make it pull the value from that env var instead of the raw value
r

rectalogic

04/03/2023, 5:18 PM
thanks
4 Views