Shame about PR #6850 - removed the ability to use ...
# deployment-ecs
l
Shame about PR #6850 - removed the ability to use the Compose CLI with the ECSRunLauncher when pulling secrets into the spawned task — forcing an explicit and separate task definition in order to get the sidecar that mounts the secrets in the first place kind of destroys the beauty of using the Compose CLI at all. Would love to see both options somehow, for those of us who depend on the sidecar and for those who want to run Dagit in the same task as the Daemon.
If someone is deploying Dagit and the Daemon in the same task, then they’re already using explicit task definitions, and not the Compose CLI, right? Now all users who want to use the ECSRunLauncher AND AWS secrets are forced to abandon the Compose CLI.
d
Hi Lee - a lot of our ECS experts are out of the office today, but we should be able to follow up on this next week and figure out how we can still support your use case. re: AWS secrets, could you share a bit more about what precisely the sidecar you have is doing? I'm assuming that the secrets manager integration we have described here isn't an option: https://docs.dagster.io/deployment/guides/aws#secrets-management-in-ecs, are you pulling your secrets from somewhere else?
l
Yeah, I’d love to help! And no rush, I’m happy with 0.14.2
j
Ah - sorry about that. We even discussed briefly in the PR if anybody was using sidecars but weren’t aware of any. I forgot about the secrets setup you had shared before. I think perhaps instead of https://github.com/dagster-io/dagster/pull/6929 to clear out
dependsOn
, we should leverage
dependsOn
. Have Dagster only carry forward containers that are dependencies of the run container. I think that would satisfy both use cases: • a sidecar that the main container depends on would get copied forward • an unrelated container (dagit) wouldn’t how does that sound?
l
Yeah, that would be great! That would keep our CI/CD and dev strategy nice and simple until we end up converting more processes into separate repositories. Nice and flexible! You all are awesome, thanks for being so responsive and helpful.