Hello, we have deployed Dagster on ECS and currently the setup reads passwords from secrets manager to access the db. From what I gather by reading the docs, it seems that any job will thus have access to the same secrets. As I am setting up dagster to be used by many projects, it would be good if we can configure the permissions in finer grain. For example, the data engineers might be building pipeline to redact sensitive information from the source. While the ML engineers might be able to use the redacted information, but blocked from reading the raw data with sensitive info. My concern is since the same secrets can be accessed by any jobs, the raw info accessed by the data engineer will hence be accessible by the ML engineer, which then defeat the purpose of redaction. Is there an easy way to configure this? Or is it currently not possible?

Hi Alvin - it's possible to give each code location a different IAM role, would that possibly help here?

I'm not so sure. Currently I don't need to give code locations any roles. It doesn't even need any permissions or access codes. Seems the permission is granted where the codes are executed (task executor permission?) Any guides?

Setting different IAM roles is how I would typically lock down AWS secretsmanager secret access

Ok. But how do you limit access on dagster? It seems to me code locations inherits all the permissions from the daemon.

It's possible to configure code locations to be loaded using specific IAM roles - you can set your grpc server for that location to use that IAM role, and configure it to launch runs from that code location using that same IAM role using the method here: https://github.com/dagster-io/dagster/discussions/12622

Cool. Let me check it out.

That can be set using the method in that discussion as well - there's a

run_resources

key that can be set to something like {"cpu": "256", "memory": "512"} and will then apply to all runs from that code location

Excellent. Thank you.

absolutely, let us know if any other questions come up!

@Alvin Lee my team has deployed Dagster in a HIPAA constrained environment and it processes PHI in a secure manner. We use IAM roles to grant the ECS task role permission to get env secrets from secrets manager. We set the IAM role for the Dagster job using the

ecs/task_overrides taskRoleArn

tag, and set cpu/mem as Daniel mentioned as well. We deliberately make sure not to emit PHI via Dagster's

context.log

. Our Dagit page is only available to dev engineers with special VPN access to the HIPAA environment. I believe if wanted authn/authz you could use the hybrid solution from Dagster, which has that built into Dagit.