@claire @sandy quick question on
https://github.com/dagster-io/dagster/pull/11900 if one of you is around. I've just upgraded to Dagster 1.1.818, but realized after a little while that we cannot start backfills anymore, for the following reason:
• We deploy a user code deployment as a container on kubernetes with the Helm chart. This container can't access the PostgreSQL database for Dagster for two reasons: (i) network policies, (ii) our database connection string has no password and relies on AWS IAM for RDS databases (i.e., we have an IAM policy to grant the Dagit/Dagster daemon/etc. containers permissions to log in using
roles for service accounts). This was fine before since it didn't need any such access.
• With 1.1.18, when we attempt to start a backfill, Dagster seems to invoke
ExternalPartitionTags
on the gRPC server of the user code deployment, which fails when trying to create a Dagster instance as it cannot connect to the database, and prevents the backfill from starting.
Is there a way to disable this, esp. as we're not using dynamic partitions, or would you recommend to make arrangements to also allow user code deployments to access the Dagster database? Thanks!