Hi everyone, my name is Fran and I'm new to dagster. I have experience with other Workflow frameworks like Airflow or Argo. At my company currently we have a custom framework which pretty much maps 1:1 to the dagster concepts, so currently I'm exploring how feasible it would be to use dagster as a library to build on top. I'm being a bit troubled with the documentation, I cannot find anywhere a good description of how the

repository.yaml

or

dagster.yaml

should be built. For example, in the case of

dagster.yaml

I don't want to hard-coded any credentials there, how can I do that? Or what are all the options for

repository.yaml

? I hope to learn more about dagster, it seems super cool and I'm excited to start exploring it.

To avoid hard-coding credentials and other config you could set env variables and write a script to sub them for default values at deployment time

Yeah, I already thought about that, I don't like it much but I guess is doable

Easiest way to see repository.yaml is to see an example: https://github.com/dagster-io/dagster/blob/master/examples/repository.yaml We are working on eliminating it entirely for our next major release.

hey @wbonelli @Fran Sanchez we support loading secrets from env vars, I’ve filed an issue to document it but something like this should work: https://github.com/dagster-io/dagster/issues/2489

@nate thanks! this is great

That's exactly what I would like to do, to load credentials from env vars. From a deployment point of view it means that we don't have to hard-code these values in any file.