The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

Is there documentation on a full list of options for config and what they mean? Like for resources (aws, spark), pipelines, solids etc?

you may find <https://docs.dagster.io/latest/api/apidocs/execution/#environment-dict-schema> helpful

i'd note though that in general the config will vary depending which resources and solids you have in your pipeline

you can explore this dynamically generated config in `dagit`