Heyo 👋 A question about job configs - how big can these get? My aim is to use a json file with potentially 100K lines to determine the structure of my graphs at the DAG build time (not the execution time). This json would actually be the end result of another job

I don’t believe we impose any limits in the dagster layer, but the config for a job is stored in the “Run” row so depending on how you are deployed you may hit some limit there.

determine the structure of my graphs at the DAG build time (not the execution time)

if this is the case I would expect you could bind the config in this step, leaving little to none for the generated Job to take at execution time

Thanks @alex 🚀 by binding do you mean using the Configured API? https://docs.dagster.io/concepts/configuration/configured#configured-api