Hi All, I am currently running a daily schedule that yields multiple of RunRequest objects. I would like to change this to a daily partition. Is it possible to return multiple configs from a partition?

cc @prha

Hi David. Could you tell me more what those multiple configs are for? Right now, a job’s partition is considered “filled” if there is a successful run corresponding tagged with that partition. I’m not sure what it would mean for a daily partitioned job to generate multiple runs… If one of the runs fails and one succeeds, should it be considered successful or failed?

Apologies if my dagster terminology is incorrect, I am new to using it. Here is a quick overview of what I am doing: I have code that gets, processes, and persists data from a single API endpoint. I have created a dagster job to do this. The API endpoint can receive different parameters to change what data it returns. So I handle this by passing in different configs (6 in total) and "yield RunRequest" for each param. Which creates 6 runs. I would like to have a partition for each config or set of parameters. Possibly it would be better if I created a new job for each one? This would also handle the question of what is considered successful or failed. Because at the moment I have runs passing and failing for different configs:

I think if those 6 different configs map to 6 different logical flows, each of which is daily partitioned, then it might be worth splitting up into six different jobs, which all share the same underlying graph (so that they all end up running the same code, just with different config). You would then be able to more clearly see that the

natgas_trades

ran successfully for every day in the past week. You could also run a backfill for the last 4 days worth of data (let’s say that the API returned some inaccurate data for some subset) just on one of the types of data.

Thank you for the help, I have now separated these out as individual jobs