People who use Dagster for ML experimentation. If you''re setting off a series of experiments which get triggered based on the results of ones before, do you:
A) Use sensors to trigger multiple jobs
B) Dynamically generate the job by creating ops (dynamic graphs or similar)
I think my most common use case would be, continue running more extreme versions of the same experiment until no further model performance gain is achieved (bit hyperparameter sweep esk). Better question might be has anyone used dagster closely with a hyperparamter optimization framework like optuna to spin off a job for each trial? Each experiment is a multi-hour process so separating them out into their own jobs (with some distinguishing tag) feels most fitting.