https://dagster.io/ logo
Title
a

Agon Shabi

04/28/2021, 3:32 PM
Hi all, Does anyone know if there is a way to "side load" inputs to dynamically-generated (via .map) solids? My use case loosely resembles the following:
@pipeline
def my_pipeline():
  features = ...
  untrained_models = generate_models()
  
  trained_models = untrained_models.map(
    train_model  # I want to also pass in "features" here
  )
  
  best_model = pick_best(trained_models.collect())
  ...
The
generate_models
solid yields a variable number of untrained model instances, driven by config. Is there another obvious way to do this that I'm missing, besides feeding the
features
output through
generate_models
?
a

alex

04/28/2021, 3:38 PM
something like
trained_models = untrained_models.map(
    lambda model: train_model(model=model, features=features)
  )
should do the trick
❤️ 1
a

Agon Shabi

04/28/2021, 4:25 PM
Yessss, thank you!