general question: is there any appetite/future plans to be able to have solids defined within .ipynb files? so one .ipynb could contain multiple solids rather than the entire notebook being defined as a single solid?
09/11/2020, 4:59 PM
Hey Kevin. I think this is really tricky and would also require substantial changes to papermill, the library that powers dagstermill
Are you more interested in defining the dep graph in the notebook, or do you really want multiple solids?
09/11/2020, 6:00 PM
Hi Nick, I would be very interested in having a clear dependency graph within the notebook that could then be displayed in dagster when executed as a pipeline. But I do think my ideal would be to have multiple solids within a notebook such that I could have the following (attached). And this would be represented within dagster as being separate solids when executed -- if you can think of a way that would approximate this without defining them as solids I am all ears!