following up on this a little bit, I think the consensus here was to use the

Nothing

input/output type to model dependencies between solids that don't rely on input/output. This seems to work well most of the time, but I'm getting stuck on how to leverage this for reusable solids that have a dynamic amount of upstream and/or downstream dependencies. An example of this is a

dbt_run

solid we have. This

dbt_run

solid can theoretically be dependent on 1 to n number of upstream solids. Currently, it feels like the only way to model this in Dagster is to create the solid with n number of

Nothing

input definitions. This doesn't feel intuitive to me so I'm wondering if there is something I'm missing here.

hmm, not that it's the most beautiful, but what about an input of type

List[Nothing]

you can fan-in on dependencies using a list if in a composition function or

MultiDependencyDefinition

if constructing directly

yeah exactly

https://dagster.slack.com/archives/CCCR6P2UR/p1580843595089100 this is basically exactly what Im after

or even (yuck) the bitshift operators

for all of Airflow's faults (and there are many) being able to arbitrarily define dependencies between tasks regardless of input/output was nice

@dwall the big question is what kind of syntax is readable and useful here

I'm certain Im missing a ton of nuance here, but it feels as if a

Nothing

type input definition shouldn't really care about the number of Nothings it receives

in the interim this should work:

x = solid_a()
y = solid_b()
z = solid_c()
dbt_run(on_complete=[x, y, z])

@alex are all

Nothing

inputs multi-dependencies? like

List[Nothing]

is

Nothing

?

Nothing type input definition shouldn’t really care about the number of Nothings it receives

If this isn’t how it works today it probably should

I can def check again. I think I remember dagster yelling at me

let me see if i can find a test - I think i made it work this way

@alex let me know what you find. Totally possible Im just doing something wrong

ya it yells at you if you try to define

List[Nothing]

- you just use

Nothing

and it accepts an incoming list

woah

oh shoot - the issue that might be biting you is that if the output types in the list are not

Nothing

yeah

ok ya that seems like an oversight

just want to make sure Im following - you're saying

input_def=InputDefinition(Nothing)

should work, but

input_defs=[InputDefinition(Nothing)]

does not work?

input_def=InputDefinition(Nothing) should work, but input_defs=[InputDefinition(Nothing)] does not work?

No I am saying

InputDefinition(List[Nothing])

is a problem

yes ^ exactly

for real types we just put the data in a list

ah okay - and what you're saying is because in this case it's a

Nothing

type, that

List[Nothing]

fan-in process doesn't work as expected

So I’m guessing (havent repro-ed) based on the fact that you said you tried it and it didnt work that:

x = solid_a() # returns Nothing
y = solid_b() # returns Nothing
z = solid_c() # returns int (oh no!)

dbt_run(on_complete=[x, y, z]) # fails because thers an int in the list

dbt_run(on_complete=[x, y]) # works fine

yeah let me try something closer to what you posted. tbh I'm not familiar with this

on_complete

param

oh thats just using keyword args

dbt_run([x, y])

is the same

oh wow you're right this totally works

poorly

Lol