Hi, I'm running into an issue in which I've create...
# announcements
b
Hi, I'm running into an issue in which I've created a dependency structure between a few composite_solids by passing Nothing between them (since I don't have real values to pass between them), however, I expected the dependency structure to be enforced for their child solids as well, but this doesn't seem to be the case. For instance, I have two composite_solids. I expect all the child solids within my first composite_solid to finish before any of the child solids in my second composite_solid begin. However, the dependency structure is only being enforced for the child solid who's return value is returned by the composite. The other child solids in my second composite_solid start at the same time as the child solids in my first composite_solid. Please let me know if what I've described makes sense sorry if that is a bit verbose/confusing. Thanks so much for any help!
m
@alex I think we have an open issue for this
a
ya - you have to treat you
Nothing
dependency like it was real data and flow it all the way to all the solids you want to block
b
Oh I have to pass it to all the child solids as well?
a
ya - I think the right mental model is treat as a piece of data that solids need to have before they can execute
given what you said - you at least need to flow it to all the origin nodes within a composite
maybe that means all the children but depends on the shape of the dags in the composites
and to @max’s point we have an issue tracking that this whole
Nothing
business is quite cumbersome and confusing
b
Ok that makes sense, good to know. Thanks for the help. Do you know the issue # so I can follow. This will work for a short term solution but yeah the Nothing stuff w/ composites doesn't feel ideal. Would love to see something a bit cleaner later on, and happy to help contribute if I can.
b
Awesome thanks!
a
ya it would be great if you could add a comment on that issue with a simplified version of what you tried to do and why you found it unpleasant
b
Definitely will do
a
just so we have more data points to factor in to our eventual solution
👍 1
b
May actually have something blocking me from flowing my
Nothing
dependency to my child solid in a specific case—I'm using dagster-dbt and I can't change the input defs to map
Nothing
for the dbt_solid I'm using within my composite solid (I left a comment on that issue with much more detail if anything is unclear). Let me know if you have any suggested workarounds to this. Thanks again!
a
b
Thanks for that! I'm not actually familiar with Phabricator—is this a change I should just look for in the next release? In the meantime, I'll just run a local copy of Dagster and make this change myself. Thanks again.
a
phabricator is just what we use for code review - it will land to github shortly and likely make it out in a release today
b
Awesome. Thanks again