19913d85-f662-4518-a9a1-7adc1793dfd4.png){kind=small}
Channels
#announcements
- s
schrockn
09/13/2018, 12:58 PMyu - s
schrockn
09/13/2018, 12:58 PMyup - s
schrockn
09/13/2018, 12:59 PMyou’re thinking very clearly about this - s
schrockn
09/13/2018, 12:59 PMalso if i’m understanding everything you’re saying, none of this requires API changes before we cut 0.2.0 - s
schrockn
09/13/2018, 1:00 PM*none of your proposals - f
freiksenet
09/13/2018, 1:00 PMyeah - f
freiksenet
09/13/2018, 1:00 PMabsolutely - f
freiksenet
09/13/2018, 1:00 PMI can work on that in parallel - f
freiksenet
09/13/2018, 1:00 PMand it should be fully backwards compatible - f
freiksenet
09/13/2018, 1:00 PMmaybe smth for 0.2.1 or 0.3.0 - s
schrockn
09/13/2018, 1:02 PMyeah i’d love that - s
schrockn
09/13/2018, 1:02 PMthing i’d be particular interested at this point - s
schrockn
09/13/2018, 1:03 PM1) what precisely does the backwards-compatible optional alias/pipeline node API look like 2) what is the concrete API proposal for higher order solids - f
freiksenet
09/13/2018, 1:05 PMyep - f
freiksenet
09/13/2018, 1:05 PMI think compute nodes made the alias thing pretty easy - f
freiksenet
09/13/2018, 1:06 PMbecause nodes are separate from solid definitions - s
schrockn
09/13/2018, 1:06 PMin terms of implementation absolute;y - s
schrockn
09/13/2018, 1:06 PMi love those goddamn compute nodes - f
freiksenet
09/13/2018, 1:06 PMmaybe in future we'll have solids that generate multiple compute nodes - s
schrockn
09/13/2018, 1:06 PMyes - s
schrockn
09/13/2018, 1:06 PMthe other thing - s
schrockn
09/13/2018, 1:07 PMis that if your head if in the space - s
schrockn
09/13/2018, 1:07 PM*is already in the space - s
schrockn
09/13/2018, 1:07 PMthe other thing the compute nodes need to be able to do is spawn parallel subdags - s
schrockn
09/13/2018, 1:07 PMso you have a solid that emits a stream of outputs - s
schrockn
09/13/2018, 1:07 PMwith the same name - s
schrockn
09/13/2018, 1:07 PMwant to be able to launch N children - s
schrockn
09/13/2018, 1:07 PM1 for each input - s
schrockn
09/13/2018, 1:07 PMimagine a solid that partitions a file into a bunch of files - s
schrockn
09/13/2018, 1:08 PMand you want to do a DAG on each partition