dagster #announcements

schrockn

09/04/2018, 2:28 PM

oh i’ve seen you comment in the code a few times about definitions key’ed by name versus having a name property

schrockn

09/04/2018, 2:28 PM

slack screwed up and sent those out of order

freiksenet

09/04/2018, 2:28 PM

well

schrockn

09/04/2018, 2:29 PM

i’ve been doing one or the other depending on where I think it suits the user-facing API best

freiksenet

09/04/2018, 2:29 PM

I still like source / matetializations decorators more

freiksenet

09/04/2018, 2:29 PM

I dunno

schrockn

09/04/2018, 2:31 PM

hmmm are we talking about two different things?

schrockn

09/04/2018, 2:31 PM

let’s talk about the decorators

freiksenet

09/04/2018, 2:31 PM

maybe source/matetialization/transform decorators that explode config and args

freiksenet

09/04/2018, 2:31 PM

and solid that is more generic and doesn't

freiksenet

09/04/2018, 2:32 PM

config or args I guess

schrockn

09/04/2018, 2:32 PM

i think solid should have a @with_config

schrockn

09/04/2018, 2:32 PM

and then only if that is there

schrockn

09/04/2018, 2:32 PM

have a config argument

schrockn

09/04/2018, 2:32 PM

and then we can hold off on @source and @materialization for now

freiksenet

09/04/2018, 2:32 PM

that doesn't make much sense cause we know if config is needed based on solid definition

schrockn

09/04/2018, 2:33 PM

well i don’t want to force every @solid to specify a config object

schrockn

09/04/2018, 2:34 PM

but that’s why you’re into @source and @materialization

schrockn

09/04/2018, 2:34 PM

i think transforms that have config will end up being more common than we think

schrockn

09/04/2018, 2:34 PM

and i think we should let that play out

freiksenet

09/04/2018, 2:35 PM

so I don't like with config cause I assume that if config is specified

schrockn

09/04/2018, 2:35 PM

with the new config system, i’m no longer a fan of exploding config into separate args

freiksenet

09/04/2018, 2:35 PM

it's going to be used

schrockn

09/04/2018, 2:35 PM

so no matter what i think it should be shuffled into user-space code as a single object

freiksenet

09/04/2018, 2:36 PM

i agree

schrockn

09/04/2018, 2:36 PM

cool

schrockn

09/04/2018, 2:36 PM

yeah re: config i see what you’re saying

freiksenet

09/04/2018, 2:36 PM

I'm wondering if inputs should also

freiksenet

09/04/2018, 2:36 PM

maybe not

freiksenet

09/04/2018, 2:36 PM

they are a flat list