https://dagster.io/ logo
Title
v

Vinnie

03/30/2023, 2:52 PM
I really like the new experimental pythonic config and resources, but it desperately needs some documentation on how to adapt tests to the new paradigm. I already migrated a few weeks ago, but remember it took a lot of playing around with how exactly to pass them to the `op`s and `asset`s, sometimes passing as an argument, sometimes sticking to the
required_resource_keys
parameter, and experimenting a lot with the
build_*_context
functions. What triggered this message was trying to add resources to a few schedules I have running and having to go through the process again. Maybe I’m trying to adopt everything a little too early, but I feel like doing that would drastically reduce the hurdle others might feel to make the jump.
a

alex

03/30/2023, 3:25 PM
cc @ben
y

yuhan

03/30/2023, 7:07 PM
thanks for the feedback! we’re actively working on updating all the docs. should have a fresh look in about 2 weeks
❤️ 1
b

ben

03/30/2023, 9:33 PM
this feedback is very helpful - one thing we’ve heard in addition to docs is to improve the error. messages around the new config/resources system to be more actionable, which should hopefully make cutting over a bit smoother
v

Vinnie

03/31/2023, 7:37 AM
@ben yes, definitely! It was a little difficult to wrap my head around the fact that the functions require multiple arguments, but tests should still be written with
build_*_context
and passing just the
context
to the underlying function. The error messages weren’t exactly helpful, but I managed it in the end. I’d assume it would be a little trickier for users who aren’t as familiar with the internals. Although the best case scenario would be a little more consistency in how the functions are written and called by the users (e.g. in tests). In my mind it would be much more natural to simply pass the config and resource arguments directly to the function.
2
b

ben

04/03/2023, 6:06 PM
Just an update here, we’re shipping better semantics for passing resources to schedules/sensors for the direct invocation case - hopefully this tracks a bit better with what you expected https://github.com/dagster-io/dagster/pull/13323
❤️ 1
v

Vinnie

04/04/2023, 7:18 AM
@ben I’m assuming there was another PR that changed the behavior to support the same ergonomics for ops and assets? That’s great! To avoid opening another thread (even though it’s not directly related), what’s the resource behavior when bound to e.g. a sensor? Does the resource initialize anew before every tick?