I really like the new experimental pythonic config...
# dagster-feedback
v
I really like the new experimental pythonic config and resources, but it desperately needs some documentation on how to adapt tests to the new paradigm. I already migrated a few weeks ago, but remember it took a lot of playing around with how exactly to pass them to the `op`s and `asset`s, sometimes passing as an argument, sometimes sticking to the
required_resource_keys
parameter, and experimenting a lot with the
build_*_context
functions. What triggered this message was trying to add resources to a few schedules I have running and having to go through the process again. Maybe I’m trying to adopt everything a little too early, but I feel like doing that would drastically reduce the hurdle others might feel to make the jump.
a
cc @ben
y
thanks for the feedback! we’re actively working on updating all the docs. should have a fresh look in about 2 weeks
❤️ 1
b
this feedback is very helpful - one thing we’ve heard in addition to docs is to improve the error. messages around the new config/resources system to be more actionable, which should hopefully make cutting over a bit smoother
v
@ben yes, definitely! It was a little difficult to wrap my head around the fact that the functions require multiple arguments, but tests should still be written with
build_*_context
and passing just the
context
to the underlying function. The error messages weren’t exactly helpful, but I managed it in the end. I’d assume it would be a little trickier for users who aren’t as familiar with the internals. Although the best case scenario would be a little more consistency in how the functions are written and called by the users (e.g. in tests). In my mind it would be much more natural to simply pass the config and resource arguments directly to the function.
2
b
Just an update here, we’re shipping better semantics for passing resources to schedules/sensors for the direct invocation case - hopefully this tracks a bit better with what you expected https://github.com/dagster-io/dagster/pull/13323
❤️ 1
v
@ben I’m assuming there was another PR that changed the behavior to support the same ergonomics for ops and assets? That’s great! To avoid opening another thread (even though it’s not directly related), what’s the resource behavior when bound to e.g. a sensor? Does the resource initialize anew before every tick?