to be more explicit about where i m starting from i have a b

Question

to be more explicit about where i m starting from i have a bunch of functions with signatures like `def make thing df1 pd DataFrame df2 pd DataFrame param1 param2 gt pd DataFrame ` and i want to use dagster to call these and memoize results EDIT made this question shorter slightly smiling face

claire · Answer

Hi William Something to note is that in Dagster assets expect to deterministically materialize based on the upstream assets What this means is that memoization for assets relies on checking if the input data is the same as it was for the previous materialization and then skipping redundant materializations if the input data and computation are both the same We do have a <https docs dagster io guides dagster asset versioning and caching|guide> that describes this in more detail Since in Dagster we only compare against the last materialization for memoization assets might not be the best use case for you since the inputs to your `make thing` functions can change each time

claire · Answer

I think one way to use your library with dagster is to create ` op`s you can create your own functions e g ``` op def my make thing fn input 1 input 2 return make thing input 1 input 2 ``` if you want to use your functions within jobs without decorating them

William Grisaitis · Answer

thank you SO much for taking the time to help me out on this i was thinking wrapping things in ops might make more sense than assets if my vague memory is right thanks for sharing the guide as well i ll do some more reading to educate myself on this and understad what you re saying about assets

claire · Answer

no problem feel free to drop any more questions in the support channel and we ll help you out rainbow daggy