How “experimental” are the new core APIs (graphs/jobs/ops)? We’re just integrating Dagster into our infrastructure and are wondering if we should start to use the 1.0 abstractions right off the bat. 🤔

I'm also at the same stage thinking same.

similar. I just switched over to the 1.0 abstractions, but all the “experimental” runtime warnings are making me sweat a little….

Warnings?? Pffft. We just call those “log chatter”.

cc @sandy - these are fantastic changes to use, and like we said in our announcement, they’ll be serving as the foundation for Dagster in 0.13.0, so you’ll get a head-start if you write your pipelines in these new paradigms now 🙂

Awesome.

@William Reed @Dylan Hunt @Donny Winston - to add, we don't have any changes in particular planned right now to those APIs. If we do make changes, it will be because of user feedback

Hello @sandy, has there been much testing on multiprocessing with the new abstractions? I seem to be able to run fine on production using the S3IOManager, but not on local with the fs or mem ones.

hey @William Reed, do you have a code snippet we could use to reproduce? definitely a concerning error cc @chris

@sandy my apologies, I think this was my mistake and not a problem with the new APIs (to your relief)! I was passing a function into a kwarg of a new

Process

when I assumed it was a string. Sorry!

that's a relief!

That pickling error was a head scratcher for a few minutes though. 🙂