i was successful running a job locally but when i upload it to dagster cloud, i get:

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/tmp9u6bywne/storage/my_projects'

what am i doing wrong here?

The default IO manager writes to

/tmp/

, local to where the asset was materialized. If downstream assets are not materialized as part of the same job they won't be able to find upstream outputs, because they'll have been saved to the ephemeral ECS container that ran the upstream materialization. I would suggest configuring your assets to use the

ConfigurablePickledObjectS3IOManager

from

dagster-aws

so the outputs get saved to S3 and are retrievable by subsequent materializations. This doc has an example of using the

ConfigurablePickledObjectS3IOManager

https://docs.dagster.io/concepts/io-management/io-managers#applying-io-managers-to-assets

Thanks for the reply! I see, I guess I’ll just deploy to my own instance and not use Cloud

Hi there! Is there a particular reason you're choosing to use instance instead of Cloud, given what Zach said? Depending on how you deploy in production, you'll get the same error message. Dagster is made to orchestrate assets being made and not to act as storage, and the local file IO manager is made for convenience. I'd recommend that you store your assets into somewhere permanent, like S3 or a database.

these 'assets' are just tmp files that are being created because of how the pipeline is made. overall, dagster focusing on assets seem to not fit our use case, so we've decided to consider other orchestration pipelines. thanks!

Sounds like you've made up your mind, but for posterity here are the docs on ops and graphs, which is Dagster's other core framework for organizing non-asset pipelines (and which assets are built on top of). Here's a discussion on when to use ops and graphs vs. when to use assets. Good luck!