Responding out of thread since this is asked a lot the new i

Question

Responding out of thread since this is asked a lot the new io managers system allows for this nicely slightly smiling face

Rubén Lopez Lozoya · Answer

I think I explained myself badly though what I meant is that now I am mounting a volume in docker compose to the docker example pipelines image where I have a data folder to store results in a csv and I d like to be able to read this csv from outside docker The initial container for docker example pipelines is reacting properly to my volume but I cannot manage to get the pipeline run containers spinned by the daemon to write into this data folder Is there any way to have a volume shared with these executors

Noah K · Answer

What launcher mode are you using dagster docker presumably

Rubén Lopez Lozoya · Answer

run launcher module dagster docker class DockerRunLauncher

Noah K · Answer

Yes from the example it s on that one

Noah K · Answer

Then no

Noah K · Answer

The volume config on the generated container is not configurable

Noah K · Answer

As I said you would use an IO manager instead of writing to the local filesystem

Noah K · Answer

That might be Minio running in a persistent container or something

Noah K · Answer

But still same idea

Rubén Lopez Lozoya · Answer

Cool I m just writing the end results there and I also upload them to GDrive for analysts The issue is that some data scientists review the results once the run is finished and for them it s easier to just read local files I guess if that is my use case I then should stick to a single container like here <https docs dagster io deploying docker> and mount a volume to it

Rubén Lopez Lozoya · Answer

Is there any plan in near term to add any kind of support for volumes in the run executors

Rubén Lopez Lozoya · Answer

Or IOManagers is the way to go here with MinIO or similar

Noah K · Answer

I m not sure how many people are using dagster docker for large scale complex stuff

Noah K · Answer

I see it mostly for quick demos slightly smiling face

Rubén Lopez Lozoya · Answer

yeah I was using it as a demo before moving to k8s

Rubén Lopez Lozoya · Answer

and wanted to do it similar to what we would do there ofc no local files there but for testing purposes

Noah K · Answer

Right and there you would 100 have to use network storage of some kind

Rubén Lopez Lozoya · Answer

I wanted to have some local files slightly smiling face

Noah K · Answer

Yeah so just hasn t really been a priority

Noah K · Answer

Since it would be a test of something not possible outside of the affordances of a demo not a very good test wink

Rubén Lopez Lozoya · Answer

So understandable haha Thanks for your support though Noah I m learning a lot from you guys since I post a lot of questions here I m new to Dagster haha