Has anyone encountered concurrency issues when dep...
# deployment-kubernetes
Has anyone encountered concurrency issues when deploying to GCS bucket storage?
Copy code
dagster._core.errors.DagsterExecutionHandleOutputError: Error occurred while handling output "result" of step [redacted]
The above exception was caused by the following exception:
py4j.protocol.Py4JJavaError: An error occurred while calling o60.parquet.
java.io.IOException: Failed to get result: java.io.IOException: Error accessing Bucket [redacted]
When I run "materialize all", one concurrent node succeeds, and the others fail, and from what I can tell, it's non-deterministic – the code works for both if you build them individually, but if you build them together it fails.
Hi Kevin, just to clarify, are you using the GCS I/O manager? What data is being stored in GCS?
@ben I'm using a custom IO manager based off the fully featured example here. we're storing intermediate results for assets in parquuet files
@ben any ideas? or would you be able to loop in some folks that might be able to help?