https://dagster.io/ logo
#ask-community
Title
# ask-community
x

Xu Zhang

07/27/2021, 12:00 AM
hey guys, if we are using LocalComputeLogManager and make hard disk as the source of truth for stdout/stderr, how do we distribute the stdout/stderr across multiple GRPC server instances? e.g., if I have a domain pointing to multiple GRPC server hosts domain.to.grpc.hosts.com, which has 3 hosts A, B, and C. If I triggered a run via Dagit, and the request landed on A, and A wrote some stuff on its machine, how could Dagit know that the stdout/stderr is on machine A? and what happens if I removed the A out from the GRPC server cluster?
d

daniel

07/27/2021, 1:26 AM
If you don't have a way to network the filesystem across multiple machines so that the same file path can point to the same place in dagit and each of the different gRPC servers, you probably want to use a different compute log manager here (for example, the S3ComputeLogManager)
x

Xu Zhang

07/27/2021, 1:51 AM
got it daniel!
2 Views