https://dagster.io/ logo
Title
d

Daniel Gafni

04/11/2023, 7:56 AM
Are stdout/stderr logs supposed to appear in Dagit in realtime when using a remote log storage like GCS? For me they do not appear in realtime. Dagster is deployed to K8s.
p

prha

04/11/2023, 9:37 PM
I think you found a gap in our documentation (not sure why the GCS compute log manager isn’t in our docs site), but I think if you supply an
upload_interval
argument to your gcs compute log manager configuration, it should upload the logs on some periodic interval. e.g.
# in dagster.yaml
compute_logs:
  module: dagster_gcp.gcs.compute_log_manager
  class: GCSComputeLogManager
  config:
    bucket: "my-bucket"
    upload_interval: 30  # every 30 seconds
👍 1