https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Anthony Reksoatmodjo

08/12/2022, 12:30 AM
Hi! Is there a way to grab the
dagster.yaml
values from
init_context
? My hope is to specify non-run-specific config in
dagster.yaml
, and keep it separate from
run_config
.
c

claire

08/12/2022, 5:21 PM
Hi Anthony. Unfortunately there is no way to retrieve dagster.yaml values from
context
object. Is there a reason why you need to read from the context object? Because you could specify non-run-specific config in another yaml file and read the config in within an op or resource (we offer a utility method here that reads config from yaml).
a

Anthony Reksoatmodjo

08/12/2022, 7:19 PM
Thanks for the suggestion @claire
Is there a reason why you need to read from the context object?
No specific reason. The problem is that
init_context.instance.storage_directory()
is too small for my outputs (10-100s of GB), so I have a separate
/path/to/large/disk
for larger files (logs and stuff still get written to default
storage
). However, this path gets defined multiple times in my `run_config`:
Copy code
"resources": {
            "local_temp_dir": {
                "config": {
                    "path": "/path/to/large/disk/temp"
                }
            },
            "my_io_manager": {
                "config": {
                    "base_dir": "/path/to/large/disk"
                }
            },
        },
So I'm looking for a place within Dagster where I can define
/path/to/large/disk
once, but have it available during
resource
init and ops
We originally had our own yaml-to-config-object parser, but I was hoping there was a more Dagster-esque way of doing things.
3 Views