https://dagster.io/ logo
Title
m

Mycchaka Kleinbort

11/24/2022, 3:45 PM
Hi, a quick question: how do you specify where to materialize dagster assets? Currently everything is in some version of
.../my-dagster-project/tmpav872908/storage/{assetkey}
:dagster-bot-responded-by-community: 1
v

Vinnie

11/24/2022, 3:54 PM
That’s determined by the IO Manager attached to the asset: https://docs.dagster.io/concepts/io-management/io-managers#overview
m

Mycchaka Kleinbort

11/24/2022, 4:01 PM
I read through it, but I can't seem to get it working I have
fs_io_manager.configured({'base_dir':'<D://Users/mkleinbort/Downloads/'}>)

@asset(group_name="snowflake_data", io_manager_key=fs_io_manager)
def companyIds()->list[int]:
    return [1,2,3]
But it is still going into /tmp/ folders
I think I need to modify init_context; but I don't know where that is set
looking at this
base_path = UPath(
        init_context.resource_config.get(
            "base_path", init_context.instance.storage_directory()
        )
    )
from https://docs.dagster.io/concepts/io-management/io-managers#custom-filesystem-based-io-manager
v

Vinnie

11/24/2022, 4:06 PM
Ah, you have to export the configured IO Manager to the repository as well:
@asset(group_name="snowflake_data", io_manager_key=configured_fs_io_manager)
def companyIds()->list[int]:
    return [1,2,3]

@repository
def repo():
    resource_defs = {
        "configured_fs_io_manager": fs_io_manager.configured({'base_dir':'<D://Users/mkleinbort/Downloads/'}>),
    }
    return [
        *with_resources(
            definitions=[
                companyIds,
            ],
            resource_defs=resource_defs,
        ),
    ]
you can also override the
fs_io_manager
, this is just to illustrate the explicitly configured option
m

Mycchaka Kleinbort

11/24/2022, 4:08 PM
I see; I was assuming the .configured was an inplace operation
Success!!
v

Vinnie

11/24/2022, 4:18 PM
The alternative if you wanna override any argument on a per-job/per-asset basis is to input the config when launching the run/materializing the asset (shift+click on the materialize button), something like the following should work
resources:
  fs_io_manager:
    config:
      auth:
        base_path: /my/cool/path
❤️ 1