https://dagster.io/ logo
Powered by
#ask-community
Title
# ask-community
o

Olivier Dupuis

02/07/2023, 3:49 PM
Good morning Dagster friends, I have a question about running dbt seed within a Docker image 🧵
I’m loading my dbt project using the following:
Copy code
DBT_PROJECT_DIR = dp_data_warehouse.__path__[0]

my_assets = with_resources(
    load_assets_from_dbt_project(
        project_dir = DBT_PROJECT_DIR, 
        profiles_dir = DBT_PROJECT_DIR, 
        key_prefix = ["data_warehouse"],
        use_build_command = True
    ),
    resource_defs = {
        "dbt": dbt_cli_resource.configured(
            {
                "project_dir": DBT_PROJECT_DIR,
                "profiles_dir": DBT_PROJECT_DIR,
            },
        ),
    },
)
This works when running locally, outside of a Docker image. But when I run it within containers, I end up with the following error:
Copy code
No such file or directory: '/Users/olivierdupuis/Git/discursus/platform/discursus_data_platform/dp_data_warehouse/data/fips_country.csv'
That’s the file that needs to be seeded. For some reason, it still looks for that file in a path that is relevant to my local machine, but not the container. Am I misunderstanding how seeding data with dbt within a Docker image works? Thanks for any help!
o

owen

02/08/2023, 7:35 PM
hm I think this is one of those funny things with mounted volumes inside a docker container. The files "actually" live in /Users/oliverdupuis/..., but are mounted into a different directory in the context of the docker container. I think two options would be to COPY the relevant directories over instead of adding them as a volume, or (the hacky thing) just mounting the volume to an identical path inside the docker container
👍 1
Powered by