Hello, I am facing a problem when loading the dbt manifest:

UserWarning: Error loading repository location repo.py:dagster._core.errors.DagsterInvalidDefinitionError: Assets can only depend on themselves if they are time-partitioned and each partition depends on earlier partitions

Is there a way to go about this or does it require before DBT compilation action?

Do you have a cyclic dependency in your dbt project? cc @sandy since this looks like an interaction with asset partition dependencies

@rex I don’t think so. I even analysed it with networkx and checked if it had cycles and it did not. Although we have about 120MB of manifest file size. I would love to get some guidence when it comes to how to load projects of this size and some troubleshooting guidence. So far we had never succeded with loading the manifest during our PoC but we are interested in using Dagster on larger scale.

Fixed with:

dbt_assets = load_assets_from_dbt_project(project_dir=DBT_PROJECT_DIR, node_info_to_asset_key = lambda node_info: AssetKey(["dbt"] + (node_info["unique_id"].split("."))))

Would love to see docs being more elaborate on this, especially when migrating big DBT projects to Dagster.