Hi! My current stack is dagster, airbyte, dbt, clickhouse. Airbyte streams are loaded in as assets(mysql to clickhouse), and dbt modules (on clickhouse) are loaded in as assets.
Question:
Is there a way to separate streams within an airbyte connection?
• E.g., I have one connection with ten tables in it.
• I want to put 5 of those tables in a daily job/schedule, and the other 5 in a weekly job/schedule.
• The error I run into is "This AssetsDefinition does not support subsetting. Please select all asset keys produced by this asset."
• In other words, this is the result of trying to replicate only a subset of steams (tables) in the airbyte connection via dagster, instead of all of them.
What I'm using:
• dagster_airbyte.AirbyteResource -> dagster_airbyte.build_airbyte_assets