anyone have any experience running unit tests on t...
# integration-dbt
t
anyone have any experience running unit tests on their dbt assets? I’m working with something like this:
Copy code
def test_dbt_assets():
    mock_dbt = ResourceDefinition.mock_resource("Mock resource for `dbt`.")
    result = materialize_to_memory(
        [dagster_demo_dbt_assets],
        resources={"dbt": mock_dbt},
    )
    assert result.success
In docs examples, the
result
can be asserted for success, as well as other, more specific things, that don’t seem to apply to the dbt assets. I’d love to be able to make some intelligent assertions about my data (things that wouldn’t necessarily overlap with
dbt test
)
plus1 1
👀 2
r
Is there anything specific that you would like to assert about your dbt assets? What I would recommend is that you use a
DbtCliResource
that uses a dbt profile and target that is configured for testing. For example, in production, you are materializing your assets in snowflake/bigquery, but in local development/testing, you can materialize them in duckdb. That way, your SQL will actually be executed. If you use a mock resource, you will not have that benefit. I could add a small example in the API docs later this week.
t
I really wanted to do that with Duckdb! Unfortunately we’re using Snowplow which is incompatible with Duckdb, so that’s a non-starter (per Snowplow and my own investigation, short of writing an entire adapter for Snowplow macros).
Is there anything specific that you would like to assert about your dbt assets?
I guess generally that there are some columns always present (
id
,
created_at
, etc.) or that they are configured to have column-level descriptions. Basically anything introspective that wouldn’t necessarily depend on running the actual models (I think unit testing will be done during merging a PR as opposed to with Dagster)
I also found that the things you can do with the
result
object are really opaque when it comes from a
@dbt_assets
decorated asset (or is it a list of assets?)
r
Got it. Sounds like you want to make assertions about your dbt project, rather than the Dagster execution of your dbt project (since mocking the entirety of Snowplow sounds non-trivial). I would suggest that you don't use
materialize_to_memory
. That API is for materializing your dbt models. But you just want to do something introspective, before execution. I suggest that you just write assertions against your dbt manifest to ensure that your project has the specifications you want (e.g. certain columns for certain models, descriptions are configured, etc)
t
yeah I guess that makes the most sense. Is there a pattern for asserting things about assets when mocking a resource?
r
I would refer you to our testing page: https://docs.dagster.io/concepts/testing