Joe Krywicki
03/08/2023, 4:46 PMdagster-kubernetes
with dagster-postgres
to store runs. All of our pipelines involves ETL on data in an S3 delta-table
.
We’re trying to figure out how to run dagster locally to invoke specific downstream assets. Unfortunately, we don’t really have access to our postgres runs, locally.
Is there a general way to invoke an asset without needing to materialize every upstream asset from it?chris
03/08/2023, 9:38 PMmaterialize
or materialize_to_memory
and replace the upstream assets with dummy assets a la
from dagster import asset
def create_dummy_asset(val, key):
@asset(name=key)
def some_asset():
return val
return some_asset