This release requires db migration how is this db migration dagster #dagster-releases

This release requires db migration, how is this db...

Binoy Shah

11/18/2022, 7:27 PM

This release requires db migration, how is this db migration supposed to be handled in Kubernetes environments ? Higher environments have limitation where direct access to

dagster

command might not be available

Binoy Shah

11/18/2022, 7:45 PM

Well Nevermind.. I found it https://docs.dagster.io/deployment/guides/kubernetes/how-to-migrate-your-instance

Qwame

11/18/2022, 8:00 PM

dagster-dbt
now supports generating software-defined assets from your dbt Cloud jobs.

Any examples in the docs on how this is done?

Sean Lopp

11/18/2022, 8:16 PM

@Qwame we're working on re-vamping the dbt Cloud guide, but you'll want to use

Copy code

dbt_cloud_assets = load_assets_from_dbt_cloud_job(
  dbt_cloud = dbt_cloud_resource, 
  job_id = 1234
)

Qwame

11/19/2022, 12:36 AM

@Sean Lopp Thank you! this works. Is there a reason why this only loads the assets but does not show the dependencies between the assets loaded from dbt cloud?

load_assets_from_dbt_manifest

and

load_assets_from_dbt_project

both appear to bring the query and other metadata about the dbt assets but this one doesn't do that. Was this removed on purpose?

Stephen Bailey

11/20/2022, 8:54 PM

Freshness policy UI is slick! Is there an out of the box way to trigger Slack alerts based on Freshness violations?

rex

11/21/2022, 2:02 PM

@Qwame I’ll follow up with you, sent a DM!

Casper Weiss Bang

11/21/2022, 3:31 PM

I love the

.env

feature - would be great if it also looked there for the

DAGSTER_HOME

variable

Stephen Bailey

11/21/2022, 3:34 PM

oh man, thanks for the callout on that @Casper Weiss Bang, that's great. can get rid of a bunch of

dotenv run

calls now

Casper Weiss Bang

11/21/2022, 3:35 PM

Haha yeah i was this close to adding a yaml config file last week. Glad i didn't 😄

Casper Weiss Bang

11/21/2022, 3:35 PM

(but the one fature with the `DAGSTER_HOME`sortof breaks the epicness for me

➕ 1

rex

11/23/2022, 9:34 PM

The guide for dbt Cloud is out now: https://docs.dagster.io/master/integrations/dbt-cloud

David Jayatillake

11/27/2022, 8:13 AM

"dagster-dbt now supports generating software-defined assets from your dbt Cloud jobs" 🙌 Will this allow subsetting in use? For example if one job has 10 assets but I describe a job based on software defined assets that uses 5 of these, will it be able to use dbt cloud to just run the 5?

rex

11/27/2022, 2:41 PM

it’s subsetting support is not out yet, but coming this week

excited 1

rex

11/27/2022, 2:43 PM

here’s what i posted in #tools-dagster on the dbt slack:

We’ll be supporting a whole list of features in the next 1-2 weeks of releases including:

• Support for asset partitions

• Support for subsetting (you’ll be able to materialize individual assets from your dbt Cloud job 👀)

• Arbitrary group names/prefixes/asset keys for your generated assets

• And much more!! Let us know if you have any feedback 🙂

❤️ 1

👍 1

6 Views

Open in Slack

Previous Next