Also https://twitter.com/schrockn/status/1341067599837204480

Is it/Would it be possible to mark a

pipeline

as a "singleton"? I have a pipeline that is rather memory/resource intensive and I would like to prevent multiple instances of it being launched, and instead have it wait until the previous run has ended (or even error out if it has to be that way)

Got a weird error from my launcher

This was in the middle of a flurry of errors so I'm guessing it's some weird timing issue?

Q: Why are Asset Materialisation details only viewable in Dagit (stored?) when using a Postgres backend? What is it about SQLite that makes it unsuitable?

Hi all! I have a question about logging

I can see intermediate object's keys, how can I see intermedaite's values?

What is the RSS url of the dagster blog?

If there is one, I don't see it

Hi. I enjoyed Nick Shrock's presentation at Coalesce and on Data Engineering podcast awhile back, and I have a noob question. If I intend to use FiveTran to replicate Oracle data into Snowflake, then use dbt (maybe just the open-source version) for transformations in Snowflake, and I want DAG orchestration including dbt models and beyond the dbt models (FiveTran upstream and, for example, Tableau Data Extracts downstream), and I want to use Dagster for end-to-end orchestration, to what extent to I need dbt-cloud for the dbt-level orchestrations? If I don't use dbt-cloud, what add'l skills does my team need to orchestrate w/ Dagster that we would otherwise not need if using dbt-cloud plus Dagster?

Q: Why are solids called “solids”? After learning (and falling in love with) dagster over the last few weeks; every concept I’ve come across has been named in a highly intuitive way that instantly makes sense. Except solids. Wouldn’t it be better to call these “units_of_work” or “compute_units”? Where did the name “solids” come from?

it is a solid name

Hello, I am trying to define a scheduler.

sync_users = dagster.ScheduleDefinition(
        "sync_users", "*/1 * * * *", "sync_users",mode='development')

I was able to provide mode to the definition. However, I am having trouble providing config to the scheduler. Is there any way to do that?

Q: Is there an elegant way to "share" config between multiple solids? Background. I have a partitioned Dagster pipeline that runs daily. A significant number of the solids in the pipeline need to be configured with current partition value - eg; the date of the partition. How can I avoid having to configure each solid separately with the same value as below?

Then use resource mythingy in each solid

Or at least that's the pattern I've been using for pipeline-level inputs

Hey! Is there a reason the solids in

dagster_gcp

that load to BQ don’t support specifying schema fields in the load job config? Is it as simple as just adding the parameter here (I see a TODO)? If so, I can make a PR https://github.com/dagster-io/dagster/blob/a07fc28ae6878235aeada88e3584c6dea01f845[…]n_modules/libraries/dagster-gcp/dagster_gcp/bigquery/configs.py

Hello Team, I'm new to Dagster and forgive me if this has been asked before. I wanted to know that if there's a way to customize dagit's UI? Or create a separate web-app and use dagster-graphql?

I went for the latter

Probably depends on how custom you mean?

Hi, Does integration with DataFlow (apache beam) on the roadmap?

Is there anywhere in a run object I can store launcher-specific data? Specifically a task ID to use later for termination.

Okay, I've made sure that everything is on exactly the same rev of all dagster libraries (I think?)

Still getting this

I had thought it was related to my code failing to load but now I'm not so sure?

Is there any hidden way to get logs out of the api daemon?