The cloud-native orchestrator for the whole development lifecycle, with integrated lineage and observability.

dagster

image.png

Q: Is there an elegant way to "share" config between multiple solids?

Background.  I have a partitioned Dagster pipeline that runs daily.  A significant number of the solids in the pipeline need to be configured with current partition value - eg; the date of the partition.

How can I avoid having to configure each solid separately with the same value as below?

Approaches I've been pondering:

1. A "partition_config" resource that holds the `at_date` config; which every solid then depends on
2. A "for_day" solid that takes a `at_date` config wraps all the other solids and passes the value to it as a via an output

Dear future readers.  I went with approach (1) following <@U018ELQ343U>’s suggestion - <https://dagster.slack.com/archives/CCCR6P2UR/p1609186943365700>

```@resource(
    {
        "at_date": Field(str, is_required=True, description="The partition date - YYYY-MM-DD used in the pipeline"),
    }
)
def partition_config(context):
    return context.resource_config

@solid(
    required_resource_keys={"partition_config"}
)
def extract_accounts_from_vmap(context) -&gt; DataFrame:
    at_date = context.resources.partition_config['at_date']
    ...snip...```

My only wish is that Dagit would show things like resource configs on the run detail page so I could see them

<@U018ELQ343U> Would a button or dropdown in Dagit that showed the full run config work for you here? 