12/28/2020, 8:18 PM
Q: Is there an elegant way to "share" config between multiple solids? Background. I have a partitioned Dagster pipeline that runs daily. A significant number of the solids in the pipeline need to be configured with current partition value - eg; the date of the partition. How can I avoid having to configure each solid separately with the same value as below?
Approaches I've been pondering: 1. A "partition_config" resource that holds the
config; which every solid then depends on 2. A "for_day" solid that takes a
config wraps all the other solids and passes the value to it as a via an output
Dear future readers. I went with approach (1) following @Noah K’s suggestion -
👌 2
Copy code
        "at_date": Field(str, is_required=True, description="The partition date - YYYY-MM-DD used in the pipeline"),
def partition_config(context):
    return context.resource_config

def extract_accounts_from_vmap(context) -> DataFrame:
    at_date = context.resources.partition_config['at_date']
🙌 2

Noah K

12/29/2020, 12:05 PM
My only wish is that Dagit would show things like resource configs on the run detail page so I could see them
👍 1
But logging the values helps at least


12/29/2020, 3:21 PM
@Noah K Would a button or dropdown in Dagit that showed the full run config work for you here?

Noah K

12/29/2020, 3:25 PM
Yep 🙂
I know it's been on the list forever