https://dagster.io/ logo
#announcements
Title
# announcements
k

King Chung Huang

06/28/2020, 4:56 PM
Is it possible to specify an
@input_hydration_config
with
required_resource_keys
that are dependent on the config? For example, in the following config, I want to say that a
glue
resource is required if the hydration is specified to be
glue
. Otherwise, it's not required.
Copy code
@input_hydration_config(
    Selector(
        {
            "glue": Permissive({
                "database": Field(String, is_required=True, description="Name of the Glue Data Catalog database."),
                "table": Field(String, is_required=True, description="Name of the table in the database."),
            }),
            "csv": Permissive({
                "path": Field(ReadPathType, is_required=True, description="Path to read from."),
            }),
            "parquet": Permissive({
                "path": Field(ReadPathType, is_required=True, description="Path to read from."),
                "columns": Field([String], is_required=False, description="Fields names to read in as columns."),

            }),
        }
    ),
    required_resource_keys = {"glue"}  # But, only if "glue" is specified in the input hydration config
)
a

alex

06/29/2020, 3:07 PM
cc @prha
p

prha

06/30/2020, 12:08 AM
We don’t have that level of flexibility right now as
required_resource_keys
is a fixed set at definition time for input hydration. It does seem like a reasonable thing to do, though, as we have a number of resource keys that get resolved at config time (e.g. system storage).