https://dagster.io/ logo
Title
y

Yang

09/20/2022, 12:05 AM
Hello, I'm getting an error that a config is missing, but I don't have a config_schema for that op
Missing required config entry "inputs" at path root:ops:pillar_scores:ops:pull_rawdata. Sample config for missing entry: {'inputs': {'dataset': '...', 'fiscal_year': 0, 'metric_settings': {}}}
@op(
    ins={
        "metric_name": In(), "metric_settings": In(),
        "fiscal_year": In(), "dataset": In()},
    out={
        "mymetric_valid_data": Out()},
    required_resource_keys={"esgdata_loader"},
)
def pull_rawdata(
        context, metric_name: str, metric_settings: dict,
        fiscal_year: int, dataset: str):
I am calling it with a dynamic output from the prev op. I don't know if that makes a difference
yield DynamicOutput((mname, msettings, fiscal_year, dataset), mapping_key)
defining in a graph.
@graph
def pillar_scores(inputs):
   
    def process_metric(metric_inputs):
        mymetric = pull_rawdata(metric_inputs)
        ...
        return metric_scores
    
    metrics_scores = pillar_definition().map(
        process_metric)
    p_scores = compute_pillar_scores(
        metrics_scores.collect())
    return p_scores
o

owen

09/20/2022, 4:42 PM
hi @Yang! that error indicates that you have unconnected inputs, which is happening because pull_rawdata has 4 input arguments, but is just being passed a single bundled set of inputs (metric_inputs). currently, it's not possible to be downstream of multiple dynamic outputs at the same time, so trying something like
pull_rawdata(*metric_inputs)
would likely cause an error. I think the simplest solution might be to bundle all 4 of those outputs into a single namedtuple output, and pass that single output into downstream ops
:dagster: 1
y

Yang

09/20/2022, 4:45 PM
Oh ok gotcha!
yeah that does work