gazpot
02/26/2021, 1:19 PMrun_config
passing to execute_pipeline
.
The error I am getting i s:
raise DagsterInvalidConfigError(
dagster.core.errors.DagsterInvalidConfigError: Error in config for pipeline run_pipeline
Error 1: Missing required config entry "solids" at the root.
The problem appears to be that the config that is being passed to the input_code
function is resolving as NoneType. I've read elsewhere that malformed returns cause the above error, and so it may be that context.solid_config
is the cause, however this is a fairly minimal, modified from https://docs.dagster.io/tutorial/basics_solids.
Any idea what I am doing wrong here?
from dagster import (
execute_pipeline,
pipeline,
solid,
String,
)
@solid
def market_iterator(context, market_now_is):
market_mod = market_now_is + '_modified'
<http://context.log.info|context.log.info>(
f"Processing {market_mod}"
)
return market_mod
@solid(config_schema={"market_id": String})
def input_code(context):
market_now_is = context.solid_config["market_id"]
return market_now_is
@pipeline
def run_pipeline():
market_mod = market_iterator(input_code())
config_dict = {
"solids": {
"input_code": {
"config": {"market_id": "DATA"}}
}
}
def batch_markets():
execute_pipeline(run_pipeline, run_config=config_dict)
if __name__ == "__main__":
batch_markets()
schrockn
02/26/2021, 1:22 PMgazpot
02/26/2021, 1:36 PMdagster pipeline execute -f minimal.py it
I get the above error. Is this the expected behaviour? I wouldn't have thought the CLI and python library would be that different.schrockn
02/26/2021, 1:53 PMbatch_markets
which were you specify the configgazpot
02/26/2021, 5:53 PMdagster pipeline execute -f minimal.py
and python minimal.py
were the same thing, but seems like they have quite different configs. Is there any benefit to not running the code from the python interpreter, aside from the option to use a separate config?