Hello friends. I am attempting to learn wonderful ...
# announcements
g
Hello friends. I am attempting to learn wonderful Dagster but am having some troubles with the
run_config
passing to
execute_pipeline
. The error I am getting i s:
raise DagsterInvalidConfigError(
dagster.core.errors.DagsterInvalidConfigError: Error in config for pipeline run_pipeline
Error 1: Missing required config entry "solids" at the root.
The problem appears to be that the config that is being passed to the
input_code
function is resolving as NoneType. I've read elsewhere that malformed returns cause the above error, and so it may be that
context.solid_config
is the cause, however this is a fairly minimal, modified from https://docs.dagster.io/tutorial/basics_solids. Any idea what I am doing wrong here?
Copy code
from dagster import (
    execute_pipeline,
    pipeline,
    solid,
    String,
)



@solid
def market_iterator(context, market_now_is):

    market_mod = market_now_is + '_modified'

    <http://context.log.info|context.log.info>(
        f"Processing {market_mod}"
    )

    return market_mod


@solid(config_schema={"market_id": String})
def input_code(context):
    market_now_is = context.solid_config["market_id"]
    return market_now_is


@pipeline
def run_pipeline():
    market_mod = market_iterator(input_code())


config_dict = {
    "solids": {
        "input_code": {
            "config": {"market_id": "DATA"}}
    }
}


def batch_markets():
    execute_pipeline(run_pipeline, run_config=config_dict)


if __name__ == "__main__":
    batch_markets()
s
I actually just copied and pasted this script and ran it successfully on my machine without any issue
g
That is strange. Must be an environment issue. I'm on dagster==0.10.6. What version are you running?
So, I just ran this code in an IDE and it run fine, however when I run it with
dagster pipeline execute -f minimal.py it
I get the above error. Is this the expected behaviour? I wouldn't have thought the CLI and python library would be that different.
s
ah that is because the CLI is trying to execute the pipeline and there is no config defined
you need to point it at a config file with the -c option
dagster pipeline execute -f minimal.py finds the pipeline declared in the file. it does not find
batch_markets
which were you specify the config
g
Thanks, it's a lot clearer now. I thought that
dagster pipeline execute -f minimal.py
and
python minimal.py
were the same thing, but seems like they have quite different configs. Is there any benefit to not running the code from the python interpreter, aside from the option to use a separate config?