Stefan Adelbert
08/10/2022, 2:38 AMRunRequest
.
The job is created from a graph using to_job
, which is being passed config
, but not the required config mentioned above.
I'm not able to start dagster with the error,
Error 1: Missing required config entry "ops" at the root. Sample config for missing entry: {'ops': {'send_statement': {'config': {'customer_number': '...'}}}}
I can get around this by providing a default value for the config of ""
and then asserting in the op that the config field is not empty.
I was wondering if there is a better pattern here.customer_number
in this case should really be an input rather than a config field.
In my case the sensor pulls a message from Google PubSub. The message contains the customer_number
.
But I don't know how to create a job from a graph, which requires an inputalex
08/10/2022, 2:23 PMI’m not able to start dagster with the errorCan you provide the full stack trace of the error? My guess would be that you just need to omit the
config
argument (default config) in the to_job
call. The way that that default config argument works is it can either be omitted or if you pass something it has to be a passing config.But I don’t know how to create a job from a graph, which requires an inputyou should be able to do this as well - did you try things that didn’t work? The main constraint here is that the input has to be loadable via config if its unconnected
Stefan Adelbert
08/11/2022, 12:04 AMto_job
which looks like
execution:
config:
in_process: {}
loggers:
google_cloud_logger:
config:
log_level: DEBUG
And that config is missing the required config, which is why the job resulting from to_job
is invalid. I suspect that if I omit the config from to_job
, things will work.
As it's turned out, I've favoured an input rather than config - it makes more sense for this data to be an input. The sensor has a run_config
which includes
"inputs": {"customer_number": customer_number}
which satisfies the graph input, which is then mapped to the op input.to_job
, which in turn caused the error on dagster startup. I will post the results when I get around to looking into this again.