Missing required config for Job that only ever get...
# ask-community
s
Missing required config for Job that only ever get runs from Sensor I have a job that should only ever gets run from a sensor. The job has required config, which the sensor creates and passes in a
RunRequest
. The job is created from a graph using
to_job
, which is being passed
config
, but not the required config mentioned above. I'm not able to start dagster with the error,
Copy code
Error 1: Missing required config entry "ops" at the root. Sample config for missing entry: {'ops': {'send_statement': {'config': {'customer_number': '...'}}}}
I can get around this by providing a default value for the config of
""
and then asserting in the op that the config field is not empty. I was wondering if there is a better pattern here.
I realise that using config in this way probably doesn't make sense. The
customer_number
in this case should really be an input rather than a config field. In my case the sensor pulls a message from Google PubSub. The message contains the
customer_number
. But I don't know how to create a job from a graph, which requires an input
a
I’m not able to start dagster with the error
Can you provide the full stack trace of the error? My guess would be that you just need to omit the
config
argument (default config) in the
to_job
call. The way that that default config argument works is it can either be omitted or if you pass something it has to be a passing config.
But I don’t know how to create a job from a graph, which requires an input
you should be able to do this as well - did you try things that didn’t work? The main constraint here is that the input has to be loadable via config if its unconnected
🤟 1
s
@alex I think you've nailed it. I am passing a minimal config to
to_job
which looks like
Copy code
execution:
  config:
    in_process: {}
loggers:
  google_cloud_logger:
    config:
      log_level: DEBUG
And that config is missing the required config, which is why the job resulting from
to_job
is invalid. I suspect that if I omit the config from
to_job
, things will work. As it's turned out, I've favoured an input rather than config - it makes more sense for this data to be an input. The sensor has a
run_config
which includes
Copy code
"inputs": {"customer_number": customer_number}
which satisfies the graph input, which is then mapped to the op input.
In general, what I think foxed me here is providing a partial config to
to_job
, which in turn caused the error on dagster startup. I will post the results when I get around to looking into this again.
295 Views