Jérôme MONTIGNY
02/15/2024, 8:50 AMException: Error in sensor publish_on_success_sensor: Sensor returned a RunRequest with job_name job2. Expected one of: ['job1_name']
Here is the code that must be the issue :
@run_status_sensor(
run_status=DagsterRunStatus.SUCCESS,
request_job=job1,
default_status=DefaultSensorStatus.RUNNING,
)
def publish_on_success_sensor(context):
if context.dagster_run.job_name == job1.name:
yield RunRequest(run_key=None, run_config={}, job_name="job2")
else:
return SkipReason("Not launching on jobs other than job1")
What can be the root cause ?Scout
02/15/2024, 8:50 AMpublish_on_success_sensor
sensor is returning a RunRequest
for a job named job2
, but the sensor is expected to return a RunRequest
for job1
or a list of jobs that includes job1_name
. Based on the code snippet you provided, the publish_on_success_sensor
is decorated with @run_status_sensor
and specifies request_job=job1
. This means that the sensor is configured to request runs for job1
when it yields a RunRequest
. However, inside the sensor function, you are attempting to yield a RunRequest
for a job named job2
, which is not the job specified in the request_job
parameter of the sensor decorator. To resolve this issue, you need to ensure that the RunRequest
you yield is for the job specified in the request_job
parameter of the sensor decorator. If you want to trigger job2
from this sensor, you need to include job2
in the list of jobs that the sensor is allowed to request runs for. This can be done by using the request_jobs
parameter (if available and supported by your version of Dagster) to specify multiple jobs that the sensor can trigger. Here's an example of how you might modify the sensor to include multiple jobs: python @run_status_sensor( run_status=DagsterRunStatus.SUCCESS, request_jobs=[job1, job2], # Assuming request_jobs is supported and allows multiple jobs default_status=DefaultSensorStatus.RUNNING, ) def publish_on_success_sensor(context): if context.dagster_run.job_name == job1.name: yield RunRequest(run_key=None, run_config={}, job_name="job2") else: return SkipReason("Not launching on jobs other than job1")
Please note that the request_jobs
parameter is marked as experimental in the documentation and may not be available in all versions of Dagster. If your version of Dagster does not support request_jobs
, you may need to define separate sensors for each job you want to trigger or update your sensor logic to only yield RunRequest
for job1
. If you continue to face issues, please ensure that your Dagster version supports the features you are trying to use and that you are following the latest documentation for run status sensors.Jérôme MONTIGNY
02/15/2024, 9:10 AMScout
02/15/2024, 9:10 AMrequest_jobs
parameter I used in the example is not actually part of the Dagster API based on the documentation available to me. I apologize for the confusion. In Dagster, a run_status_sensor
is typically configured to monitor the status of one or more jobs and can trigger another job upon a successful run. However, the sensor must be configured correctly to do so. To trigger job2
upon the successful completion of job1
, you would need to set up your sensor to monitor job1
and then yield a RunRequest
for job2
. Here's how you might write a sensor that does this: python @run_status_sensor( run_status=DagsterRunStatus.SUCCESS, request_job=job2, # This should be the job you want to trigger default_status=DefaultSensorStatus.RUNNING, ) def publish_on_success_sensor(context): if context.dagster_run.job_name == job1.name: # Trigger job2 if job1 succeeds yield RunRequest(run_key=None, run_config={}, job_name=job2.name) else: # Skip if the job that succeeded is not job1 return SkipReason(f"Not launching job2 because the successful job was not {job1.name}")
In this revised example, the sensor is configured with request_job=job2
, which means it is set up to trigger job2
. Inside the sensor function, it checks if the successful job is job1
and, if so, yields a RunRequest
for job2
. Please ensure that job1
and job2
are defined correctly in your code and that job1.name
and job2.name
match the actual names of the jobs you are working with. Remember to consult the latest Dagster documentation for the correct usage of sensors and any version-specific features or changes.