Gabriel Montañola
08/09/2023, 11:52 PMrepository_name
in JobSelector and __repository__
(the default) isn't reliable since it's a implementation detail.
_____
Hi there folks,
I'm trying to build sensors that rely on the execution status of jobs defined via Schedules (build_schedule_from_dbt_selection
).
• Since there is no job
to import I tried to use JobSelector
to select the job by it's name but the sensor picks nothing.
• If I import the schedule
and access the job
property things work as intended.
Any clues here?Gabriel Montañola
08/09/2023, 11:52 PMdef make_sucessfull_job_sensor(name, trigger_jobs, request_job, run_config=None) -> SensorDefinition:
@run_status_sensor(
name=name,
request_job=request_job,
run_status=DagsterRunStatus.SUCCESS,
monitored_jobs=trigger_jobs,
default_status=DefaultSensorStatus.RUNNING,
)
def _successful_job_sensor(context: RunStatusSensorContext):
yield RunRequest(run_key=f"triggered_by_{context.dagster_run.run_id}", run_config=run_config)
return _successful_job_sensor
yuhan
08/09/2023, 11:56 PMjob_name
in build_schedule_from_dbt_selection
, then in run_status_sensor
, you can provide the job name to JobSelector
Gabriel Montañola
08/09/2023, 11:57 PMbrain_assets_schedule = build_schedule_from_dbt_selection(
dbt_assets=[arado_dbt_assets],
job_name="brain_assets_job",
dbt_select="stg_brain__warehouse",
cron_schedule="@hourly",
)
job names are matching sadblobyuhan
08/09/2023, 11:58 PMDefinitions
and are both the sensor and the schedule provided there?Gabriel Montañola
08/09/2023, 11:59 PMGabriel Montañola
08/10/2023, 12:01 AMrepository_name
since it gets the default from dunder repository but... When I harcoded it worked :TGabriel Montañola
08/10/2023, 12:01 AMyuhan
08/10/2023, 12:01 AMGabriel Montañola
08/10/2023, 12:01 AMdefs = create_repository_using_definitions_args(
name="dagarado",
assets=all_assets,
resources=resources_by_deployment_name[get_deployment_name()],
jobs=all_jobs,
executor=in_process_executor,
schedules=all_schedules,
sensors=all_sensors,
)
yuhan
08/10/2023, 12:02 AMschedule.job
as hardcoding str might be hard to maintain.Gabriel Montañola
08/10/2023, 12:02 AM__repository___
comes fromGabriel Montañola
08/10/2023, 12:03 AMbuild_schedule_from_dbt_selection
appeared we were passing jobs all 'round! After it our code got DRY-ish and the jobs were gone.
But since I can access the job
property there is no issue 🙂yuhan
08/10/2023, 12:04 AM__repository__
is kinda an implementation detail. “definition” used to be “repository” (https://github.com/dagster-io/dagster/discussions/11167) so internal representation wise it’s still going through the repository abstraction, in order to maintain backward compatibility.Gabriel Montañola
08/10/2023, 12:04 AMGabriel Montañola
08/10/2023, 12:07 AMyuhan
08/10/2023, 12:07 AM