Hi i have some issues with sensors I am trying to add a sens dagster #announcements

Hi, i have some issues with sensors. I am trying t...

Avri Naamani

04/08/2021, 7:11 AM

Hi, i have some issues with sensors. I am trying to add a sensor to an existing working pipeline and i keep getting

Error: There are no sensors defined for repository...

while running

dagster sensor list -d path_to_working_dir/ -f /path_to_pipeline_file.py

I tried to put the sensor function and decorator in a separate file and also in the same file as the pipeline. When i try to run the same command on the sensor file i get

dagster.core.errors.DagsterUserCodeProcessError: dagster.core.errors.DagsterInvariantViolationError: No pipelines or repositories found in "address_enrichment_sensor".

What am i missing here?

Avri Naamani

04/08/2021, 7:18 AM

sensor definition:

Copy code

@sensor(pipeline_name="address_enrichment")
def my_s3_sensor(context):
    new_s3_keys = get_s3_keys("BUCKET", prefix="/raw/", since_key=context.last_run_key)
    if not new_s3_keys:
        yield SkipReason("No new s3 files found for bucket my_s3_bucket.")
        return
    for s3_key in new_s3_keys:
        yield RunRequest(run_key=s3_key, run_config={})

Rubén Lopez Lozoya

04/08/2021, 7:20 AM

Have you tried including the sensor in the same dagster repository as address_enrichment pipeline? You need to do that

Avri Naamani

04/08/2021, 7:39 AM

yes i added the sensor name to the

@repository

in the

repo.py

file

daniel

04/08/2021, 11:11 AM

Hi Avri - is it possible to post the full repo.py file?

Avri Naamani

04/08/2021, 1:27 PM

Hey Daniel, i got it figured out! turns out the issue was with the import paths in the repo.py file. Thanks anyway!

condagster 1

2 Views

Open in Slack

Previous Next