Hey I have been trying out sensors But for some reason the ` dagster #announcements

Hey! I have been trying out sensors. But for some ...

Jai Kumaran

02/22/2021, 12:08 PM

Hey! I have been trying out sensors. But for some reason the

dagster-daemon

errors out:

Copy code

2021-02-22 17:35:18 - SensorDaemon - INFO - Checking for new runs for the following sensors: some_sensor
2021-02-22 17:35:20 - SensorDaemon - ERROR - Sensor failed for some_sensor : dagster.core.errors.DagsterUserCodeProcessError: FileNotFoundError: [Errno 2] No such file or directory: 'sensor_repo.py'

Stack Trace:
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 356, in ListRepositories
    self._repository_symbols_and_code_pointers.loadable_repository_symbols,
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 107, in loadable_repository_symbols
    self.load()
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 97, in load
    self._loadable_repository_symbols = load_loadable_repository_symbols(
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 121, in load_loadable_repository_symbols
    loadable_targets = get_loadable_targets(
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/utils.py", line 25, in get_loadable_targets
    else loadable_targets_from_python_file(python_file, working_directory)
  File "/usr/local/lib/python3.8/site-packages/dagster/cli/workspace/autodiscovery.py", line 11, in loadable_targets_from_python_file
    loaded_module = load_python_file(python_file, working_directory)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 87, in load_python_file
    os.stat(python_file)


Stack Trace:
  File "/usr/local/lib/python3.8/site-packages/dagster/scheduler/sensor.py", line 130, in execute_sensor_iteration
    with origin.create_handle() as repo_location_handle:
  File "/usr/local/lib/python3.8/site-packages/dagster/core/host_representation/origin.py", line 163, in create_handle
    return ManagedGrpcPythonEnvRepositoryLocationHandle(self)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/host_representation/handle.py", line 223, in __init__
    list_repositories_response = sync_list_repositories_grpc(self.client)
  File "/usr/local/lib/python3.8/site-packages/dagster/api/list_repositories.py", line 16, in sync_list_repositories_grpc
    raise DagsterUserCodeProcessError(

The sensor repo is:

Copy code

@solid(config_schema={"filename": str})
def process_file(context):
    filename = context.solid_config["filename"]
    <http://context.log.info|context.log.info>(filename)


@pipeline
def log_file_pipeline():
    process_file()


@sensor(pipeline_name="log_file_pipeline")
def some_sensor():
    yield RunRequest(
        run_config={"solids": {"process_file": {"config": {"filename": "hi"}}}},
    )


@repository
def define_sensor_repository():
    repoDict = {}
    repoDict["sensors"] = {"some_sensor": some_sensor}
    repoDict["pipelines"] = {"log_file_pipeline": log_file_pipeline}
    return repoDict

daniel

02/22/2021, 1:40 PM

Hi Jaikant - two questions - is the sensor_repo.py file available on whatever machine the dagster-daemon process is running on, and if so has the sensor_repo.py file been moved since you turned the sensor on?

Jai Kumaran

02/22/2021, 3:06 PM

@daniel I have the DAGSTER_HOME environment variable set to a directory different from where the sensor_repo.py file is located. The dagster-daemon is also started from a different directory from where the sensor_repo.py file is located. Could this be a problem?

daniel

02/22/2021, 3:30 PM

Hm, it should use the path relative to your workspace.yaml file (assuming you're using a workspace.yaml file) but I will double-check - is it possible to share your workspace.yaml file?

daniel

02/22/2021, 3:40 PM

(or alternatively, if you started the sensor from dagit or the CLI without using a workspace, what command you ran to do so)

Jai Kumaran

02/22/2021, 3:57 PM

I am not using a workspace.yaml. I am using the

dagster

cli commands to stop/start the sensor from the directory in which the sensor_repo.py file lies, using the

-f

argument. Or dagit with the

-f

argument. I can now, confirm the issue goes away if the

dagster-daemon

is started from the same directory in which the repo file lies …

daniel

02/22/2021, 3:59 PM

Got it. -f does use the relative path (and store that relative path in a database for the daemon to later load the sensor code), so that makes sense. you'll need to either run dagster-daemon from the same folder, or use a workspace.yaml file, which will store it as an absolute path instead. This is something we should document better or change.

🙏 1

daniel

02/22/2021, 4:00 PM

(More on workspaces here: https://docs.dagster.io/overview/repositories-workspaces/workspaces)

daniel

02/22/2021, 4:02 PM

And filed https://github.com/dagster-io/dagster/issues/3723 to track fixing or better explaining this case, thanks for the report

Jai Kumaran

02/22/2021, 4:12 PM

thanks @daniel! 🙏

Jai Kumaran

02/23/2021, 7:22 AM

@daniel I am not sure if the issue is with the

-f

relative path. But probably the dagster-daemon not utlising the

working-directory

field in the jobs table. Tough issue as hard coding the paths takes away the flexibility 😬 - I am sure you guys have a solution for this in the worspace.yaml file which I still need to read up properly! I checked the db and the

jobs

table has a

working_directory

field which is an absolute path and a location name which is a file name. The difficulty in the naming is symptomatic of the larger problem of flexibility probably!

Open in Slack

Previous Next