https://dagster.io/ logo
Powered by
j

Jai Kumaran

02/22/2021, 12:08 PM
Hey! I have been trying out sensors. But for some reason the 
dagster-daemon
errors out:
Copy code
2021-02-22 17:35:18 - SensorDaemon - INFO - Checking for new runs for the following sensors: some_sensor
2021-02-22 17:35:20 - SensorDaemon - ERROR - Sensor failed for some_sensor : dagster.core.errors.DagsterUserCodeProcessError: FileNotFoundError: [Errno 2] No such file or directory: 'sensor_repo.py'

Stack Trace:
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 356, in ListRepositories
    self._repository_symbols_and_code_pointers.loadable_repository_symbols,
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 107, in loadable_repository_symbols
    self.load()
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 97, in load
    self._loadable_repository_symbols = load_loadable_repository_symbols(
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/server.py", line 121, in load_loadable_repository_symbols
    loadable_targets = get_loadable_targets(
  File "/usr/local/lib/python3.8/site-packages/dagster/grpc/utils.py", line 25, in get_loadable_targets
    else loadable_targets_from_python_file(python_file, working_directory)
  File "/usr/local/lib/python3.8/site-packages/dagster/cli/workspace/autodiscovery.py", line 11, in loadable_targets_from_python_file
    loaded_module = load_python_file(python_file, working_directory)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/code_pointer.py", line 87, in load_python_file
    os.stat(python_file)


Stack Trace:
  File "/usr/local/lib/python3.8/site-packages/dagster/scheduler/sensor.py", line 130, in execute_sensor_iteration
    with origin.create_handle() as repo_location_handle:
  File "/usr/local/lib/python3.8/site-packages/dagster/core/host_representation/origin.py", line 163, in create_handle
    return ManagedGrpcPythonEnvRepositoryLocationHandle(self)
  File "/usr/local/lib/python3.8/site-packages/dagster/core/host_representation/handle.py", line 223, in __init__
    list_repositories_response = sync_list_repositories_grpc(self.client)
  File "/usr/local/lib/python3.8/site-packages/dagster/api/list_repositories.py", line 16, in sync_list_repositories_grpc
    raise DagsterUserCodeProcessError(
The sensor repo is:
Copy code
@solid(config_schema={"filename": str})
def process_file(context):
    filename = context.solid_config["filename"]
    <http://context.log.info|context.log.info>(filename)


@pipeline
def log_file_pipeline():
    process_file()


@sensor(pipeline_name="log_file_pipeline")
def some_sensor():
    yield RunRequest(
        run_config={"solids": {"process_file": {"config": {"filename": "hi"}}}},
    )


@repository
def define_sensor_repository():
    repoDict = {}
    repoDict["sensors"] = {"some_sensor": some_sensor}
    repoDict["pipelines"] = {"log_file_pipeline": log_file_pipeline}
    return repoDict
d

daniel

02/22/2021, 1:40 PM
Hi Jaikant - two questions - is the sensor_repo.py file available on whatever machine the dagster-daemon process is running on, and if so has the sensor_repo.py file been moved since you turned the sensor on?
j

Jai Kumaran

02/22/2021, 3:06 PM
@daniel I have the DAGSTER_HOME environment variable set to a directory different from where the sensor_repo.py file is located. The dagster-daemon is also started from a different directory from where the sensor_repo.py file is located. Could this be a problem?
d

daniel

02/22/2021, 3:30 PM
Hm, it should use the path relative to your workspace.yaml file (assuming you're using a workspace.yaml file) but I will double-check - is it possible to share your workspace.yaml file?
(or alternatively, if you started the sensor from dagit or the CLI without using a workspace, what command you ran to do so)
j

Jai Kumaran

02/22/2021, 3:57 PM
I am not using a workspace.yaml. I am using the 
dagster
cli commands to stop/start the sensor from the directory in which the sensor_repo.py file lies, using the 
-f
argument. Or dagit with the 
-f
argument. I can now, confirm the issue goes away if the 
dagster-daemon
is started from the same directory in which the repo file lies …
d

daniel

02/22/2021, 3:59 PM
Got it. -f does use the relative path (and store that relative path in a database for the daemon to later load the sensor code), so that makes sense. you'll need to either run dagster-daemon from the same folder, or use a workspace.yaml file, which will store it as an absolute path instead. This is something we should document better or change.
🙏 1
And filed https://github.com/dagster-io/dagster/issues/3723 to track fixing or better explaining this case, thanks for the report
j

Jai Kumaran

02/22/2021, 4:12 PM
thanks @daniel! 🙏
@daniel I am not sure if the issue is with the 
-f
relative path. But probably the dagster-daemon not utlising the 
working-directory
field in the jobs table. Tough issue as hard coding the paths takes away the flexibility 😬 - I am sure you guys have a solution for this in the worspace.yaml file which I still need to read up properly! I checked the db and the 
jobs
table has a 
working_directory
field which is an absolute path and a location name which is a file name. The difficulty in the naming is symptomatic of the larger problem of flexibility probably!
Powered by