https://dagster.io/ logo
Title
c

chrispc

08/12/2021, 1:57 PM
Hi team, I am so sad to say this but I am still getting troubles using Dagster in windows. My team installed Dagster in their machines (Windows) and randomly we are getting this error. It looks that Dagster has troubles running in windows machines because in the linux server we never faced the same issue.  
Exception: Timed out waiting for tail process to start
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\execution\api.py", line 756, in pipeline_execution_iterator
for event in pipeline_context.executor.execute(pipeline_context, execution_plan):
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\executor\in_process.py", line 38, in execute
yield from iter(
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\execution\api.py", line 835, in __iter__
yield from self.iterator(
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\execution\plan\execute_plan.py", line 75, in inner_plan_execution_iterator
active_execution.verify_complete(pipeline_context, step.key)
File "C:\Users\***\Anaconda3\envs\borrar\lib\contextlib.py", line 120, in __exit__
next(self.gen)
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\storage\compute_log_manager.py", line 56, in watch
yield
File "C:\Users\***\Anaconda3\envs\borrar\lib\contextlib.py", line 120, in __exit__
next(self.gen)
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\storage\local_compute_log_manager.py", line 51, in _watch_logs
yield
File "C:\Users\***\Anaconda3\envs\borrar\lib\contextlib.py", line 120, in __exit__
next(self.gen)
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\execution\compute_logs.py", line 31, in mirror_stream_to_file
yield pids
File "C:\Users\***\Anaconda3\envs\borrar\lib\contextlib.py", line 120, in __exit__
next(self.gen)
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\execution\compute_logs.py", line 75, in tail_to_stream
yield pids
File "C:\Users\***\Anaconda3\envs\borrar\lib\contextlib.py", line 120, in __exit__
next(self.gen)
File "C:\Users\***\Anaconda3\envs\borrar\lib\site-packages\dagster\core\execution\compute_logs.py", line 104, in execute_windows_tail
raise Exception("Timed out waiting for tail process to start")
d

daniel

08/12/2021, 2:25 PM
Hi - As a short term fix, you could turn off the compute log manager that surfaces solid output in dagit. Try adding this to your dagster.yaml:
compute_logs:
  module: dagster.core.storage.noop_compute_log_manager
  class: NoOpComputeLogManager
My guess is this is similar to the issue you were having with the gRPC servers where your execution environment is unable (or very slow) to write to the local filesystem
c

chrispc

08/12/2021, 2:42 PM
ohhh @daniel I found that using the API, the ´base_dir´ of the io_manager doesn't use the folder I configured in dagster.yaml. It used temp folder of my user.
d

Daniel Kim

08/12/2021, 9:09 PM