Hi I can t get the scheduler to work and I don t know why I

Question

Hi I can t get the scheduler to work and I don t know why I m running Dagit inside a container on a Windows host The container has cron installed The pipelines execute fine if I trigger them in the Dagit UI but the schedule just doesn t activate More details point right

Ted Conbeer · Answer

Output of `dagter schedule debug` ``` dagster schedule debug Scheduler Configuration ======================= Scheduler module dagster cron cron scheduler class SystemCronScheduler config Scheduler Info ============== Running Cron Jobs 30 7 opt dagster dagster home schedules scripts c2d87ddff6867c6180a2bf164218586ffbb64c0b sh gt opt dagster dagster home schedules logs c2d87ddff6867c6180a2bf164218586ffbb64c0b scheduler log 2 gt amp 1 dagster schedule c2d87ddff6867c6180a2bf164218586ffbb64c0b 5 11 opt dagster dagster home schedules scripts 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 sh gt opt dagster dagster home schedules logs 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 scheduler log 2 gt amp 1 dagster schedule 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 Scheduler Storage Info ====================== my first pipeline cron schedule 30 7 pipeline origin id c2d87ddff6867c6180a2bf164218586ffbb64c0b python path usr local bin python repository origin id 8651c1dcac6632fd200ddfc60b20f7e7bee30fc6 repository pointer f opt dagster app pipelines py a analytics d opt dagster app schedule origin id c2d87ddff6867c6180a2bf164218586ffbb64c0b status RUNNING my second pipeline cron schedule 5 11 pipeline origin id 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 python path usr local bin python repository origin id 8651c1dcac6632fd200ddfc60b20f7e7bee30fc6 repository pointer f opt dagster app pipelines py a analytics d opt dagster app schedule origin id 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 status RUNNING```

sashank · Answer

Try running `dagster schedule logs schedule name`

Ted Conbeer · Answer

Thanks for the quick reply It points me to a file that appears to be empty ``` dagster schedule logs my second schedule opt dagster dagster home schedules logs 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 scheduler log cat opt dagster dagster home schedules logs 2d84fba9c7a01dae736f3ca8ec65c4625b2d5608 scheduler log ```

sashank · Answer

Hm if there were any errors in executing the schedule they would show up there

sashank · Answer

Some things to check

sashank · Answer

Make sure cron is actually running on your machine You can quickly test that by adding a custom cron job that writes to a file every minute

sashank · Answer

If you see that works try running that cron job command yourself and see if you get anything ` opt dagster dagster home schedules scripts c2d87ddff6867c6180a2bf164218586ffbb64c0b sh`

Ted Conbeer · Answer

ohhhh I just figured it out Not a cron issue

sashank · Answer

Nice What was it

Ted Conbeer · Answer

when viewing the schedule config in the dagit ui I m getting ```Operation name FetchScheduleYaml Message lt InactiveRpcError of RPC that terminated with status = StatusCode UNAVAILABLE details = failed to connect to all addresses debug error string = created 1605387068 582981700 description Failed to pick subchannel file src core ext filters client channel client channel cc file line 4165 referenced errors created 1605387068 582973300 description failed to connect to all addresses file src core ext filters client channel lb policy pick first pick first cc file line 397 grpc status 14 gt ``` I m trying to read from a yaml file containing secrets so I set the yaml file s path as an ENV var but the schedule needs that ENV var set separately if I understand correctly

sashank · Answer

Yes there s an env vars argument on the schedule definition that you need to set

sashank · Answer

But you re getting this error in Dagit This looks like a gRPC issue that s not related to env vars

Ted Conbeer · Answer

yeah if on the schedules page I click the down arrow in the Execution Params field I get the error in Dagit I think it s because I m setting `run config` my reading in from yaml files ```run config = merge yamls file relative path file config urt prod yaml DBT CLOUD CONFIG FILE REDSHIFT CONFIG FILE ``` where those CONFIG FILE paths are set elsewhere in the repo py file as ``` DBT CLOUD CONFIG FILE = os environ DBT CLOUD CONFIG FILE REDSHIFT CONFIG FILE = os environ REDSHIFT CONFIG FILE ``` which I m doing because I m using docker compose to copy those config files in as secrets and I thought it would be handy to store the paths as ENV vars at the same time

Ted Conbeer · Answer

This works great for presets but I guess not for schedules since they execute in a different environment

sashank · Answer

Hm when you re previewing the schedule config in dagit it s in the same environment as your presets

sashank · Answer

Can you show us your schedule definition

sashank · Answer

Btw we have a new scheduler coming out in around 3 weeks that greatly improves both the operational experience and this interaction with env variables

Ted Conbeer · Answer

Thanks again for your help < sashank> ok if I DM you

sashank · Answer

Yup I can make a DM thread slightly smiling face