https://dagster.io/ logo
z

Zach

08/01/2020, 3:57 PM
Currently having a rough time figuring out why none of my schedules are running atm. Inside my test dagster setup I've made four hourly_schedule's, one for each 15 minute mark of the hour, and none of them seem to be running. Dagit isn't showing any errors for their configuration, and running them in the playground seems to work? Any tips/tricks on what I may have done wrong here? I made a cron schedule as well (every 5 minutes), and that's not running either.
s

sashank

08/01/2020, 5:14 PM
Hey @Zach, you can use the
dagster schedule logs {schedule_name}
command to see the logs for a given schedule
You may find an error there
z

Zach

08/01/2020, 6:04 PM
Thanks @sashank. It looks that crontab isn't actually running the tasks, but if I manually run the command after grabbing it from
crontab -l
, then it works perfectly fine and there aren't any errors for the pipeline in dagit. No idea why crontab isn't running them...
This is dagster running in a python docker container (debian buster). The cron service is running, the dagster scripts all work, but for whatever reason cron is not executing any of them. How frustrating
Turns out this wasn't an issue with Dagster, but my misunderstanding of what environment a command ran by cron had access to. All of my database creds were being passed as environmental variables, weren't available to the script ran by cron, and for whatever reason this caused no logs to be sent to the scheduler.log. Solved by putting this in the entrypoint script for the docker container:
env | grep SOME_PATTERN >> /etc/environment