https://dagster.io/ logo
#ask-community
Title
# ask-community
v

Vladyslav Ivanov

06/02/2022, 1:46 PM
Hey guys, good morning! I wanted to ask for help with Dagster's scheduler because I am really confused. Say I have an example like the one in the getting started guide:
Copy code
from dagster import job, op


@op
def get_name():
    return "dagster"


@op
def hello(name: str):
    print(f"Hello, {name}!")


@job
def hello_dagster():
    hello(get_name())
If I want to run hello_dagster() every minute, what should I do? I've tried:
Copy code
basic_schedule = ScheduleDefinition(job=hello_dagster, cron_schedule="* * * * *")
and then created a repo and ran dagster-daemon. Would love to learn how to do this properly. Thank you!
s

Stephen Bailey

06/02/2022, 2:13 PM
yep, that looks and sounds right to me! the schedule is a separate entity from the job, so you have to build it separately.
v

Vladyslav Ivanov

06/02/2022, 2:51 PM
So.. Do I need to run dagster and dagster-daemon or how does it work?
s

Stephen Bailey

06/02/2022, 2:55 PM
ah yes. if you just run the dagit webserver -- e.g.
dagit -f my_file.py
-- you'll see warning flags on the "schedules" and "sensors" part of the UI. so you have to also run the
dagster-daemon
. It's kind of a pain because you also have to set up a
dagster.yaml
and
workspaces.yaml
file for local development. I would recommend getting this `docker compose` example running locally, then editing the
repo.py
file to play around with the scheduling stuff.
but basically that dagster-daemon is going to run the process that checks whether anything needs to be scheduled every X seconds or so.
v

Vladyslav Ivanov

06/02/2022, 2:57 PM
I see... Thank you, Stephen! So dagster-daemon should be always running in-parallel to dagster UI (dagit) or the dagster CLI?
s

Stephen Bailey

06/02/2022, 2:58 PM
yes, in your main deployment you'll need a postgres DB, dagit, the daemon, and your user code deployment. you can see these services in the compose file here.
v

Vladyslav Ivanov

06/02/2022, 2:59 PM
what is postgres being used for? Storing the runs and logs or something?
s

Stephen Bailey

06/02/2022, 2:59 PM
yep
v

Vladyslav Ivanov

06/02/2022, 3:00 PM
Interesting. Thank you for explaining. What if I have a file that runs a real-time process, such as the data feed and then I need dagster to run another process on schedule. How should I go about that?
s

Stephen Bailey

06/02/2022, 4:21 PM
it sounds like your real-time data process might be a
sensor
, which is also run by the daemon. the way i think of it -- might not be 100% right -- is this: 1. write code 2. dagit reads code and parses objects from repos -- jobs, ops, schedules, sensors 3. daemon takes any parsed schedules, sensors and starts evaluating them every N seconds
❤️ 1
v

Vladyslav Ivanov

06/02/2022, 5:08 PM
Thanks!!
🎉 1
2 Views