https://dagster.io/ logo
Title
m

Matthias Queitsch

09/02/2021, 10:59 AM
Hi, I have some questions regarding schedules and modes. If I deploy a pipeline with schedules, the schedule is by default off. How can I configure that to be on by default?. Second question is regarding modes. I know how to configure them and how to choose the modes in the dagit frontend. We have to instances of Dagster running in different AWS Accounts (staging and prod), and I want to run scheduled pipelines in both accounts. Can I tell the scheduler to use a specific mode? I wrote a resource class which can figure out the account id.
d

daniel

09/02/2021, 2:04 PM
Hi Matthias: the first question is one of our most requested features that we're looking into how to support (https://github.com/dagster-io/dagster/issues/4103) - no specific timeline available for that yet though. For the second question, right now the mode is defined as part of the ScheduleDefinition in code - so if you have a function that can check the AWS account ID when the schedule definition is being created, you could define it there (that wouldn't be possible in a resource though, since resources run when the pipeline executes).
m

Matthias Queitsch

09/02/2021, 2:26 PM
Ok, thanks for the hint. Than I will try to specify the mode when i define the schedule.
:dagstir: 1
On a side-note. What would be the best practice for code that is not a resource/solid etc. so essentially helper functions. Is there a place inside the module where I should place that?
d

daniel

09/02/2021, 2:57 PM
I don't think Dagster as a tool has a strong opinion on that - as long as it can load the Python code it's happy, so you're really free to follow your own best practices for writing Python code. In some of our examples we use a "utils" module to put common helper functions: https://github.com/dagster-io/dagster/tree/master/examples/hacker_news/hacker_news/utils
👍 1
Hi Matthias - just letting you know that the schedule/sensor feature above is now available as of the 0.14.0 release that just went live