https://dagster.io/ logo
Join Slack
Powered by
would be great if we could organize jobs in folder...
# dagster-feedback
c
would be great if we could organize jobs in folders on the web UI
➕ 9
don't have everything built out yet, but imagine this has 10 jobs starting with 
powerschool_
and another 10 starting with 
deanslist_
I think it would really add to the UX if I could keep them in collapsible folders
maybe in the repo definition we could provide a dict to the list of jobs etc, and the dict keys would be interpreted as folders on the UI
jobs = [{"powerschool": [powerschool_historical_backfill]}]
z
I just found the repo filtering button at the bottom left, that's been helping my organization a bit
j
Thanks for the feedback @Charlie Bini. This is something we're actively exploring right now.
The use case you mentioned only requires one level of folder nesting. Do you anticipate needing to organize jobs with folders within folders?
s
I have a mental hierarchy of code location / repo / job that helps me. Tags could also be a useful way to organize
c
@josh I don’t really think so at the moment, but might be a good idea to design it recursively just in case
👍 1
Also I didn’t even consider organizing them into separate repos. Might give that a try
j
@Stephen Bailey @Charlie Bini I'm curious if you have any strong reactions to either of these two concepts. We've been thinking about using labels to help organize objects in Dagster. It would allow you to create groups of jobs that span across repositories, but it might also introduce more complexity than is really needed. I'd appreciate any feedback while we iterate on this
z
+1 for the grouped by repositories, I like that a lot
I like the idea of being able to tag repos (generally speaking I think being able to tag objects in most systems is super helpful down the line), but I'm not sure I'd want to label every one of them just to get groupings - the repository split seems like a natural way to organize them visually to me
c
Ditto
Can def see labels being useful but idk how i would use them right now
Not well versed in schedules and sensors yet but I could see sharing them across repos as better than redefining them for each repo
s
i like the repository grouping for sure, but it's also a hard change to the code and reorganizing has repercussions in terms of jobs getting new unique ids, so it's not very flexible. but what if a repo was a label? so you get repo-level organization for free, but you can manually layer on labels if you want more cuts. could be confusing that jobs show up multiple times, but 🤷
i also know that jobs have a tag concept already as well. might be nice to try and combine concepts rather than introducing a new one, especially since the cloud slack alert policies encourage you to tags things 
tags={'owner": "sales"}
to jobs.
j
This is super helpful. Thanks guys!
3 Views
Open in Slack
PreviousNext
Powered by