While experimenting with Dagster, I have come acro...
# ask-community
s
While experimenting with Dagster, I have come across a couple of questions. • Firstly, I am trying to determine if there is a possibility to include approval steps. Based on this issue (https://github.com/dagster-io/dagster/issues/11661), it appears that currently, there is no direct way to achieve this. However, have your customers found a viable solution to implement approval steps in their processes? • Additionally, suppose I materialize an asset. In such a case, what would be the best approach to manually modify entries within it? This could be necessary, for example, if an ML model occasionally generates inaccurate results. (edited)
t
There isn't a direct way to implement approval steps, but a workflow could be: • Split your job into two: one job pre-approval and another post • Once the pre-approval job finishes, send a Slack/email with a preview or link to the asset • On approval, push to some object storage/Google sheet/database that the approval happened • Have a sensor listen storage for the approval to be made, then kick off job 2 afterward For manually modifying entries, you have a couple options depending on your needs. What I'd recommend is to use our Config system to customize your run
s
Regarding the approval steps. I like this idea, and maybe this can also be connected to GitHub so a GitHub issue is created after the pre-approval job, and if the issue is closed, a sensor triggers.
For modifying entries. Let's take the following small example: • Asset `books`: Fetches information about 100 books and materializes it • The description for book number 10, 15, and 20 needs to be changed manually because it contains typos Would the config system be the right approach here?
t
Couple options given the context, thanks! 1. Partition by your 100 books and backfill those 3 partitions 2. Have a downstream
fixed_books
asset that reconciles the changes from an alternative asset, if possible. 3. Use config and a manual run, but then you'll have to write logic to handle what do about the config 4. Don't track the manual edits in Dagster if you don't need to keep a log of them.
s
Regarding the second option. Let's say I have partitioned the books by week. Would you then also partition the
fixed_books
asset?