https://dagster.io/ logo
#ask-community
Title
# ask-community
a

Alexis Manin

02/08/2024, 8:28 AM
Has anyone tried to use DAGster as a test platform ? A little context: I am currently wondering how to efficiently create and automate a set of functional tests for one of our products at work. The goals I would like to achieve : • I need to organize tests in multiple steps (step 1: upload file A, step 2: ingest it as a product dataset, step 3 edit its metadata, etc.), so maybe a test step could be an asset or op. • Maybe a step could have additional validation operations, like
asset_check
do • I need to automate test scenario, either by reacting on a commit (via a sensor ?) or by scheduling them on a regular basis (every night, every week, etc.). Maybe even a mix of both (launch every night, but only if a change occurred in the project to test). • And, last but not least, I want a comprehensive dashboard to check what happened, which tests failed or succeed, having detailed reports on individual failures (maybe through asset metadata ?), and if possible, a broader long-term report (for example, test failure ratio on last year). I wonder if there is any documentation or user story about that somewhere, or if anyone has any experience in favor or against such use case.
j

Joe Napolitano

02/08/2024, 3:21 PM
Any reason to not just use a CI system? Pretty much every CI supports directed graphs, etc. Except they "easily support" use cases for testing, whereas Dagster has its roots in data. You could absolutely use Dagster and you can even run Dagster from or directly on your CI. If you want to trigger tests on every commit (to tests changes in isolation), then leaning on CI for that seems like the easy road.
Technically that what I do from my CI (CircleCI). Every commit tests my data process.
a

Alexis Manin

02/08/2024, 3:32 PM
We currently use CI for unit tests, i.e. tests that are simple/low-level and does not take too much time. But I would like to create a set of end-to-end tests that would test "real-world" scenarios (i.e. mimic end user interactions). That would potentially incur many long tasks, and would require to interact with multiple services. So, I was thinking that a more interactive platform than a CI would allow to better visualize/control test state. I do not know CircleCI, though. I currently use Gitlab-CI, and it is very good for simple automations, but I am more skeptical for complex use-cases.
My reasoning is that as DAGster is a data platform, we could: 1. model our data-oriented workflows (integration, analytics, viz) as DAGster assets 2. Add as much quality/metadata/checks as possible to get good visual reports/insights 3. Then configure it to run in an isolated environment with a set of test data and services But I do not know if this is a good idea, or if there are just better tools for high-level testing and reporting.
j

Joe Napolitano

02/08/2024, 10:27 PM
Understood. It's a perfect valid use case. I think mixing the two is the best result, particularly when it comes to triggering it on commits/merges. You can have separate nodes for your unit tests and then the dagster run. You would run your dagster pipeline just as you would locally. That keeps it very very easy.
And to visualize it, you would want to deploy and trigger the job, not just run it directly on CI
👍 1
2 Views