Hi there, we’re kinda interested in asset (which i...
# ask-community
s
Hi there, we’re kinda interested in asset (which is not experimental anymore in 0.15.0) and want to move our ops/graphs style to assets style. The question is: We had a simple pipeline like this: Op 1: Query the database then writing result to a temporary file Op 2: Upload the file to cloud storage then delete the temporary file What is the best way/practice to write this in asset style?
b
Hey Son, each asset contains an Op attached that defines steps to produce the asset from any upstream asset (see https://docs.dagster.io/concepts/assets/software-defined-assets). I think you can either: 1. Create an asset that contains Op 1 - with the query inside and an io_manager that can handle the writing of the temporary file. 2. Same asset as above, but your io_manager directly writes to the cloud storage. So really, you just have 1 asset in this case.
s
Hi @Barry Sun, thanks for the answer. We currently have a graph that Op1 -> Op2. If we only have 1 asset, how do we define the dependency between 2 ops?
s
Hi @Son Giang - if you're defining an asset that's built using multiple ops, you can use graph-backed assets: https://docs.dagster.io/concepts/assets/software-defined-assets#graph-backed-assets
1
b
Best go by @sandy’s advice @Son Giang