Ben Gotow
12/06/2018, 11:42 PMschrockn
12/06/2018, 11:47 PMschrockn
12/06/2018, 11:47 PMmax
12/07/2018, 12:31 AMmax
12/07/2018, 12:32 AMdagster pipeline execute -f pipeline.py -n define_spew_pipeline -e env.yml
max
12/07/2018, 12:33 AMBen Gotow
12/07/2018, 12:34 AMBen Gotow
12/07/2018, 12:34 AMBen Gotow
12/07/2018, 6:47 AMmax
12/07/2018, 7:27 PM>>> import this
max
12/07/2018, 7:30 PMmax
12/07/2018, 9:56 PMdbters: do you ever get the request to have a materialization option for a table that drops the table after the pipeline finishes? I’ve used this pattern a lot in the past for performance reasons. You run a pipeline and instead of using ctes, sometimes you need either a materialized view or a temp table to keep performance of transformations humming along. (classic example is when cross joins are involved in a temp table creation, but you don’t want the table hanging around after you use it).
max
12/07/2018, 10:29 PMfreiksenet
12/09/2018, 9:09 AMschrockn
12/09/2018, 1:47 PMfreiksenet
12/10/2018, 8:21 AMschrockn
12/10/2018, 12:01 PMschrockn
12/10/2018, 12:01 PMschrockn
12/10/2018, 12:02 PMfreiksenet
12/10/2018, 12:15 PMschrockn
12/10/2018, 12:16 PMschrockn
12/10/2018, 12:16 PMschrockn
12/10/2018, 12:17 PMschrockn
12/10/2018, 12:17 PMfreiksenet
12/10/2018, 12:18 PMschrockn
12/10/2018, 12:19 PMschrockn
12/10/2018, 12:19 PMschrockn
12/10/2018, 12:19 PMfreiksenet
12/10/2018, 12:41 PMfreiksenet
12/10/2018, 12:41 PM