villebro
07/12/2019, 6:45 PMuser
07/12/2019, 7:00 PMschrockn
07/12/2019, 7:03 PMPrincipal: Errors that can be caught by unit tests should be caught by unit tests.
Corollary: Do not attempt to unit test for errors than cannot be caught by unit tests.
E.g. you’ll still want to run spark against large data sets in a staging environment because there are bugs that can only be caught at that stage. Our goal is to structure this so that we can catch more bugs earlier in the process. E.g. we shouldn’t be encountering any config parsing errors or python syntax errors in prod or a late stage integration test. Then once you trust those earlier stage processes more, you can start, for example, doing refactoring tasks with confidence that you will very likely not break prod.
Our eventual goal is to then kickstart an ecosystem based on these new abstractions where folks do build things like fakes for common cases that are in fact fake-able. I would bucket (ha ha) s3 in this category, as an example.user
07/12/2019, 11:17 PMTomas Vykruta
07/13/2019, 5:29 PMNorman Rosner
07/15/2019, 11:11 AMMeng Lee
07/16/2019, 7:15 AMnicocti
07/16/2019, 1:18 PMschrockn
07/16/2019, 1:23 PMnicocti
07/16/2019, 1:29 PMschrockn
07/16/2019, 1:31 PMschrockn
07/16/2019, 1:32 PMschrockn
07/16/2019, 1:32 PMschrockn
07/16/2019, 1:32 PMnicocti
07/16/2019, 1:32 PMnicocti
07/16/2019, 1:34 PMschrockn
07/16/2019, 1:35 PMexecute_solid
in python_modules/dagster/dagster/utils/test.pyschrockn
07/16/2019, 1:35 PMschrockn
07/16/2019, 1:36 PMnicocti
07/16/2019, 2:31 PMAndrey Savov
07/16/2019, 6:34 PMAndrey Savov
07/17/2019, 12:22 AMdagit
stored? Seems like workspaces and runs (from UI) have different persistency.Florian
07/17/2019, 7:00 AMuser
07/17/2019, 6:42 PMAndrey Savov
07/17/2019, 7:28 PMKamal Goyal
07/19/2019, 1:44 PMKamal Goyal
07/19/2019, 1:45 PMStephen Greszczyszyn
07/22/2019, 8:37 PMAlexei
07/23/2019, 11:03 AMMarkus Binsteiner
07/24/2019, 8:15 AM