Kazushi Nagayama
08/09/2023, 6:43 AM@op
def create_test_view() -> None:
bq_op_for_queries([
"CREATE OR REPLACE VIEW `test.view`"
])
@op
def depends_on_test_view(creat_test_view) -> None:
bq_op_for_queries([
"CREATE OR REPLACE VIEW `test.another_view`"
])
@graph_asset
def generate_views():
return depends_on_test_view(create_test_view())
this is the error i’m getting:
google.api_core.exceptions.NotFound: 404 Not found: Table test:test.result was not found in location US
(i’m obfuscating the project names)
i’m not querying the result
table. what am i missing?Kazushi Nagayama
08/09/2023, 6:56 AMjamie
08/09/2023, 3:21 PMbq_op_for_queries
actually makes the op for you (so you dont need to then wrap it in your own op. dagster is getting confusing about some of the outputs and looking for tables that don’t exist because of the double wrapping
tbh i would recommend either using assets like i mentioned in the above thread, or if you want to use ops, then using the bigquery resource to execute the sql within your own op. The helper function bq_op_for_queries
can be nice in some circumstances, but it isn’t as customizableKazushi Nagayama
08/09/2023, 3:30 PM