dwall
02/26/2020, 11:37 PM@usable_as_dagster_type
have required_resource_keys
?prha
02/26/2020, 11:40 PMdwall
02/26/2020, 11:42 PMSnowflakeRelation
class that will house some useful properties/methods for snowflake objects (fully qualified object decomposition, exists()
and size()
methods, etc.) Ideally, I'd like to leverage the already existing snowflake
resource to run the queries contained within the methodsprha
02/26/2020, 11:42 PMrequired_resource_keys
on custom dagster types are resources that need to be initialized in the type check functiondwall
02/26/2020, 11:42 PMSnowflakeRelation
as a dagster type for solid I/Oprha
02/26/2020, 11:42 PMusable_as_dagster_type
just uses the python isinstance
call for type checkingprha
02/26/2020, 11:44 PMrequired_resource_keys
which I believe might be the hooks you want to use for solid I/Oprha
02/26/2020, 11:49 PMdagster_pyspark
library: https://github.com/dagster-io/dagster/blob/8fd9edd70c753ecefcb586c25a8f326a02479752/python_modules/libraries/dagster-pyspark/dagster_pyspark/__init__.py#L71prha
02/26/2020, 11:50 PMPythonObjectDagsterType
instead of the decorator, but it’s the same ideaprha
02/26/2020, 11:51 PMdwall
02/26/2020, 11:51 PM