https://dagster.io/ logo
w

Will Brown

03/24/2020, 6:34 PM
I've run in to some trouble using the dagster-pandas DataFrame validation. The builtin ColumnTypeConstraints are limited in the dtypes they recognize. For example the string_column only checks for 'object' and not the Pandas 'string' dtype, boolean_cooumn only checks 'bool' and not the Pandas 'boolean' dtype, etc. Would there be any interest in reworking the ColumnTypeConstraints to use the built in Pandas dtype checking API (i.e. is_string_dtype)?
1
d

dwall

03/24/2020, 6:38 PM
agreed that having post pd 1.0 dtypes available in the dagster-pandas library would be v nice
a

alex

03/24/2020, 6:38 PM
cc @abhi
a

abhi

03/24/2020, 6:39 PM
Ah that's a great point. We definitely need to expand the supported types. Filing an issue