https://dagster.io/ logo
#ask-community
Title
# ask-community
r

Rubén Briones

03/15/2023, 6:12 PM
Hello, I'm looking for moving my Airflow DAGs to Dagster Assets/Jobs, and I'm wondering if there is something like Airflow
PostgresHook(conn_id='db').get_pandas_df(query)
in Dagster?
y

yuhan

03/16/2023, 4:45 AM
we currently don’t have postgrest-pandas integration. this can be done via resources or I/O manager. for example, we have native snowflake-pandas integration (doc) where you can write an asset whose input or output is pandas dataframe, the I/O manager can read and write df from/to snowflake. here’s the concept mapping from airflow to dagster: https://docs.dagster.io/integrations/airflow#airflow-vs-dagster-concept-map
for your case, you may do something like, where you customize the `get_pandas_df`:
Copy code
from dagster import asset, Definitions, ConfigurableResource

class MyDBResource(ConfigurableResource):
    conn_id: str
    def get_pandas_df(self) -> Response:
        ...

@asset
def data_from_db(my_conn: MyDBResource) -> Dict[str, Any]:
    return my_conn.get_pandas_df(query)

defs = Definitions(
    assets=[data_from_db],
    resources={
        "my_conn": MyDBResource(conn_id="..."),
    },
)
D 1
2 Views