Hi dagster support I would be so grateful if anyone could he

Question

Hi dagster support I would be so grateful if anyone could help me get a basic running example of my dbt project running on dagster I got past the dependency issue by installing an older version of dagster dbt im no longer getting any errors on that I still would love to talk to someone about setting up a basic running example of my DBT project running on dagster

johann · Answer

Hi Nicholas have you gotten a chance to look at <https docs dagster io integrations dbt using dbt with dagster>

Nicholas Buck · Answer

Johann I did i guess im still trying to wrap my head around all the new concepts and get myself acquainted with dagster it would be nice to chat with someone for a few minutes sometimes talking it out helps me understand it better

owen · Answer

hi < Nicholas Buck> I can meet with you tomorrow if there s a convenient time there s also this example repo which might be helpful

Nicholas Buck · Answer

Thanks for your offer to help I am on a time crunch and because of other obligations I am pushed to try to get a working example today

Nicholas Buck · Answer

ill give this a shot but i am not integrating with all the different toys at the moment like docker snowflake etc

Nicholas Buck · Answer

just trying to weed through and get only the most basic components running

owen · Answer

gotcha we re right in the thick of pushing out a major release for tomorrow so I won t have time today for what it s worth you can basically delete any of the airbyte pandas etc stuff from the example repo if you don t want to use those you ll also be able to load that example without any modification you just won t be able to kick off runs from dagit

Nicholas Buck · Answer

my man giving it a shot now

Nicholas Buck · Answer

stuck again im almost there but i get an error related to airbyte when i materialize all in dagit

owen · Answer

hm if you want to circumvent airbyte I believe you can modify constants py so that `PG DESTINATION CONFIG = PG SOURCE CONFIG` and then set `DBT CONFIG = project dir DBT PROJECT DIR profiles dir DBT PROFILES DIR target skip airbyte `

owen · Answer

you won t be able to hit materialize all but you can select the non airbyte assets cmd+click and go from there

Nicholas Buck · Answer

tried that but still gives airbyte related errors just ran the DBT and python

owen · Answer

not at my computer right now but do you mind sharing the error message

Nicholas Buck · Answer

sec i might have stumbled on the answer

Nicholas Buck · Answer

using pgadmin in the database postgres replica im not getting any tables appearing

Nicholas Buck · Answer

heres where the errors start

Nicholas Buck · Answer

2022 06 08 20 35 01 0400 dagster INFO resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt 2 of 3 START table model public users augmented RUN 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt 1 of 3 ERROR creating table model public orders cleaned ERROR in 0 44s 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt 2 of 3 ERROR creating table model public users augmented ERROR in 0 44s 2022 06 08 20 35 02 0400 dagster INFO resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt 3 of 3 SKIP relation public daily order summary SKIP 2022 06 08 20 35 02 0400 dagster INFO resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt Finished running 3 table models in 3 30s 2022 06 08 20 35 02 0400 dagster INFO resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt Completed with 2 errors and 0 warnings 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt Database Error in model orders cleaned models orders cleaned sql 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt relation public airbyte raw orders does not exist 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt LINE 11 from postgres replica public airbyte raw orders 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt ^ 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt compiled SQL at target run mds dbt models orders cleaned sql 2022 06 08 20 35 02 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt Database Error in model users augmented models users augmented sql 2022 06 08 20 35 03 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt relation public airbyte raw users does not exist 2022 06 08 20 35 03 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt LINE 10 from postgres replica public airbyte raw users 2022 06 08 20 35 03 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt ^ 2022 06 08 20 35 03 0400 dagster ERROR resource dbt 7052dbfa 996b 41e3 82a2 2a022c79e551 run dbt mds dbt compiled SQL at target run mds dbt models users augmented sql

owen · Answer

ah i see sorry about that rabbit hole i think airbyte would indeed need to work for this demo to function at least without some more intensive modifications of the demo

owen · Answer

if you have a different dbt project that you can run locally you can always toss that in instead

Nicholas Buck · Answer

oh

Nicholas Buck · Answer

i have a local dbt project

owen · Answer

the airbyte and postgres stuff is really just to get some sample data in

Nicholas Buck · Answer

thats what i originally wanted to get running tbh

Nicholas Buck · Answer

it uses our athena data

owen · Answer

ah that makes sense

Nicholas Buck · Answer

the dbt project works i know for sure

owen · Answer

yeah if you just point load assets from dbt project at that and update DBT CONFIG in the constants it should work

Nicholas Buck · Answer

trying to sell dagster intead of windows task scheduler for everything

Nicholas Buck · Answer

ill give it a try ty bro

owen · Answer

nice hope it works

Nicholas Buck · Answer

crazy too i was so excited when i finally had docker and airbyte running

Nicholas Buck · Answer

im sure its some dumb little thing