https://dagster.io/ logo
#deployment-kubernetes
Title
# deployment-kubernetes
r

Rubén Lopez Lozoya

06/09/2021, 12:09 PM
Hey guys, I have a partition of ~300 items that take forever to load in Dagit (most of the times it wont even load). The partition is generated from a list of urls found in a 5MB spreadsheet, so it should not be a big problem. It takes a while to load locally, lets say 10-20 seconds, and it won't load most of the times in Kubernetes. Where should I look into in order to enhance this process? If it is a matter of resources, should I increase dagit resources, my deployment resources, or dagster daemon?
j

johann

06/09/2021, 2:05 PM
cc @prha
p

prha

06/09/2021, 3:18 PM
@Rubén Lopez Lozoya Is this on the partitions tab of the pipeline?
r

Rubén Lopez Lozoya

06/09/2021, 3:57 PM
One question, where does the partition query gets executed, in Dagit or as part of my user deployment, because in order to retrieve partition items I need to access some google secret
d

daniel

06/09/2021, 4:18 PM
Hey Ruben - it'd be the user deployment (dagit never executes your code directly)
r

Rubén Lopez Lozoya

06/09/2021, 4:19 PM
Cool, thanks, I am getting now some auth weird errors that should not be there because the secrets are part of my env variables, will check this