Akshat Nigam08/04/2021, 8:27 PM
DBT needs it for its SQL Models. Now, as far as, Dagster is running in
mode, they (Dagster & DBT Solids) can easily access the EFS, as: 1. We have mounted EFS to main Dagster Deployment. 2. All solids will be executed in same Dagster Deployment as a system process, which will also has the access to EFS. But the question comes, when Dagster will be running in
mode, where each solid is executed in a separate ephemeral kubernetes pod.
_*InBecause, the EFS PV/PVC changes are done for main dagster deployment. I am not that much expert in k8s, but is it possible, if those ephemeral kubernetes pod uses the same main helm chart image as base image, which already has a mount with EFS, the DBTs will have access to EFS as well? It will be really great if someone can give some feedback on this. Thanks.mode, will DBT solids which will get executed in a separate ephemeral k8s pods, have access to the EFS?*_