I am curious the best way to exclude some Dagster Airbyte st

Question

I am curious the best way to exclude some Dagster Airbyte stream configs from the application build and instead provide it as a ConfigMap I am not experienced in using ConfigMaps or Volumes but curious what the best method and allowable using Helm deployment to reference configuration yamls within the application code

daniel · Answer

Hi dusty I think you would use a volume for this <https kubernetes io docs concepts storage volumes configmap>

daniel · Answer

<https docs dagster io deployment guides kubernetes deploying with helm step 61 configure the deployment> has an example of configuring a volume and volume mount in your code you typically need to specify both its a bit finnicky

daniel · Answer

looking it over now that example yaml is a bit weird typically the volume mount would have the same name as the volume

daniel · Answer

our example is weird that is the k8s example is fine

Dusty Shapiro · Answer

+1 sounds good I ll have a look

Dusty Shapiro · Answer

So it just makes that volume data accessible in the user deployment mode in the ` etc config ` dir

daniel · Answer

yeah it d be available in any pod that dagster spins up for that code

daniel · Answer

runs too

Dusty Shapiro · Answer

+1 Sounds great will get into it Looks to be the perfect solution for the Airbyte stream configurations

Dusty Shapiro · Answer

Another question specific to Volumes Configmaps does the configMap name value under `volumes` correspond to an external ConfigMap I ve already created on my infra

Dusty Shapiro · Answer

or will that create a configMap for me with that value

daniel · Answer

the former

Dusty Shapiro · Answer

Got it thanks Got it up and wired up now I m just brainstorming on best way to push changes to the volume I assume just a redploy

Andrea Giardini · Answer

If the configmap is mounted as a volume then updating the configmap will automatically update the file in all the pods mounting it This might take some time depending on the Kubernetes configuration of your provider up to two minutes is expected Otherwise if that s not a viable solution you can always create a new configmap and point your deployment to it but this will require a rollout