https://dagster.io/ logo
Join Slack
Powered by
hey! i'm having difficulty getting my k8s deployme...
# deployment-kubernetes
b
hey! i'm having difficulty getting my k8s deployment to properly start the scheduler. i have a standard version of the helm 
values.yaml
file with 
k8sEnabled
set to 
false
in the scheduling section. my 
dagster.yaml
only contains this section:
Copy code
scheduler:
  module: dagster_cron.cron_scheduler
  class: SystemCronScheduler
The deploy seems to work fine and i can access the dagit instance, and it properly imports my scheduled pipelines, but the scheduler doesn't appear to be running
should i be running 
dagster schedule up
somewhere in my user code deployment image?
a
you are using 
userDeployments.enabled=true
? or did you set the 
dagit
image
you may be better suited to setting 
k8sEnabled
to true and using the 
K8sScheduler
- my best guess for what is going wrong is that 
cron
is not running on the 
dagit
pod
b
yes, i'm using 
userDeployments
and it seems that bit is working properly as my pipelines and schedules show up in the interface
im just using 
repository: "dagster/k8s-dagit-example"
for 
dagit
a
hm alright ya I think your best bet is to use the 
K8sScheduler
instead of 
cron
the helm chart is in kind of an awkward place (will improve next week with 
0.10.0
) where when user deployments are turned on cron isn’t started for you
Copy code
scheduler:
  k8sEnabled: true
  image:
    repository: "dagster/k8s-dagit"
^ there is a bug in the current published chart so you’ll need to set the image repository
b
okay, i'll try it alex, thanks!
sorry, do you mean i shouldn't use 
latest
?
a
no the repository is set to 
dagster/k8s-dagster
by default which doesn’t exist
b
ahh, i see
okay, that seemed to work! i'm now getting an error that seems to suggest i need to properly configure scheduler storage:
Copy code
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNKNOWN
	details = "Exception calling application: Failure condition: Couldn't import module dagster_postgres.run_storage when attempting to load the configurable class dagster_postgres.run_storage.PostgresRunStorage"
	debug_error_string = "{"created":"@1610042430.266011933","description":"Error received from peer ipv4:10.245.184.71:3030","file":"src/core/lib/surface/call.cc","file_line":1062,"grpc_message":"Exception calling application: Failure condition: Couldn't import module dagster_postgres.run_storage when attempting to load the configurable class dagster_postgres.run_storage.PostgresRunStorage","grpc_status":2}"
i assume that is something i would do in 
dagster.yaml
correct?
a
uhh this might actually just mean that you need to install 
dagster_postgres
in your user deployment image
b
okay, that makes sense actually, as im seeing the same error elsewhere.
thank you
if i just restart the 
dagit
pod will it re-pull my user code deployment? I have it set to pull from 
latest
and that's deployed via CI.
what's the best practice for registering user code deployment changes?
a
there is a reload button in dagit you can press to refetch the metadata from the user deployment server when it gets redeployed
in 
0.10.0
that should come out next week we automatically detect and refresh for you
b
that's cool!
final question (i hope). if i choose to run the k8s configuration without celery (using the k8s job runner) can i forgo redis/rabbitmq?
a
yep
b
amazing
Open in Slack
PreviousNext
Powered by