This seems to be the case -- attached graphs shows number of jobs on the y axis and time on the x axis.
I tried modifying
K8S_JOB_TTL_SECONDS_AFTER_FINISHED = 5 # 5 seconds
in my user deployment
but it seems to not be the right place for this as the jobs are still sticking around.
I switched the daemon to use the user deployment image and jobs are now deleting as expected.. I will report back in a few hours with results
I also manually ran a job with the following manifest to confirm that ttl feature was enabled in my cluster and the job deleted as expected.
- name: pi
command: ["perl", "-Mbignum=bpi", "-wle", "print bpi(2000)"]