Browse Source

envConfigmap, restartPolicy, backoffLimit, concurrencyPolicy, history limits

tags/0.0.4
Vladimir Smagin 1 month ago
parent
commit
63815efbd4

+ 50
- 67
README.md View File

@@ -1,17 +1,28 @@
# Cron operator for Kubernetes

Manages multiple CronJobs with same image, but different commands and workdirs. Supports mounting of persistent volume to save results somewhere.
Manages multiple CronJobs with same image, but different commands and workdirs. Supports mounting of persistent volume to save results somewhere, ENV variables configmaps, concurrency and restart policy.

## Planning

- global and local SecurityContext
- global SubPath for volume mounts
- global node selector
- global resources limits
- global tolerations
- global affinity
- global imagePullPolicy

## Build sources

- You definitly need a Golang installed
- You definitly need a Golang installed ([easy install for Ubuntu](http://deb.blindage.org/readme.txt))
- Clone repository https://git.blindage.org/21h/cron-operator.git to your computer
- Open `Makefile` and change images registry to yours, then run `make && make push`
- Or run `docker build -t cron-operator .` to make image and push to your repo later
- Open `Makefile` and change images registry to yours, then run `make && make push`
Or run `docker build -t cron-operator .` to make image and push to your repo later
- Upload image to your registry

## Prebuilt images

You can find images here https://hub.docker.com/r/iam21h/cron-operator
You can find images here https://hub.docker.com/r/iam21h/cron-operator. Pin version number in `operator.yaml` if you do not want automaticaly get new features and bugs.

## Deploy operator

@@ -19,16 +30,45 @@ Cluster scoped installation:

```
kubectl -f deploy/crds/blindage_v1alpha1_cronop_crd.yaml
kubectl -f deploy/service_account.yaml
kubectl -f deploy/clusterrole.yaml
kubectl -f deploy/clusterrole_binding.yaml
kubectl -f deploy/operator.yaml
kubectl -f deploy/clusterrole_binding.yaml # service account namespace here
kubectl -n default -f deploy/service_account.yaml
kubectl -n default -f deploy/operator.yaml
```

Change `operator.yaml` if you want operator watch only specified namespace. Check `deploy` directory, may be you want to use Role instead of ClusterRole.

## Settings

In your `CronOp` resource you can set image, parameters and tasks. CronJobs is a result of reconciling tasks section, some parameters can be global and overrided in each task. Remember, CronJob, Pod and Job is a different things.

This parameters can be set in global section or each task:

- Set Configmap with ENV variables
`envConfigmap`: _default empty_

- How to restart Pod
`restartPolicy`: **OnFailure** | *Never* | *Always*

- Limit Job restarts
`backoffLimit`: **0** | integer

- Allow Cronjob to run concurrent Jobs
`concurrencyPolicy`: **Allow** | *Forbid* | *Replace*

Global history limits:

- successfulJobsHistoryLimit: **3** | integer
- failedJobsHistoryLimit: **3** | integer

Also you can save intermediate results in mounted persistent volume. You can set PVC name with `PersistentVolumeName` and mount point with `PersistentVolumePath`. It's easy.

Look into `pkg/apis/blindage/v1alpha1/cronop_types.go` to find more interesting parameters!

## Your first CronOp deployment

Good sample you can find in `deploy/test-cron.yaml`, see it in case of questions.

This resource can help you run your first tasks:

```
@@ -39,70 +79,13 @@ metadata:
name: mycron
spec:
image: "python:3"
persistentVolumeName: app-cache
persistentVolumePath: /persistent
tasks:
- name: print-test-in-pvc-dir
workdir: /persistent
- name: print-test
commands: ['python','-c', 'print("doing nothing")']
schedule: "* * * * *"
- name: print-another-test
commands: ['python','-c', 'print("koalas loves trees")']
schedule: "* * * * *"

---
apiVersion: batch/v1
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: app-cache
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
storageClassName: my-efs
```

If you delete this resource operator will delete CronJobs.

## Full Spec

```
type CronOpSpec struct {
// default busybox, override it with our image
Image string `json:"image,omitempty"`
// if you need some persistence set here to your PersistentVolumeClaim name
PersistentVolumeName string `json:"persistentVolumeName,omitempty"`
// mount path
PersistentVolumePath string `json:"persistentVolumePath,omitempty"`

// default is empty, but empty is useless ;)
Tasks []CronTask `json:"tasks,omitempty"`
}

type CronTask struct {
// use naming same as containers name, its required
Name string `json:"name"`

// default "0 * * * *", hourly
Schedule string `json:"schedule,omitempty"`

// command, can't be empty
Commands []string `json:"commands"`

// set command working directory, default empty
Workdir string `json:"workdir,omitempty"`

// default OnFailure
RestartPolicy *corev1.RestartPolicy `json:"restartPolicy,omitempty"`

// default Allow
ConcurrencyPolicy *batchv1beta1.ConcurrencyPolicy `json:"concurrencyPolicy,omitempty"`
}
```
If you delete this resource operator will delete CronJobs and Jobs.

---
Copyright by Vladimir Smagin (21h) 2019

+ 1
- 1
VERSION View File

@@ -1 +1 @@
0.0.3
0.0.4

+ 49
- 8
deploy/test-cron.yaml View File

@@ -4,16 +4,57 @@ metadata:
name: mycron
spec:
image: "python:3"

# global ENVs
envConfigmap: cron-global

# do not run same job again
concurrencyPolicy: Forbid

# do not restart job pods
restartPolicy: Never

# do not restart job
backoffLimit: 0
# history limits
failedJobsHistoryLimit: 2
successfulJobsHistoryLimit: 2
tasks:
- name: print-test
# special task with failed command
- name: var-local
# override global ENV vars with local and print
envConfigmap: cron-local
commands:
- python
- -c
- print("test")
- cat
- eating
- turd
schedule: "* * * * *"
- name: print-test2
# simple task without overrides, print only global ENV vars
- name: var-global
commands:
- python
- -c
- print("test task 2")
- printenv
schedule: "* * * * *"
# just print something on python, but shorter command syntax
- name: python-print-test
commands: ['python', '-c', 'print("test")']
schedule: "* * * * *"

# configmaps with variables for tasks
---
apiVersion: v1
kind: ConfigMap
metadata:
creationTimestamp: "2019-09-28T19:20:00Z"
name: cron-global
data:
VAR_GLOBAL: "GLOBAL"
MY_VARIABLE: "MYMY"
---
apiVersion: v1
kind: ConfigMap
metadata:
creationTimestamp: "2019-09-28T19:20:00Z"
name: cron-local
data:
VAR_LOCAL: "LOCAL"
MY_VARIABLE: "OVERRIDE"

+ 10
- 0
go.sum View File

@@ -94,6 +94,7 @@ github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMo
github.com/getsentry/raven-go v0.2.0 h1:no+xWJRb5ZI7eE8TWgIq1jLulQiIoLG0IfYxv5JYMGs=
github.com/getsentry/raven-go v0.2.0/go.mod h1:KungGk8q33+aIAZUIVWZDr2OfAEBsO49PX4NzFV5kcQ=
github.com/ghodss/yaml v0.0.0-20150909031657-73d445a93680/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/gliderlabs/ssh v0.1.1/go.mod h1:U7qILu1NlMHj9FlMhZLlkCdDnU1DBEAqr0aevW3Awn0=
github.com/globalsign/mgo v0.0.0-20180905125535-1ca0a4f7cbcb/go.mod h1:xkRDCp4j0OGD1HRkm4kmhM+pmpv3AKq5SU7GMg4oO/Q=
@@ -132,6 +133,7 @@ github.com/go-openapi/validate v0.17.2/go.mod h1:Uh4HdOzKt19xGIGm1qHf/ofbX1YQ4Y+
github.com/go-openapi/validate v0.18.0/go.mod h1:Uh4HdOzKt19xGIGm1qHf/ofbX1YQ4Y+MYsct2VUrAJ4=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gobuffalo/envy v1.6.5/go.mod h1:N+GkhhZ/93bGZc6ZKhJLP6+m+tCNPKwgSpH9kaifseQ=
github.com/gobuffalo/envy v1.6.15 h1:OsV5vOpHYUpP7ZLS6sem1y40/lNX1BZj+ynMiRi21lQ=
github.com/gobuffalo/envy v1.6.15/go.mod h1:n7DRkBerg/aorDM8kbduw5dN3oXGswK5liaSCx4T5NI=
github.com/gobwas/glob v0.2.3/go.mod h1:d3Ez4x06l9bZtSvzIay5+Yzi0fmZzPgnTbPcKjJAkT8=
github.com/gogo/protobuf v0.0.0-20170330071051-c0656edd0d9e/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
@@ -207,6 +209,7 @@ github.com/imdario/mergo v0.3.7 h1:Y+UAYTZ7gDEuOfhxKWy+dvb5dRQ6rJjFSdX2HZY1/gI=
github.com/imdario/mergo v0.3.7/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/jellevandenhooff/dkim v0.0.0-20150330215556-f50fe3d243e1/go.mod h1:E0B/fFc00Y+Rasa88328GlI/XbtyysCtTHZS8h7IrBU=
github.com/joho/godotenv v1.3.0 h1:Zjp+RcGpHhGlrMbJzXTrZZPrWj+1vfm90La1wgB6Bhc=
github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/json-iterator/go v0.0.0-20180612202835-f2b4162afba3/go.mod h1:+SdeFBvtyEkXs7REEP0seUULqWtbJapLOCVDaaPEHmU=
@@ -231,6 +234,7 @@ github.com/mailru/easyjson v0.0.0-20160728113105-d5b7844b561a/go.mod h1:C1wdFJiN
github.com/mailru/easyjson v0.0.0-20180823135443-60711f1a8329/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/mailru/easyjson v0.0.0-20190403194419-1ea4449da983 h1:wL11wNW7dhKIcRCHSm4sHKPWz0tt4mwBsVodG7+Xyqg=
github.com/mailru/easyjson v0.0.0-20190403194419-1ea4449da983/go.mod h1:C1wdFJiN94OJF2b5HbByQZoLdCWB1Yqtg26g4irojpc=
github.com/markbates/inflect v1.0.4 h1:5fh1gzTFhfae06u3hzHYO9xe3l3v3nW5Pwt3naLTP5g=
github.com/markbates/inflect v1.0.4/go.mod h1:1fR9+pO2KHEO9ZRtto13gDwwZaAKstQzferVeWqbgNs=
github.com/martinlindhe/base36 v0.0.0-20180729042928-5cda0030da17/go.mod h1:+AtEs8xrBpCeYgSLoY/aJ6Wf37jtBuR0s35750M27+8=
github.com/mattbaird/jsonpatch v0.0.0-20171005235357-81af80346b1a/go.mod h1:M1qoD/MqPgTZIk0EWKB38wE28ACRfVcn+cU08jyArI0=
@@ -309,6 +313,7 @@ github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6So
github.com/rogpeppe/go-charset v0.0.0-20180617210344-2471d30d28b4/go.mod h1:qgYeAmZ5ZIpBWTGllZSQnw97Dj+woV0toclVaRGI8pc=
github.com/rogpeppe/go-internal v1.1.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.2.2/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/rogpeppe/go-internal v1.3.0 h1:RR9dF3JtopPvtkroDZuVD7qquD0bnHlKSqaQhgwt8yk=
github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFRclV5y23lUDJ4=
github.com/russross/blackfriday v0.0.0-20151117072312-300106c228d5/go.mod h1:JO/DiYxRf+HjHt06OyowR9PTA263kcR/rfWxYHBV53g=
github.com/sclevine/spec v1.0.0/go.mod h1:W4J29eT/Kzv7/b9IWLB055Z+qvVC9vt0Arko24q7p+U=
@@ -319,6 +324,7 @@ github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPx
github.com/sirupsen/logrus v1.4.1/go.mod h1:ni0Sbl8bgC9z8RoU9G6nDWqqs/fq4eDPysMBDgk/93Q=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/afero v1.2.2 h1:5jhuqJyZCZf2JRofRvN/nIFgIWNzPa3/Vz8mYylgbWc=
github.com/spf13/afero v1.2.2/go.mod h1:9ZxEEn6pIJ8Rxe320qSDBk6AsU0r9pR7Q4OcevTdifk=
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cobra v0.0.3/go.mod h1:1l0Ry5zgKvJasoi3XT1TypsSe7PqH0Sj9dhYf7v3XqQ=
@@ -442,6 +448,7 @@ golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGm
golang.org/x/tools v0.0.0-20190213015956-f7e1b50d2251/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190408170212-12dd9f86f350 h1:0USRhKWpISljvJE8egltEaoJb+VD0IUA4eOH6W1yss8=
golang.org/x/tools v0.0.0-20190408170212-12dd9f86f350/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
google.golang.org/api v0.0.0-20180910000450-7ca32eb868bf/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0=
google.golang.org/api v0.0.0-20181030000543-1d582fd0359e/go.mod h1:4mhQ8q/RsB7i+udVvVy5NUi08OU8ZlA0gRVgrF7VFY0=
@@ -493,6 +500,7 @@ honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWh
honnef.co/go/tools v0.0.0-20190106161140-3f1c8253044a/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
k8s.io/api v0.0.0-20190222213804-5cb15d344471 h1:MzQGt8qWQCR+39kbYRd0uQqsvSidpYqJLFeWiJ9l4OE=
k8s.io/api v0.0.0-20190222213804-5cb15d344471/go.mod h1:iuAfoD4hCxJ8Onx9kaTIt30j7jUFS00AXQi6QMi99vA=
k8s.io/apiextensions-apiserver v0.0.0-20190228180357-d002e88f6236 h1:JfFtjaElBIgYKCWEtYQkcNrTpW+lMO4GJy8NP6SVQmM=
k8s.io/apiextensions-apiserver v0.0.0-20190228180357-d002e88f6236/go.mod h1:IxkesAMoaCRoLrPJdZNZUQp9NfZnzqaVzLhb2VEQzXE=
k8s.io/apimachinery v0.0.0-20190221213512-86fb29eff628 h1:UYfHH+KEF88OTg+GojQUwFTNxbxwmoktLwutUzR0GPg=
k8s.io/apimachinery v0.0.0-20190221213512-86fb29eff628/go.mod h1:ccL7Eh7zubPUSh9A3USN90/OzHNSVN6zxzde07TDCL0=
@@ -505,6 +513,7 @@ k8s.io/code-generator v0.0.0-20181203235156-f8cba74510f3/go.mod h1:MYiN+ZJZ9HkET
k8s.io/gengo v0.0.0-20181106084056-51747d6e00da/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20181113154421-fd15ee9cc2f7/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20190128074634-0689ccc1d7d6/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/gengo v0.0.0-20190327210449-e17681d19d3a h1:QoHVuRquf80YZ+/bovwxoMO3Q/A3nt3yTgS0/0nejuk=
k8s.io/gengo v0.0.0-20190327210449-e17681d19d3a/go.mod h1:ezvh/TsK7cY6rbqRK0oQQ8IAqLxYwwyPxAX1Pzy0ii0=
k8s.io/helm v2.13.1+incompatible/go.mod h1:LZzlS4LQBHfciFOurYBFkCMTaZ0D1l+p0teMg7TSULI=
k8s.io/klog v0.0.0-20181102134211-b9b56d5dfc92/go.mod h1:Gq+BEi5rUBO/HRz0bTSXDUcqjScdoY3a9IHpCEIOOfk=
@@ -525,6 +534,7 @@ k8s.io/kubernetes v1.11.8-beta.0.0.20190124204751-3a10094374f2/go.mod h1:ocZa8+6
k8s.io/utils v0.0.0-20190308190857-21c4ce38f2a7/go.mod h1:8k8uAuAQ0rXslZKaEWd0c3oVhZz7sSzSiPnVZayjIX0=
sigs.k8s.io/controller-runtime v0.1.12 h1:ovDq28E64PeY1yR+6H7DthakIC09soiDCrKvfP2tPYo=
sigs.k8s.io/controller-runtime v0.1.12/go.mod h1:HFAYoOh6XMV+jKF1UjFwrknPbowfyHEHHRdJMf2jMX8=
sigs.k8s.io/controller-tools v0.1.11-0.20190411181648-9d55346c2bde h1:ZkaHf5rNYzIB6CB82keKMQNv7xxkqT0ylOBdfJPfi+k=
sigs.k8s.io/controller-tools v0.1.11-0.20190411181648-9d55346c2bde/go.mod h1:ATWLRP3WGxuAN9HcT2LaKHReXIH+EZGzRuMHuxjXfhQ=
sigs.k8s.io/structured-merge-diff v0.0.0-20190525122527-15d366b2352e/go.mod h1:wWxsB5ozmmv/SG7nM11ayaAW51xMvak/t1r0CSlcokI=
sigs.k8s.io/testing_frameworks v0.1.1/go.mod h1:VVBKrHmJ6Ekkfz284YKhQePcdycOzNH9qL6ht1zEr/U=

+ 14
- 6
pkg/apis/blindage/v1alpha1/cronop_types.go View File

@@ -14,15 +14,23 @@ type CronTask struct {
Schedule string `json:"schedule,omitempty"` // default "0 * * * *", hourly
Workdir string `json:"workdir,omitempty"` // default empty
Commands []string `json:"commands"`
RestartPolicy *corev1.RestartPolicy `json:"restartPolicy,omitempty"` // default OnFailure
ConcurrencyPolicy *batchv1beta1.ConcurrencyPolicy `json:"concurrencyPolicy,omitempty"` // default Allow
RestartPolicy *corev1.RestartPolicy `json:"restartPolicy,omitempty"` // default OnFailure, overrides global policy
ConcurrencyPolicy *batchv1beta1.ConcurrencyPolicy `json:"concurrencyPolicy,omitempty"` // default Allow, overrides global policy
EnvConfigmap string `json:"envConfigmap,omitempty"` // default empty, overrides global configmap
BackoffLimit *int32 `json:"backoffLimit,omitempty"` // default 0, overrides global limit
}

type CronOpSpec struct {
Image string `json:"image,omitempty"` // default busybox
PersistentVolumeName string `json:"persistentVolumeName,omitempty"` // default emtpy, if present mounts
PersistentVolumePath string `json:"persistentVolumePath,omitempty"` // default emtpy
Tasks []CronTask `json:"tasks,omitempty"` // default empty
RestartPolicy *corev1.RestartPolicy `json:"restartPolicy,omitempty"` // default OnFailure, global
ConcurrencyPolicy *batchv1beta1.ConcurrencyPolicy `json:"concurrencyPolicy,omitempty"` // default Allow, global
SuccessfulJobsHistoryLimit *int32 `json:"successfulJobsHistoryLimit,omitempty"` // default 3
FailedJobsHistoryLimit *int32 `json:"failedJobsHistoryLimit,omitempty"` // default 3
BackoffLimit *int32 `json:"backoffLimit,omitempty"` // default 0, global
Image string `json:"image,omitempty"` // default busybox
PersistentVolumeName string `json:"persistentVolumeName,omitempty"` // default emtpy, must be created before use
PersistentVolumePath string `json:"persistentVolumePath,omitempty"` // default emtpy
EnvConfigmap string `json:"envConfigmap,omitempty"` // default empty, global
Tasks []CronTask `json:"tasks,omitempty"` // default empty
}

// CronOpStatus defines the observed state of CronOp

+ 30
- 0
pkg/apis/blindage/v1alpha1/zz_generated.deepcopy.go View File

@@ -74,6 +74,31 @@ func (in *CronOpList) DeepCopyObject() runtime.Object {
// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
func (in *CronOpSpec) DeepCopyInto(out *CronOpSpec) {
*out = *in
if in.RestartPolicy != nil {
in, out := &in.RestartPolicy, &out.RestartPolicy
*out = new(v1.RestartPolicy)
**out = **in
}
if in.ConcurrencyPolicy != nil {
in, out := &in.ConcurrencyPolicy, &out.ConcurrencyPolicy
*out = new(v1beta1.ConcurrencyPolicy)
**out = **in
}
if in.SuccessfulJobsHistoryLimit != nil {
in, out := &in.SuccessfulJobsHistoryLimit, &out.SuccessfulJobsHistoryLimit
*out = new(int32)
**out = **in
}
if in.FailedJobsHistoryLimit != nil {
in, out := &in.FailedJobsHistoryLimit, &out.FailedJobsHistoryLimit
*out = new(int32)
**out = **in
}
if in.BackoffLimit != nil {
in, out := &in.BackoffLimit, &out.BackoffLimit
*out = new(int32)
**out = **in
}
if in.Tasks != nil {
in, out := &in.Tasks, &out.Tasks
*out = make([]CronTask, len(*in))
@@ -128,6 +153,11 @@ func (in *CronTask) DeepCopyInto(out *CronTask) {
*out = new(v1beta1.ConcurrencyPolicy)
**out = **in
}
if in.BackoffLimit != nil {
in, out := &in.BackoffLimit, &out.BackoffLimit
*out = new(int32)
**out = **in
}
return
}


+ 174
- 0
pkg/controller/cronop/cronjob.go View File

@@ -0,0 +1,174 @@
package cronop

import (
"github.com/go-logr/logr"
batchv1beta1 "k8s.io/api/batch/v1beta1"

blindagev1alpha1 "git.blindage.org/21h/cron-operator/pkg/apis/blindage/v1alpha1"
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// generate one cronjob for specific task
func generateCronjob(reqLogger logr.Logger, cr *blindagev1alpha1.CronOp, jobSpec blindagev1alpha1.CronTask) batchv1beta1.CronJob {
labels := map[string]string{
"operator": "cron-operator",
"cronop": cr.Name,
}

imageName := "busybox"
if cr.Spec.Image != "" {
reqLogger.Info("Found Image override", "Namespace", cr.Namespace, "Name", cr.Name, "Image", cr.Spec.Image)
imageName = cr.Spec.Image
}

cronjob := batchv1beta1.CronJob{
ObjectMeta: metav1.ObjectMeta{
Name: cr.Name + "-" + jobSpec.Name,
Namespace: cr.Namespace,
Labels: labels,
},
Spec: batchv1beta1.CronJobSpec{},
}

cronjob.Spec.Schedule = "0 * * * *"
if jobSpec.Schedule != "" {
cronjob.Spec.Schedule = jobSpec.Schedule
}

workdir := ""
if jobSpec.Workdir != "" {
workdir = jobSpec.Workdir
}

//
// set backoff limit
//

// default backoff limit
backoffLimit := int32(0)

// override default limit with global
if cr.Spec.BackoffLimit != nil {
backoffLimit = *cr.Spec.BackoffLimit
}

// override default limit with global
if jobSpec.BackoffLimit != nil {
backoffLimit = *jobSpec.BackoffLimit
}

cronjob.Spec.JobTemplate.Spec.BackoffLimit = &backoffLimit

//
// restart policy
//

// default restart policy
cronjob.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = corev1.RestartPolicyOnFailure

// override default policy from global spec
if cr.Spec.RestartPolicy != nil {
cronjob.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = *cr.Spec.RestartPolicy
}

// override global policy from task spec
if jobSpec.RestartPolicy != nil {
cronjob.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = *jobSpec.RestartPolicy
}

//
// concurrency policy
//

// default concurrency policy
cronjob.Spec.ConcurrencyPolicy = batchv1beta1.AllowConcurrent

// override default policy from global spec
if cr.Spec.ConcurrencyPolicy != nil {
cronjob.Spec.ConcurrencyPolicy = *cr.Spec.ConcurrencyPolicy
}

// override global policy from task spec
if jobSpec.ConcurrencyPolicy != nil {
cronjob.Spec.ConcurrencyPolicy = *jobSpec.ConcurrencyPolicy
}

//
// ENV variables
//

envs := []corev1.EnvFromSource{}
// add global ENVs
if cr.Spec.EnvConfigmap != "" {
envsGlobal := corev1.EnvFromSource{
ConfigMapRef: &corev1.ConfigMapEnvSource{
LocalObjectReference: corev1.LocalObjectReference{Name: cr.Spec.EnvConfigmap},
},
}
envs = append(envs, envsGlobal)
}

// override global ENVs with local for task
if jobSpec.EnvConfigmap != "" {
envsTask := corev1.EnvFromSource{
ConfigMapRef: &corev1.ConfigMapEnvSource{
LocalObjectReference: corev1.LocalObjectReference{Name: jobSpec.EnvConfigmap},
},
}
envs = append(envs, envsTask)
}

//
// history limits
//

SuccessfulJobsHistoryLimit := int32(3)
if cr.Spec.SuccessfulJobsHistoryLimit != nil {
SuccessfulJobsHistoryLimit = *cr.Spec.SuccessfulJobsHistoryLimit
}
cronjob.Spec.SuccessfulJobsHistoryLimit = &SuccessfulJobsHistoryLimit

FailedJobsHistoryLimit := int32(3)
if cr.Spec.FailedJobsHistoryLimit != nil {
FailedJobsHistoryLimit = *cr.Spec.FailedJobsHistoryLimit
}

cronjob.Spec.FailedJobsHistoryLimit = &FailedJobsHistoryLimit

jobContainer := corev1.Container{
Name: "crontask",
Image: imageName,
Command: jobSpec.Commands,
WorkingDir: workdir,
}

jobContainer.EnvFrom = envs

// If persistence is present
if cr.Spec.PersistentVolumeName != "" {
// Attach PVC to Pod
cronjob.Spec.JobTemplate.Spec.Template.Spec.Volumes = []corev1.Volume{
{
Name: "persistent-volume",
VolumeSource: corev1.VolumeSource{
PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{
ClaimName: cr.Spec.PersistentVolumeName,
},
},
},
}

// Set mount point
jobContainer.VolumeMounts = []corev1.VolumeMount{
{
Name: "persistent-volume",
MountPath: cr.Spec.PersistentVolumePath,
},
}
}

cronjob.Spec.JobTemplate.Spec.Template.Spec.Containers = []corev1.Container{jobContainer}

return cronjob
}

+ 0
- 121
pkg/controller/cronop/cronop_controller.go View File

@@ -2,17 +2,13 @@ package cronop

import (
"context"
"reflect"

"github.com/getsentry/raven-go"
"github.com/go-logr/logr"
batchv1beta1 "k8s.io/api/batch/v1beta1"
"k8s.io/apimachinery/pkg/labels"

blindagev1alpha1 "git.blindage.org/21h/cron-operator/pkg/apis/blindage/v1alpha1"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/api/errors"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/types"
"sigs.k8s.io/controller-runtime/pkg/client"
@@ -27,11 +23,6 @@ import (

var log = logf.Log.WithName("controller_cronop")

/**
* USER ACTION REQUIRED: This is a scaffold file intended for the user to modify with their own Controller
* business logic. Delete these comments after modifying this file.*
*/

// Add creates a new CronOp Controller and adds it to the Manager. The Manager will set fields on the Controller
// and Start it when the Manager is Started.
func Add(mgr manager.Manager) error {
@@ -168,115 +159,3 @@ func (r *ReconcileCronOp) Reconcile(request reconcile.Request) (reconcile.Result
reqLogger.Info("Reconcile complete", "Namespace", instance.Namespace, "Name", instance.Name)
return reconcile.Result{}, nil
}

// decide to reconcile CronJob
func reconcileCronJob(found batchv1beta1.CronJob, new batchv1beta1.CronJob) (bool, batchv1beta1.CronJob) {

reconcileRequired := false

if !reflect.DeepEqual(found.Spec.Schedule, new.Spec.Schedule) {
found.Spec.Schedule = new.Spec.Schedule
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.ConcurrencyPolicy, new.Spec.ConcurrencyPolicy) {
found.Spec.ConcurrencyPolicy = new.Spec.ConcurrencyPolicy
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy, new.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy) {
found.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = new.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.JobTemplate.Spec, new.Spec.JobTemplate.Spec) {
found.Spec.JobTemplate.Spec = new.Spec.JobTemplate.Spec
reconcileRequired = true
}

return reconcileRequired, found

}

// generate one cronjob for specific task
func generateCronjob(reqLogger logr.Logger, cr *blindagev1alpha1.CronOp, jobSpec blindagev1alpha1.CronTask) batchv1beta1.CronJob {
labels := map[string]string{
"operator": "cron-operator",
"cronop": cr.Name,
}

imageName := "busybox"
if cr.Spec.Image != "" {
reqLogger.Info("Found Image override", "Namespace", cr.Namespace, "Name", cr.Name, "Image", cr.Spec.Image)
imageName = cr.Spec.Image
}

cronjob := batchv1beta1.CronJob{
ObjectMeta: metav1.ObjectMeta{
Name: cr.Name + "-" + jobSpec.Name,
Namespace: cr.Namespace,
Labels: labels,
},
Spec: batchv1beta1.CronJobSpec{},
}

cronjob.Spec.Schedule = "0 * * * *"
if jobSpec.Schedule != "" {
//reqLogger.Info("Found Schedule override", "Namespace", cr.Namespace, "Name", cr.Name, "Schedule", jobSpec.Schedule)
cronjob.Spec.Schedule = jobSpec.Schedule
}

workdir := ""
if jobSpec.Workdir != "" {
//reqLogger.Info("Found Workdir override", "Namespace", cr.Namespace, "Name", cr.Name, "Workdir", jobSpec.Workdir)
workdir = jobSpec.Workdir
}

cronjob.Spec.ConcurrencyPolicy = batchv1beta1.AllowConcurrent
if jobSpec.ConcurrencyPolicy != nil {
//reqLogger.Info("Found Concurrency override", "Namespace", cr.Namespace, "Name", cr.Name, "Concurrency", jobSpec.ConcurrencyPolicy)
cronjob.Spec.ConcurrencyPolicy = *jobSpec.ConcurrencyPolicy
}

cronjob.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = corev1.RestartPolicyOnFailure
if jobSpec.RestartPolicy != nil {
//reqLogger.Info("Found RestartPolicy override", "Namespace", cr.Namespace, "Name", cr.Name, "RestartPolicy", jobSpec.RestartPolicy)
cronjob.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = *jobSpec.RestartPolicy
}

cronjob.Spec.JobTemplate.Spec.Template.Spec.Containers = []corev1.Container{
corev1.Container{
Name: "crontask",
Image: imageName,
Command: jobSpec.Commands,
WorkingDir: workdir,
},
}

// If persistence is present
if cr.Spec.PersistentVolumeName != "" {
//reqLogger.Info("Persistence is present", "Namespace", cr.Namespace, "Name", cr.Name, "Persistent volume name", cr.Spec.PersistentVolumeName)

// Attach PVC to Pod
cronjob.Spec.JobTemplate.Spec.Template.Spec.Volumes = []corev1.Volume{
{
Name: "persistent-volume",
VolumeSource: corev1.VolumeSource{
PersistentVolumeClaim: &corev1.PersistentVolumeClaimVolumeSource{
ClaimName: cr.Spec.PersistentVolumeName,
},
},
},
}

// Set mount point
cronjob.Spec.JobTemplate.Spec.Template.Spec.Containers[0].VolumeMounts = []corev1.VolumeMount{
{
Name: "persistent-volume",
MountPath: cr.Spec.PersistentVolumePath,
},
}
}

return cronjob
}

+ 46
- 0
pkg/controller/cronop/reconcile.go View File

@@ -0,0 +1,46 @@
package cronop

import (
"reflect"

batchv1beta1 "k8s.io/api/batch/v1beta1"
)

// decide to reconcile CronJob
func reconcileCronJob(found batchv1beta1.CronJob, new batchv1beta1.CronJob) (bool, batchv1beta1.CronJob) {

reconcileRequired := false

if !reflect.DeepEqual(found.Spec.Schedule, new.Spec.Schedule) {
found.Spec.Schedule = new.Spec.Schedule
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.ConcurrencyPolicy, new.Spec.ConcurrencyPolicy) {
found.Spec.ConcurrencyPolicy = new.Spec.ConcurrencyPolicy
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy, new.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy) {
found.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy = new.Spec.JobTemplate.Spec.Template.Spec.RestartPolicy
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.JobTemplate.Spec, new.Spec.JobTemplate.Spec) {
found.Spec.JobTemplate.Spec = new.Spec.JobTemplate.Spec
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.SuccessfulJobsHistoryLimit, new.Spec.SuccessfulJobsHistoryLimit) {
found.Spec.SuccessfulJobsHistoryLimit = new.Spec.SuccessfulJobsHistoryLimit
reconcileRequired = true
}

if !reflect.DeepEqual(found.Spec.FailedJobsHistoryLimit, new.Spec.FailedJobsHistoryLimit) {
found.Spec.FailedJobsHistoryLimit = new.Spec.FailedJobsHistoryLimit
reconcileRequired = true
}

return reconcileRequired, found

}

+ 1
- 1
version/version.go View File

@@ -1,5 +1,5 @@
package version

var (
Version = "0.0.3"
Version = "0.0.4"
)

Loading…
Cancel
Save