DevOps di applicazioni Python(e non solo) su OpenShiftFrancesco Fiore, System ArchitectPyCon Nove, 20 aprile 2018

2

Agenda

• Cos’è OpenShift• Cenni sull’architettura• Setup di una semplice applicazione• OpenShift per applicazioni Python: source-to-image

• Dietro le quinte: API objects• Parametrizzare il setup di applicazioni usando i Template

• Git, OpenShift e Jenkins: integrazione nativa per la CI/CD• Application lifecycle management• Application promotion• Strategie di deployment

3

OpenShift overview

• Platform as a Service di Red Hat• OpenShift Origin è un progetto opensource• Basato su:

– Kubernetes (CaaS)– Docker– Atomic

• Add-on rispetto a k8s– API objects– Web console– Software Defined Network– Docker registry– logging centralizzato

• Focus su application lifecycle management e automation• Red Hat OpenShift Container Platform

4

OpenShift vs Kubernetes

Source to Image (S2I)

IntegratedRegistry

BuildConfiguration Image Stream Deployment

Configuration

PersistentVolume Claims

Replication Controller Replica Sets Daemon Sets Persistent

Volumes

Secrets Config Maps Services RoutesSoftware Defined

Network (SDN)

Web console Users and Groups Projects Namespaces

OpenShift Kubernetes

5

OpenShift: architettura

6

Setup di una applicazione

oc new-project demooc new-app https://github.com/ffiore/devops-on-openshift-demo.git

--context-dir samples/flask--name flask-sample

• OpenShift rileva automaticamente qual è la strategy di build– if Jenkinsfile -> Pipeline strategy– elif Dockerfile -> Docker strategy– else -> Source strategy

File Linguaggio

requirements.txt, setup.py python

pom.xml jee

app.json, package.json nodejs

Godeps, main.go golang

cpanfile, index.pl perl

composer.json, index.php php

Gemfile, Rakefile, config.ru ruby

build.sbt scala

• Se Source strategy, OpenShiftrileva il linguaggio

7

Setup di una applicazione

oc new-app https://github.com/ffiore/devops-on-openshift-demo.git--context-dir samples/flask--name flask-sample--env WELCOME_MESSAGE=‘Hello Pycon9!’--build-env HTTP_PROXY=‘http://x.y.z.w:8080’

oc new-app python:2.7~https://github.com/ffiore/devops-on-openshift-demo.git

--context-dir samples/flask--name flask-sample-python27--env WELCOME_MESSAGE=‘Hello Pycon9!’

oc expose svc flask-sample

• Per esporre l’applicazione all’esterno (Route)

8

Source-to-image: Python

• s2i è la modalità utilizzata con Source strategy

• Nessuna modifica all’applicazione– sufficiente requirements.txt

• 4 modalità di esecuzione– Gunicorn

• se in requirements.txt o in setup.py (install_requires)

• entrypoint: wsgi.py (wsgi:application) o APP_MODULE

– Django development server

– Python script• APP_FILE, default app.py

– Script file• APP_SCRIPT, default app.sh

9

Setup di una applicazione: API objects

• BuildConfig– Build

• ImageStream– ImageStreamTag

• DeploymentConfig– ReplicationController

• Pod

• Service

spec:output:to:kind: ImageStreamTagname: flask-sample:latest

source:git:uri: https://github.com/ffiore/devops-on-

openshift-democontextDir: samples/flasktype: Git

strategy:sourceStrategy:from:kind: ImageStreamTagname: python:3.5namespace: openshift

type: Sourcetriggers:- github:

secret: iopmuY9undV5grr7Z8zVtype: GitHub

- generic:secret: QOhldIxeE2ciwomAgoY9

type: Generic- type: ConfigChange- type: ImageChange

apiVersion: v1kind: BuildConfigmetadata:annotations:openshift.io/generated-by: OpenShiftNewApp

labels:app: flask-sample

name: flask-samplenamespace: demo...

10

Setup di una applicazione: API objects

• BuildConfig– Build

• ImageStream– ImageStreamTag

• DeploymentConfig– ReplicationController

• Pod

• Service

apiVersion: v1kind: ImageStreammetadata:annotations:openshift.io/generated-by: OpenShiftNewApp

labels:app: flask-sample

name: flask-samplenamespace: demo

spec: {}status:dockerImageRepository:

172.30.0.128:5000/demo/flask-sample

11

Setup di una applicazione: API objects

• BuildConfig– Build

• ImageStream– ImageStreamTag

• DeploymentConfig– ReplicationController

• Pod

• Service

template: metadata:...labels:app: flask-sampledeploymentconfig: flask-sample

spec:containers:- image: flask-sample:latestimagePullPolicy: Alwaysname: flask-sampleports:- containerPort: 8080protocol: TCP

dnsPolicy: ClusterFirstrestartPolicy: AlwaysterminationGracePeriodSeconds: 30

triggers:- type: ConfigChange- imageChangeParams:

automatic: truecontainerNames:- flask-samplefrom:kind: ImageStreamTagname: flask-sample:latestnamespace: demo

type: ImageChange

apiVersion: v1kind: DeploymentConfigmetadata:name: flask-samplenamespace: demo

spec:replicas: 1strategy:rollingParams:...

type: Rolling

12

Setup di una applicazione: API objects

• BuildConfig– Build

• ImageStream– ImageStreamTag

• DeploymentConfig– ReplicationController

• Pod

• Service

apiVersion: v1kind: Servicemetadata:annotations:openshift.io/generated-by: OpenShiftNewApp

labels:app: flask-sample

name: flask-samplenamespace: demo

spec:clusterIP: 172.30.82.11ports:- name: 8080-tcpport: 8080protocol: TCPtargetPort: 8080

selector:app: flask-sampledeploymentconfig: flask-sample

type: ClusterIP

13

Creare un Template

• Aggiungere altri API object– ConfigMap

– Secret

– PersistentVolumeClaim

– ...

• Personalizzare gli oggetti– consumare ConfigMap, Secret, PVC, ecc.

– creare legami tra componenti applicativi (es. applicazione + database)

• Parametrizzare gli oggetti

• Riutilizzare il tutto– su project/ambienti diversi

– per applicazioni diverse

14

Usare i Template

oc process –f python-flask-sample-s2i.yaml

-p APPLICATION_NAME=‘welcome-app’

-p SOURCE_REPOSITORY_URL=‘https://github.com/ffiore/devops-on-openshift-demo’

-p CONTEXT_DIR=‘samples/flask’

-p WELCOME_MESSAGE=‘Hello Pycon9!’ –o yaml | oc create –f -

• Creare l’applicazione usando il Template

• Inserire un Template «a catalogo»

oc –n devops create –f django-postgresql-persistent.yaml

• Usare un Template «a catalogo»

oc –n devops process django-psql-persistent

-p NAME=‘django-app’

-p SOURCE_REPOSITORY_URL=‘https://github.com/sclorg/django-ex’

| oc –n demo create –f -

15

Automatizzare con Jenkins

• BuildConfig possono avere strategy jenkinsPipelineStrategy– jenkinsfile– webhook per il triggering da Git

• OpenShift si aspetta di trovare una istanza di Jenkins– creazione della pipeline in Jenkins– creazione di un Build

• In alternativa, jenkins-ephemeral• Jenkins

– autoconfigurazione al primo deploy• puntamenti a Kubernetes• provisioning pipeline esistenti

– openshift jenkins plugin (deprecato da OpenShift 3.7)– openshift jenkins client plugin (GA da OpenShift 3.7)

• Parametrizzare usando Template– automatic setup e CI/CD

16

Lifecycle management

• git flow/gitlabflow/...

• github/genericwebhook

gitpush/merge

• build applicazione• unit test

(postCommit)• build image

build• setup/update

database (pre)• deploy applicazione• integration test

(post)

deploy

17

Lifecycle management e software promotion

gitpush/merge

•git flow/gitlab flow/...•github/generic webhook

build

•build applicazione•unit test (postCommit)•build image

deploy DEV

•setup/update database (pre)•deploy applicazione•functional/integration test (post)

deploySTAGING

•setup/update database (pre)•deploy applicazione•smoke test (post)

deployPRODUCTION

•setup/update database (pre)•deploy applicazione•smoke test (post)

18

Rolling deployment

• Rimpiazzare le istanze della vecchia versione con la nuova– readiness check– canary deployment– rollback automatico

• builtin in OpenShift• quando:

– no downtime– N-1 compatibility

• lifecycle hooks– pre– post

pre hook

scale in old rc

scale out new rc

repeatscaling

post hook

git push

build

deploy

19

Recreate deployment

• Rimpiazzare tutte le istanze della vecchia versione con la nuova– readiness check– rollback automatico

• builtin in OpenShift• quando:

– migrazione dati– no N-1 compatibility– RWO volume

• downtime• lifecycle hooks

– pre– mid– post

pre hook

scale in to 0

mid hook

scale out to N

post hook

git push

build

deploy

20

Blue-green deployment

• Obiettivo: testing prima dello switch del traffico– smoke test– integration test

• 2 versioni dell’applicazione– production– internal (release candidate)

• ‘internal’ può essere pubblica o privata

• 2 service• 2 deployment configuration• switch gestito da Jenkins• alternativa, API manager

– <public, staging, private> URL

• N-1 compatibility

git push

build

deploy

test

go live

21

Blue-green deployment

Pod

Service

Route app

app-blue

app-blue-1-x app-blue-1-y

app-green

app-green-2-z app-green-2-k

app.os.local.int

22

Blue-green deployment

Pod

Service

Route app

app-blue

app-blue-1-x app-blue-1-y

app-green

app-green-2-z app-green-2-k

app.os.local.int

oc patch route app –p ‘{“spec”: {“to”: {“name”: “app-green”}}}’

23

Canary deployment

• Obiettivo: ridurre il rischio di deploy di una nuova versione

• come blue/green, ma entrambeversioni online

• 1 service• 2 deployment configuration• molteplici shard• proxy shard

– scale out/in deployment configs -> traffic splitter (%)

• N-1 compatibility• gestito da Jenkins

git push

build

deploy

go live

test

scale out/in

24

Canary deployment

Pod

Service

Route app

app

app1-1-x app1-1-y app1-1-z app2-1-k

app.os.local.int

25

Canary deployment

Pod

Service

Route app

app

app1-1-x app1-1-y app2-1-k app2-1-t

app.os.local.int

oc scale --replicas=2 dc app2 && oc scale --replicas=2 dc app1

26

Canary deployment

Pod

Service

Route app

app

app2-1-k app2-1-t app2-1-j app2-1-w

app.os.local.int

oc scale --replicas=4 dc app2 && oc scale --replicas=0 dc app1

27

A/B deployment

• Obiettivi:– testing di 2 (o più) versioni

contemporaneamente– 2 (o più) configurazioni, stessa

versione (es. più region)

• come A/B testing, ma codice + config

• 1 service• 2 (o più) deployment configuration• molteplici shard

• proxy shard– scale out/in deployment configs ->

traffic splitter (%)

• N-1 compatibility• gestito da Jenkins

git push

build

deploy

go live

test

scale out/in

28

A/B deployment

Pod

Service

Route app

app

app-1-x app-1-k app-a-1-y app-b-2-z

app.os.local.int

29

A/B deployment

Pod

Service

Route app

app

app-1-x app-a-1-y app-b-2-z app-b-2-k

oc scale --replicas=2 dc app-b && oc scale --replicas=1 dc app

app.os.local.int

30

A/B deployment

Pod

Service

Route app

app

app-a-1-y app-a-1-t app-b-2-z app-b-2-k

oc scale --replicas=2 dc app-a && oc scale --replicas=0 dc app

app.os.local.int

31

A/B deployment

Pod

Service

Route app

app

app-b-2-z app-b-2-k app-b-2-t app-b-2-w

oc scale --replicas=4 dc app-b && oc scale --replicas=0 dc app-a

app.os.local.int

Sede Legale e Unità OperativaVia Alfredo Campanini, 620124 MilanoTel: +39 02.66732.1 – Fax: +39 02.66732.300

Unità OperativaVia Cristoforo Colombo, 16300147 RomaTel: +39 06.9826.9600 – Fax: +39 06.9826.9680

Grazie per l’attenzione!

francesco.fiore@par-tec.it | @ffiore81

https://www.par-tec.it