Alles Docker oder Was ?

BASEL BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENF HAMBURG KOPENHAGEN LAUSANNE MÜNCHEN STUTTGART WIEN ZÜRICH

Alles Docker oder was?Grundlagen und Vorteile von Container-Architekturen

Anatole Tresch, Principal Consultant

Alles Docker oder was?2 25.11.15

BIO

Anatole Tresch

● Principal Consultant, Trivadis AG (Switzerland)● Star Spec Lead JSR 354, Most significant spec lead 2015● Expert Group Member● Technical Architect, Lead Engineer● Apache PPMC Member

● Twitter: @atsticks● [email protected]● [email protected]●


Agenda

● Motivation

● Containers

● Docker

● Docker in Practice

Why Docker will rule the world!4 10.02.16


Docker?

● > 50000 pulls

● > 4000 Github stars

● > 100 significant contributors

● > 150 projects on top of Docker

● 1000s of dockerized applications

● Integrated with Jenkins, Travis, Chef, Puppet, Vagrant and Openstack

● Used by Ebay, Uber, Mozilla, Cloudfare, Rackspace and more...

http://docker.com


Let‘s go back to the

history of computation...


We started with...


Abstract symbols and formulas...


...and improved...


We scaled !


We started with automation...


...and miniaturization...


...and then we added electricity...


First we used it…

… as engines for our mechanical devices...


Then we used itfor performing the calculations itself...


...and things evolved further...


We made computers talking with each other.

So they can collaborate.


We started small…

...and evolved quickly...


...until...


Recapitulation...

Thick Client/Server appon thick client

Well defined stack:● OS● Runtime● Middleware

Monolithic physicalinfrastructure

1995 Today

Thin apps (mobile, tablet)

Assembled using (micro)services

Running on any resources (public, private, virtualized)


Container


Challenges 2016

How to ensure servicesinteract consistently?

How to avoid dependency hell?

How to avoid N x NDifferent configs?

How to fail-over, migrate & scale quickly?

How to ensure compatibility?

How to ensure decouplingon client side? Manage

Javascript...How to ensure location

Transparency andResilience at Big Scale?


Mul

tiple

Sta

cks

Mul

tiple

HW

env

Challenges

Smooth and quick migration?

Do services andapps interact

appropriately?


Results in N x N compatibility nightmareStatic website

? ? ? ? ? ? ?

Web frontend

? ? ? ? ? ? ?

Background workers

? ? ? ? ? ? ?

User DB ? ? ? ? ? ? ?

Analytics DB

? ? ? ? ? ? ?

Queue ? ? ? ? ? ? ?

Development VM

QA Server Prod Server

Onsite Cloud

Pulic Cloud

Contributors Laptop

Customer Services


Why Containers help?


Let‘s look at the „real“ thing...


Mul

tiple

Goo

dsM

ultip

le m

etho

dsfo

r st

orin

g/tr

ansp

ort

Cargo Transport Pre 1960

Can I transport quicklyand smoothly (egfrom boat to trainTo truck)

Do I worry abouthow goods interact(eg coffe beansnext to spices)


Results in N x N compatibility matrix nightmare

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?

? ? ? ? ? ? ?


What was the solution?


Containers !


Mul

tiple

Goo

dsM

ultip

le m

etho

dsfo

r st

orin

g/tr

ansp

ort

Containers!

Can I transport quicklyand smoothly (egfrom boat to trainTo truck)

Do I worry abouthow goods interact(eg coffe beansnext to spices)


N x N compatibility problem solved !


Benefits

● 90 % of all cargo shipped in containers

● Order of magnitude cost and time

benefits

● Massive reduction in losses due to theft

or damage

● Freight cost reduced from >26% to < 3%

● Massive Globalization

● 500 ships deliver 200M containers/year

● An new eco-system !


And what about Docker ?


= container system for code !


Docker ContainersM

ultip

le S

tack

sM

ultip

le H

W e

nv



appropriately?


In other words...M

ultip

le S

tack

sM

ultip

le H

W e

nv



appropriately?


Compatibility nightmare solved !Static website

? ? ? ? ? ? ?

Web frontend

? ? ? ? ? ? ?

Background workers

? ? ? ? ? ? ?

User DB ? ? ? ? ? ? ?

Analytics DB

? ? ? ? ? ? ?

Queue ? ? ? ? ? ? ?

Development VM

QA Server Prod Server

Onsite Cloud

Pulic Cloud

Contributors Laptop

Customer Services


Physical and Software Containers ?Aspect Physical Container Software Container

Content Agnostic Any type of cargo Any system and its dependencies

Hardware Agnostic Standard shape and interface allow to move between transport systems without modification or opening

Can run consistently on virtually any hardware – Vms, bare metal, openstack, public cloud – without any modification

Content Isolation Containers can be stacked and shipped together

Resource, network, content isolation.Stacking by orchestration layer.

Automation Easy (loading, unloading, moving) Standard operations: start, stop, status, search, commit,... Matches perfectly with auto-scaling, CI/CD, hybrid clouds

Efficiency No opening or modification needed. Lightweight, no perf or startup penalty, quick to move and manipulate

Separation of concerns Shippers: content, worriors: outside Developers: code, Ops: infrastructure


Advantages

● Clean and portable runtime environment

● Solves dependency hell, eliminates compatibility issues (almost)

● Allows multiple versions running in parallel

● Supports test automatation, integration, packaging

● Cheap, overhead-less

● Instant replay and reset of images


Docker Basics

● Build

● build images that contain your application’s entire environment, or application

operating system

● Ship

● Share images (dockerhub.com, private registry)

● Run

● Docker Daemon, Docker Registry

● Docker Client


Docker Components


Docker – Simple Example

Source: http://blog.arungupta.me/getting-started-with-docker/


Let‘s dive deeper...


Containers vs VMs

● Containers are isolated

● Containers share OS, and where

appropriate bins/libs

● Short Startup < 1 second


Why are Containers so lightweight ?

Original App:● No OS!

VMs Containers

Copy of App:● No OS!

● Can share bins/libs

Modofied App:

● Contains only diffs

between A nd A'


Docker Extension Points

Docker Functionality

● Versioning

● Lifecycle Management

● Networking

● Storage

Other Docker Products

● Machine

● Clustering (Swarm)

● Cloud Integration (Tutum)

● Orchestration (DataCenter)

Storage Storage

Storage Storage

Networking

Networking

NetworkingNetworking

NetworkingStorage Storage

StorageStorage


Size matters!


Participating Companies

Source: Lawrence Hecht / TheNewStack


Usage

Source: Lawrence Hecht / TheNewStack Books


Deployment Infrastructure

Source: Lawrence Hecht / TheNewStack


Docker in Practice


For Operations Docker provides

● A standard format for an „executable“

● A unified API for operating these executables

For Developers Docker provides

● Isolated, repeatable and relocatable runtime environment setup

● Isolation between containers and against the host

● Fast and Simple in Use


Nice.

What can I do with it ?


Developers

● Use Microservices !

● Let the infrastructure handle load-balancing, scaling and fail-over

● Focus on the business logic

● Build CI/CD pipelines

● Do canary production testing

● Use Containers as deployable artifact

● Use Containers as building blocks when exploring new solution designs


Operations

● Use containers as the default deployment format

● Execute containers fully automated

● Load-balancing

● Auto-scaling (up and down)

● Resilience and Fail-Over Handling

● Monitoring and Logging

● Use a Datacenter OS

● Dynamic Resource Management

● Global Service Location

● Integration with classic Hypervisor technology


Apache Mesos Apache Mesos Apache Mesos

The Leading Edge...

Ceph Storage Cluster

Orchetration (Kubernetes/Marathon/...)

Cloud Provider 1 Cloud Provider 2 On Premise

VM

Software Defined Network SDN

Distributed Configuration

Ceph Storage Cluster Ceph Storage Cluster

Hypervisor

OrchetrationOrchetration

SDN SDN

VM


The Leading Edge

● Distributed Storage allows to move

your data within minutes.

● Services automatically scale.

● No Outages anymore.

● Cloud Provider has no idea what is

happening inside (which is good)

● Very high degree of automation.

● Microservices and Resilient

Architecture are at its core.

● Machine Learning not yet concidered!

In 5-10 years computation and storage will

be

→ commodity

→ dynamically (automatically?) managed

→ fully distributed and provider transparent

→ resilient

→ 10-20x cheaper

→ This is the 4th revolution of IT


Sounds like Future ?


NO.


It is already here.


Open Source.


Tested.


Production Ready.


Summary


Summary

● Containers and Microservices are a natural match.

● Use a computational platform to get the maximum benefits.

● Scalability and Resiliency moved from development into operations.

● Get comfortable with modern software design principles.

● Changes affect everything: your infrastructure, your organization, your data, your

skills.

● Containers are at the core.

● The change is disruptive, but IMO inevitable! *

* with no alternative! Compare the impact on taxi companies when Uber.com entered the market


Act NOW !

Thank You !


Referenzen● Docker: http://docker.com, Docker e-books: http://thenewstack.io/ebookseries/ , Performance:

http://de.slideshare.net/Flux7Labs/performance-of-docker-vs-vms, Basics:

http://de.slideshare.net/larrycai/learn-docker-in-90-minutes?related=1

● Docker vs. VMs:

https://dzone.com/articles/vms-vs-docker-dan-jones-in-network-world?utm_source=Weekly%20Digest

&utm_medium=email&utm_content=cloud&utm_campaign=wd%202016-02-17● Linux Containers: https://linuxcontainers.org/

● Open Container: announcement: https://coreos.com/blog/app-container-and-the-open-container-project

● Appc:

http://thenewstack.io/coalition-for-app-container-spec-shows-docker-is-not-the-standard-for-everyone/

● Apache Mesos: http://mesos.apache.org

● Mesosphere: http://mesosphere.com

● Kubernetes: http://kubernetes.io

● Martin Fowler: Microservices. http://martinfowler.com/articles/microservices.html

http://docker.com/

http://thenewstack.io/ebookseries/

http://de.slideshare.net/Flux7Labs/performance-of-docker-vs-vms

http://de.slideshare.net/larrycai/learn-docker-in-90-minutes?related=1

https://dzone.com/articles/vms-vs-docker-dan-jones-in-network-world?utm_source=Weekly%20Digest&utm_medium=email&utm_content=cloud&utm_campaign=wd%202016-02-17

https://dzone.com/articles/vms-vs-docker-dan-jones-in-network-world?utm_source=Weekly%20Digest&utm_medium=email&utm_content=cloud&utm_campaign=wd%202016-02-17

https://linuxcontainers.org/

https://coreos.com/blog/app-container-and-the-open-container-project

http://thenewstack.io/coalition-for-app-container-spec-shows-docker-is-not-the-standard-for-everyone/

http://mesos.apache.org/

http://mesosphere.com/

http://kubernetes.io/

http://martinfowler.com/articles/microservices.html


Alles Docker oder was?Anatole TreschPrincipal Consultant

Tel. +41 58 459 53 [email protected]

Alles Docker oder Was ?

Engineering

Transcript of Alles Docker oder Was ?