Digioli.co.uk

It's called Docker for a reason. Imagine all the worlds goods being transported from country to country as-is. What a nightmare. Instead lets create a standard container, known dimensions, structure etc, then we can put what we want inside, and then transport it as a single unit. Wow.

Docker enables us to package an app with all its dependencies as a single unit. When I say app I mean a component, a single process whether it be an Nginx server, a frontend Javascript App, an API, an instance of Elasticsearch, whatever.

Lets cover a few definitions first.

Definitions

Container: A cut down version of Linux
Dockerfile: A file containing a list of commands to build an image
Image: Software you load onto a container. A saved state
Docker Engine: A background service that lets you create and manage containers
Docker Hub: Registry of docker images

Introduction

Docker is Linux based, meaning it needs to run on a flavour of Linux. A container is a 'cut down' version of Linux, which means it appears to be an operating system. This is the foundation of Docker. Next you want to load your software you want to run. So you would do that with a Dockerfile - instructions of what to install, add users, run scripts, etc. This installs into an image. That image can be started or stopped.

Take a look at the best practices for useful tips.

Namespaces

When a container process is started the kernel makes a lot of information available to it in its own isolated views via namespaces.

There are 6 namespaces that containers use:

mount - provides file system access
uts - host and domain name
ipc - Unix Interprocess Communication
network - its own unique and isolated network access
pid - its own pid table
usr/user

The mount namespace

The mount namespace gives processes belonging to the namespace a separate list of mount points that it doesn't share with other processes.

When images are loaded, all their drives are mounted using a Union File System (UFS), so containers referencing /data for example all have access to that same folder because the UFS 'unions' all those apparent individual /data directories. Think of UFS as a layer of files and directories, when another layer is added, folders and files at the upper-most layer are the ones that will be used. This is how source code can be installed and run locally, mounted onto a container, and the container has its own installation folder.

UTS namespace

UTS is the unix timesharing hostname domain name of your machine.

IPC - Unix interprocess communication

Take a look at Beej's guide for this.

Network namespace

The network namespace allocates another copy of the entire networking stack with its own routes firewall rules and network devices.. When Docker starts it automatically creates three networks.

To see the network run:

docker network ls

To create a new network:

docker network create testnetwork

When you spin up a container you can specify the network. Then you can add others onto the same network:

docker run -d --name bob --net=testnetwork mycontainer

docker run -d --name pete --net=testnetwork mycontainer

Overlay networks, can be a bit hairy. And Bridged networks too. To get the ip address you can run ip a

The user namespace

The user namespace provides a mapping from uid and gid outside the namespace to a different set inside the namespace. docker will map the specified user to be root inside the container.

Containers and Images

Containers are created from images, which in turn are created from Dockerfiles.

The docker run command takes a minimum of one argument - an image name. If you pass in the name of an image that doesn't exist locally, docker will go ahead and look for it on the "public registry" and then "pull" it if it exists.

There are two ways to create Docker images. Create an image from a running container, or programatically using a Dockerfile (preferred method). An elementary process for finding and using containers and images would be:

Create your Docker hub
Search for containers at Docker cloud
Tag, push, pull your image
Push to a Digital ocean droplets or similar

Containers use a few tools by which the resources given to a process are controlled:

cgroup - memory, bulk io, cpu, and devices
pivot root

The memory cgroup limits memory so it won't take up too much resources. You can specify the amount using the -m flag, so -m 100 would mean 100 MB.

same goes for CPUs, --cpuset-cpus 0 would set the container to only use one CPU. You can also supply a range (eg 0-2) or a combination such as 0-2,6,8.

pid is given a new process tree with pid 1 in that tree. If pid 1 dies, all other process in that tree are ungracefully terminated. So if we run a container we can see the pid will be 1:

docker run -it --rm debian ps aux

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   9084   816 ?        Rs+  19:18   0:00 ps aux

The unshare tool allows you to start a process with its new namespace. sigint is usually ctrl + c, sends an interrupt signal, which terminates the process.

Docker Commands

Get a list of commands used in docker by:

docker

List all running containers:

docker ps

or to list all containers:

docker ps -a

To list all images:

docker images

Running an image takes the format of docker run [options] image [command] [args]

docker run imagename

For example:

docker run ubuntu /bin/echo 'Hello world'

Remove the image:

docker rmi imagename

Stats:

docker stats

A full list of commands can be displayed with man docker.

Dockerfile

Dockerfiles are named Dockerfile with no extension or spaces. Just like Makefiles, the convention is to start with a capital letter. Your docker file will look something like this:

FROM [image]
MAINTAINER [your name] [<your@email.com>]
RUN [do something]
RUN [do something]
ADD [some files]
EXPOSE [port number]
CMD [command 1, command 2]
ENTRYPOINT [command args]

Dockerfile Commands

The FROM instruction specifies which base image to start from.

Then the convention is to add the MAINTAINER instruction, which is the owner of the docker image.

We can then run commands using RUN instruction to issue commands. Just like in the terminal you can concatenate commands by using &&.

We can add files to the docker image we use the ADD instruction.

Using WORKDIR sets the current working directory on the image.

The ENTRYPOINT is the main process that runs in the container. This process will be PID 1. The container lifecycle will be bound to that PID.

Then to build an image, use the build command:

docker build .

You can also tag to name your image using the -t flag:

docker build -t namespace/appname .

docker run -p 127.0.0.1:3000:3000 --name [container] -t [image]

Docker Volumes and data containers

When you run a container data persists for the life span of the container only. In some cases this is not a good thing. In order to persist data we have to expose the host file system, and we can do that in 2 ways:

Specifying VOLUME /some/dir in a Dockerfile
Specifying it as part of your run command as docker run -v /some/dir

VOLUME [host/path] [container/path]

docker run ubuntu -v /docker/test:/var/log /bin/echo 'Hello world' > /var/log/test.log

docker run -v /Users/markrobson/devrepos/broadband-mongo/dump:/opt/dump -it roppa/broadband-mongo

docker create -v /data/db:/data/db roppa/broadband-mongo /bin/true

The above do exactly the same thing, which is telling the host to create a directory in its own filesystem. Either way, these two things do exactly the same thing. It tells Docker to create a directory on the host, within the docker root path (by default /var/lib/docker), and mount it to the path you've specified (/some/dir above). When you remove the container using this volume, the volume itself continues to live on.

In the real world we are going to run docker images as services.

Docker tools

More and more tools are starting to appear... Docker compose, Docker machine, Docker swarm.

Docker compose

Docker Compose for running the docker-compose command. Containerise application with multiple components. Can start multiple containers, all networked together. Great for dev environments but not recommended for production. It is a list of services that connect together to form the whole application stack in one config yaml file.

Docker Cloud

A node is a host for containers. A service is a group of containers. Docker CLI client is for running Docker Engine to create images and containers.

Kitematic

A Docker GUI. the Docker QuickStart shell preconfigured for a Docker command-line environment.

Cache and cleaning up after yourself

Clean up after yourself

For docker instances:

docker stop $(docker ps -a -q)
docker rm $(docker ps -a -q)

For volumes:

docker volume rm $(docker volume ls -qf dangling=true)

With docker compose:

docker-compose rm &&
 docker-compose pull &&
 docker-compose build --no-cache &&
 docker-compose up -d --force-recreate

Testing Docker

A couple of resources are docker testing and testing with Bash.

Troubleshooting

If you get a timeout error run:

$ docker-machine restart default # Restart the environment
$ eval $(docker-machine env default) # Refresh environment settings

Logging

Forward request and error logs to docker log collector:

RUN ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log

Docker