It's called Docker for a reason. Imagine all the worlds goods being transported from country to country as-is. What a nightmare. Instead lets create a standard container, known dimensions, structure etc, then we can put what we want inside, and then transport it as a single unit. Wow.
Lets cover a few definitions first.
- A cut down version of Linux
- A file containing a list of commands to build an image
- Software you load onto a container. A saved state
- Docker Engine
- A background service that lets you create and manage containers
- Docker Hub
- Registry of docker images
Docker is Linux based, meaning it needs to run on a flavour of Linux. A container is a 'cut down' version of Linux, which means it appears to be an operating system. This is the foundation of Docker. Next you want to load your software you want to run. So you would do that with a Dockerfile - instructions of what to install, add users, run scripts, etc. This installs into an image. That image can be started or stopped.
Take a look at the best practices for useful tips.
When a container process is started the kernel makes a lot of information available to it in its own isolated views via namespaces.
There are 6 namespaces that containers use:
- mount - provides file system access
- uts - host and domain name
- ipc - Unix Interprocess Communication
- network - its own unique and isolated network access
- pid - its own pid table
The mount namespace
The mount namespace gives processes belonging to the namespace a separate list of mount points that it doesn't share with other processes.
When images are loaded, all their drives are mounted using a Union File System (UFS), so containers referencing
/data for example all have access to that same folder because the UFS 'unions' all those apparent individual /data directories. Think of UFS as a layer of files and directories, when another layer is added, folders and files at the upper-most layer are the ones that will be used. This is how source code can be installed and run locally, mounted onto a container, and the container has its own installation folder.
UTS is the unix timesharing hostname domain name of your machine.
IPC - Unix interprocess communication
Take a look at Beej's guide for this.
The network namespace allocates another copy of the entire networking stack with its own routes firewall rules and network devices.. When Docker starts it automatically creates three networks.
To see the network run:
docker network ls
To create a new network:
docker network create testnetwork
When you spin up a container you can specify the network. Then you can add others onto the same network:
docker run -d --name bob --net=testnetwork mycontainer docker run -d --name pete --net=testnetwork mycontainer
Overlay networks, can be a bit hairy. And Bridged networks too. To get the ip address you can run
The user namespace
The user namespace provides a mapping from uid and gid outside the namespace to a different set inside the namespace. docker will map the specified user to be root inside the container.
Containers and Images
Containers are created from images, which in turn are created from Dockerfiles.
docker run command takes a minimum of one argument - an image name. If you pass in the name of an image that doesn't exist locally, docker will go ahead and look for it on the "public registry" and then "pull" it if it exists.
There are two ways to create Docker images. Create an image from a running container, or programatically using a Dockerfile (preferred method). An elementary process for finding and using containers and images would be:
- Create your Docker hub
- Search for containers at Docker cloud
- Tag, push, pull your image
- Push to a Digital ocean droplets or similar
Containers use a few tools by which the resources given to a process are controlled:
- cgroup - memory, bulk io, cpu, and devices
- pivot root
The memory cgroup limits memory so it won't take up too much resources. You can specify the amount using the
-m flag, so
-m 100 would mean 100 MB.
same goes for CPUs,
--cpuset-cpus 0 would set the container to only use one CPU. You can also supply a range (eg 0-2) or a combination such as
pid is given a new process tree with pid 1 in that tree. If pid 1 dies, all other process in that tree are ungracefully terminated. So if we run a container we can see the pid will be 1:
docker run -it --rm debian ps aux
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 9084 816 ? Rs+ 19:18 0:00 ps aux
The unshare tool allows you to start a process with its new namespace.
sigint is usually ctrl + c, sends an interrupt signal, which terminates the process.
Get a list of commands used in docker by:
List all running containers:
or to list all containers:
docker ps -a
To list all images:
Running an image takes the format of
docker run [options] image [command] [args]
docker run imagename
docker run ubuntu /bin/echo 'Hello world'
Remove the image:
docker rmi imagename
A full list of commands can be displayed with
Dockerfiles are named
Dockerfile with no extension or spaces. Just like Makefiles, the convention is to start with a capital letter. Your docker file will look something like this:
FROM [image] MAINTAINER [your name] [<email@example.com>] RUN [do something] RUN [do something] ADD [some files] EXPOSE [port number] CMD [command 1, command 2] ENTRYPOINT [command args]
FROM instruction specifies which base image to start from.
Then the convention is to add the
MAINTAINER instruction, which is the owner of the docker image.
We can then run commands using
RUN instruction to issue commands. Just like in the terminal you can concatenate commands by using
We can add files to the docker image we use the
WORKDIR sets the current working directory on the image.
ENTRYPOINT is the main process that runs in the container. This process will be PID 1. The container lifecycle will be bound to that PID.
Then to build an image, use the build command:
docker build .
You can also tag to name your image using the
docker build -t namespace/appname .
docker run -p 127.0.0.1:3000:3000 --name [container] -t [image]
Docker Volumes and data containers
When you run a container data persists for the life span of the container only. In some cases this is not a good thing. In order to persist data we have to expose the host file system, and we can do that in 2 ways:
- Specifying VOLUME /some/dir in a Dockerfile
- Specifying it as part of your run command as docker run -v /some/dir
VOLUME [host/path] [container/path]
docker run ubuntu -v /docker/test:/var/log /bin/echo 'Hello world' > /var/log/test.log
docker run -v /Users/markrobson/devrepos/broadband-mongo/dump:/opt/dump -it roppa/broadband-mongo
docker create -v /data/db:/data/db roppa/broadband-mongo /bin/true
The above do exactly the same thing, which is telling the host to create a directory in its own filesystem. Either way, these two things do exactly the same thing. It tells Docker to create a directory on the host, within the docker root path (by default /var/lib/docker), and mount it to the path you've specified (/some/dir above). When you remove the container using this volume, the volume itself continues to live on.
In the real world we are going to run docker images as services.
More and more tools are starting to appear... Docker compose, Docker machine, Docker swarm.
Docker Compose for running the docker-compose command. Containerise application with multiple components. Can start multiple containers, all networked together. Great for dev environments but not recommended for production. It is a list of services that connect together to form the whole application stack in one config yaml file.
A node is a host for containers. A service is a group of containers. Docker CLI client is for running Docker Engine to create images and containers.
A Docker GUI. the Docker QuickStart shell preconfigured for a Docker command-line environment.
Cache and cleaning up after yourself
For docker instances:
docker stop $(docker ps -a -q) docker rm $(docker ps -a -q)
docker volume rm $(docker volume ls -qf dangling=true)
With docker compose:
docker-compose rm && docker-compose pull && docker-compose build --no-cache && docker-compose up -d --force-recreate
If you get a timeout error run:
$ docker-machine restart default # Restart the environment $ eval $(docker-machine env default) # Refresh environment settings
Forward request and error logs to docker log collector:
RUN ln -sf /dev/stdout /var/log/nginx/access.log \ && ln -sf /dev/stderr /var/log/nginx/error.log
- Docker cheat sheet
- Linux namespaces
- Arch linux
- Docker in depth volumes
- Docker on Mac OS
- What is Docker?
- Getting started
- Docker cloud - your first node
- Docker Images
- Nodejs docker guide
- Dockerising a nodejs app
- Dockerising node apps
- Basic MEAN workflow
- Digital Ocean
- Docker user guide - networking
- UNIX namespaces
- How to work with volumes
- Lessons building a node app in docker
- Liz Rice presentation