Docker Engine

What is Docker Engine?

Docker Engine is the underlying client-server technology that supports the tasks and workflows involved in building, shipping and running containerized applications using Docker's components and services.

Used alone, the term Docker can refer either to Docker Engine or to the company Docker Inc., which offers various editions of containerization technology around Docker Engine.

Components of Docker Engine

Docker Engine is an open source technology comprising a server with a daemon process called dockerd, a REST API and a client-side command-line interface (CLI) called docker. The engine creates a server-side daemon process that hosts images, containers, networks and storage volumes. The CLI lets users interact with the Docker daemon via the API.

Docker Engine is declarative, meaning that administrators program a specific set of conditions as the desired state. The engine then automatically adjusts the actual settings and conditions to ensure they match the desired state at all times.

Docker Engine vs. Docker Machine

Docker Engine was initially developed for Linux systems and has since been extended to operate natively on both Windows and macOS. Docker Machine is a tool used to install and manage Docker Engine on various virtual hosts or older versions of macOS and Windows. When Docker Machine is installed on the local system, executing a command through Docker Machine not only creates virtual hosts, but also installs Docker and configures its clients.

As of 2021, Docker Inc. no longer actively maintains Docker Machine and recommends the Docker Desktop application for macOS and Windows container development.

Docker Engine and swarm mode

A swarm is a cluster of connected Docker Engines onto which admins can deploy application services. In version 1.12, Docker incorporated Docker Swarm directly into Docker Engine and renamed the orchestration service swarm mode. Swarm mode is Docker Engine's native clustering and orchestration service, although Docker Engine can run other options, such as Kubernetes.

With Docker Engine, admins can instantiate both manager and worker nodes from a single disk image at runtime instead of deployment. Because Docker Engine runs on a declarative model, swarm mode maintains and returns to the declared desired state in the event of an outage or scaling event.

Docker Engine plugins and storage volumes

Docker Engine can use a range of plugins, available as images hosted in a private registry or public repository such as GitHub or Docker Hub. Admins can manage a plugin's entire lifecycle with Docker Engine, from installation to deletion.

Plugins create items such as data volumes, which are directories that exist in a container. There are three types of volumes:

• Host volumes live in the file system.
• Named volumes are managed by Docker on the disk where the volume is created and named.
• Anonymous volumes are similar to named volumes, but are not associated with a specific source outside the container, making them more difficult to reference.

Docker Engine creates a data volume concurrently with a container image and can include data copied from a parent image. Containers can share and reuse volumes, and volumes are not deleted when a container is deleted. Because Docker Engine does not delete or collect alienated data volumes, users are responsible for their data volumes.

Networking in Docker Engine

Docker Engine provides default network drivers for users to create unique bridge networks that control container communication. Docker Inc. recommends that users define their own bridge networks for security purposes.

Containers can connect to multiple or no networks, and can connect and disconnect from networks without disrupting container operation. Docker Engine includes three network models:

• Bridge adds containers to the default docker0 network.
• None adds containers to a container-specific network stack, but does not give containers external network access.
• Host adds containers to the host's network stack, with no isolation between the host machine and containers.

Users can also create network driver plugins if Docker's three standard network types don't suit their needs. These plugins feature the same restrictions and rules as other plugins and use the plugin API.

Docker Engine networking and swarm mode can work together to create overlay networks on manager nodes without an external key-value store, which is necessary for clusters not managed by swarm mode. This overlay network is only available to the worker nodes that require it for a service and extends automatically to any new nodes running the service. Creating overlay networks without using swarm mode requires a valid key-value store service and is not recommended for most users.