$ ls

01-cnm.md 04-docker-network-cp.md 07-macvlan.md

10-load-balancing.md 13-troubleshooting.md img
02-drivers.md 05-bridge-networks.md 08-host-networking.md
11-security.md 14-network-models.md
03-linux-networking.md 06-overlay-networks.md 09-physical-networking.md
12-ipaddress-management.md README.md
[node1] (local) [email protected] ~/labs/networking/concepts
$ cat *.md

## <a name="cnm"></a>The Container Networking Model

The Docker networking architecture is built on a set of interfaces called the
_Container Networking Model_ (CNM). The philosophy of CNM is to provide application
portability across diverse infrastructures. This model strikes a balance to achieve
application portability and also takes advantage of special features and
capabilities of the infrastructure.


### CNM Constructs

There are several high-level constructs in the CNM. They are all OS and
infrastructure agnostic so that applications can have a uniform experience no
matter the infrastructure stack.

- __Sandbox__ — A Sandbox contains the configuration of a container's network

stack. This includes management of the container's interfaces, routing table, and
DNS settings. An implementation of a Sandbox could be a Linux Network Namespace, a
FreeBSD Jail, or other similar concept. A Sandbox may contain many endpoints from
multiple networks.
- __Endpoint__ — An Endpoint joins a Sandbox to a Network. The Endpoint construct
exists so the actual connection to the network can be abstracted away from the
application. This helps maintain portability so that a service can use different
types of network drivers without being concerned with how it's connected to that
network.
- __Network__ — The CNM does not specify a Network in terms of the OSI model. An
implementation of a Network could be a Linux bridge, a VLAN, etc. A Network is a
collection of endpoints that have connectivity between them. Endpoints that are not
connected to a network will not have connectivity on a Network.

Next: **[Drivers](02-drivers.md)**
## CNM Driver Interfaces
The Container Networking Model provides two pluggable and open interfaces that can
be used by users, the community, and vendors to leverage additional functionality,
visibility, or control in the network.

### Categories of Network Drivers

- __Network Drivers__ — Docker Network Drivers provide the actual implementation

that makes networks work. They are pluggable so that different drivers can be used
and interchanged easily to support different use-cases. Multiple network drivers
can be used on a given Docker Engine or Cluster concurrently, but each Docker
network is only instantiated through a single network driver. There are two broad
types of CNM network drivers:
- __Built-In Network Drivers__ — Built-In Network Drivers are a native part
of the Docker Engine and are provided by Docker. There are multiple to choose from
that support different capabilities like overlay networks or local bridges.
- __Plug-In Network Drivers__ — Plug-In Network Drivers are network drivers
created by the community and other vendors. These drivers can be used to provide
integration with incumbent software and hardware. Users can also create their own
drivers in cases where they desire specific functionality that is not supported by
an existing network driver.
- __IPAM Drivers__ — Docker has a built-in IP Address Management Driver that
provides default subnets or IP addresses for Networks and Endpoints if they are not
specified. IP addressing can also be manually assigned through network, container,
and service create commands. Plug-In IPAM drivers also exist that provide
integration to existing IPAM tools.

### Docker Built-In Network Drivers

The Docker built-in network drivers are part of Docker Engine and don't require any
extra modules. They are invoked and used through standard `docker network`
commands. The follow built-in network drivers exist:

- __Bridge__ — The `bridge` driver creates a Linux bridge on the host that is
managed by Docker. By default containers on a bridge will be able to communicate
with each other. External access to containers can also be configured through the
`bridge` driver.

- __Overlay__ — The `overlay` driver creates an overlay network that supports

multi-host networks out of the box. It uses a combination of local Linux bridges
and VXLAN to overlay container-to-container communications over physical network
infrastructure.

- __MACVLAN__ — The `macvlan` driver uses the MACVLAN bridge mode to establish a
connection between container interfaces and a parent host interface (or sub-
interfaces). It can be used to provide IP addresses to containers that are routable
on the physical network. Additionally VLANs can be trunked to the `macvlan` driver
to enforce Layer 2 container segmentation.

- __Host__ — With the `host` driver, a container uses the networking stack of the
host. There is no namespace separation, and all interfaces on the host can be used
directly by the container.

- __None__ — The `none` driver gives a container its own networking stack and
network namespace but does not configure interfaces inside the container. Without
additional configuration, the container is completely isolated from the host
networking stack.

### Default Docker Networks

By default a `none`, `host`, and `bridge` network will exist on every Docker host.
These networks cannot be removed. When instantiating a Swarm, two additional
networks, a bridge network named `docker_gwbridge` and an overlay network named
`ingress`, are automatically created to facilitate cluster networking.

The `docker network ls` command shows these default Docker networks for a Docker
Swarm:

```
NETWORK ID NAME DRIVER SCOPE
1475f03fbecb bridge bridge local
e2d8a4bd86cb docker_gwbridge bridge local
407c477060e7 host host local
f4zr3zrswlyg ingress overlay swarm
c97909a4b198 none null local
```

In addition to these default networks, [user defined networks](#userdefined) can

also be created. They are discussed later in this document.
### Network Scope
As seen in the `docker network ls` output, Docker network drivers have a concept of
_scope_. The network scope is the domain of the driver which can be the `local` or
`swarm` scope. Local scope drivers provide connectivity and network services (such
as DNS or IPAM) within the scope of the host. Swarm scope drivers provide
connectivity and network services across a swarm cluster. Swarm scope networks will
have the same network ID across the entire cluster while local scope networks will
have a unique network ID on each host.

### Docker Plug-In Network Drivers

The following community- and vendor-created plug-in network drivers are compatible
with CNM. Each provides unique capabilities and network services for containers.

| Driver | Description |
|------|------|
| [**contiv**](http://contiv.github.io/) | An open source network plugin led by
Cisco Systems to provide infrastructure and security policies for multi-tenant
microservices deployments. Contiv also provides integration for non-container
workloads and with physical networks, such as ACI. Contiv implements plug-in
network and IPAM drivers. |
| [**weave**](https://www.weave.works/docs/net/latest/introducing-weave/) | A
network plugin that creates a virtual network that connects Docker containers
across multiple hosts or clouds. Weave provides automatic discovery of
applications, can operate on partially connected networks, does not require an
external cluster store, and is operations friendly. |
| [**calico**](https://www.projectcalico.org/) | Calico is an open source
solution for virtual networking in cloud datacenters. It targets datacenters where
most of the workloads(VMs, containers, or bare metal servers) only require IP
connectivity. Calico provides this connectivity using standard IP routing.
Isolation between workloads — whether according to tenant ownership, or any finer
grained policy — is achieved via iptables programming on the servers hosting the
source and destination workloads. |
| [**kuryr**](https://github.com/openstack/kuryr) | A network plugin developed
as part of the OpenStack Kuryr project. It implements the Docker networking
(libnetwork) remote driver API by utilizing Neutron, the OpenStack networking
service. Kuryr includes an IPAM driver as well. |

### Docker Plug-In IPAM Drivers

Community and vendor created IPAM drivers can also be used to provide integrations
with existing systems or special capabilities.

| Driver | Description |
|------|------|
| [**infoblox**](https://store.docker.com/community/images/infoblox/ipam-driver) |
An open source IPAM plugin that provides integration with existing Infoblox tools.
|

> There are many Docker plugins that exist and more are being created all the time.
Docker maintains a list of the [most common
plugins.](https://docs.docker.com/engine/extend/legacy_plugins/)

Next: **[Linux Network Fundamentals](03-linux-networking.md)**

## <a name="drivers"></a><a name="linuxnetworking"></a>Linux Network Fundamentals

The Linux kernel features an extremely mature and performant implementation of the
TCP/IP stack (in addition to other native kernel features like DNS and VXLAN).
Docker networking uses the kernel's networking stack as low level primitives to
create higher level network drivers. Simply put, _Docker networking <b>is</b> Linux
networking._

This implementation of existing Linux kernel features ensures high performance and
robustness. Most importantly, it provides portability across many distributions and
versions which enhances application portability.

There are several Linux networking building blocks which Docker uses to implement
its built-in CNM network drivers. This list includes **Linux bridges**, **network
namespaces**, **veth pairs**, and **iptables**. The combination of these tools
implemented as network drivers provide the forwarding rules, network segmentation,
and management tools for complex networkpolicy.

### <a name="linuxbridge"></a>The Linux Bridge

A **Linux bridge** is a Layer 2 device that is the virtual implementation of a
physical switch inside the Linux kernel. It forwards traffic based on MAC addresses
which it learns dynamically by inspecting traffic. Linux bridges are used
extensively in many of the Docker network drivers. A Linux bridge is not to be
confused with the `bridge` Docker network driver which is a higher level
implementation of the Linux bridge.

### Network Namespaces

A Linux **network namespace** is an isolated network stack in the kernel with its
own interfaces, routes, and firewall rules. It is a security aspect of containers
and Linux, used to isolate containers. In networking terminology they are akin to a
VRF that segments the network control and data plane inside the host. Network
namespaces ensure that two containers on the same host will not be able to
communicate with each other or even the host itself unless configured to do so via
Docker networks. Typically, CNM network drivers implement separate namespaces for
each container. However, containers can share the same network namespace or even be
a part of the host's network namespace. The host network namespace contains the
host interfaces and host routing table. This network namespace is called the global
network namespace.

### Virtual Ethernet Devices

A **virtual ethernet device** or **veth** is a Linux networking interface that acts
as a connecting wire between two network namespaces. A veth is a full duplex link
that has a singleinterface in each namespace. Traffic in one interface is directed
out the other interface. Docker network drivers utilize veths to provide explicit
connections between namespaces whenDocker networks are created. When a container is
attached to a Docker network, one end of the veth is placed inside the container
(usually seen as the `ethX` interface) while the other is attached to the Docker
network.

### iptables
**`iptables`** is the native packet filtering system that has been a part of the
Linux kernel since version 2.4. It's a feature rich L3/L4 firewall that provides
rule chains for packet marking, masquerading, and dropping. The built-in Docker
network drivers utilize `iptables` extensively to segment network traffic, provide
host port mapping, and to mark traffic forload balancing decisions.

Next: **[Docker Network Control Plane](04-docker-network-cp.md)**

## <a name="controlplane"></a>Docker Network Control Plane
The Docker-distributed network control plane manages the state of Swarm-scoped
Docker networks in addition to propagating control plane data. It is a built-in
capability of Docker Swarm clusters and does not require any extra components such
as an external KV store. The control plane uses a
[Gossip](https://en.wikipedia.org/wiki/Gossip_protocol) protocol based on [SWIM]
(https://www.cs.cornell.edu/~asdas/research/dsn02-swim.pdf) to propagate network
state information and topology across Docker container clusters. The Gossip
protocol is highly efficient at reaching eventual consistency within the cluster
while maintaining constant rates of message size, failure detection times, and
convergence time across very large scale clusters. This ensures that the network is
able to scale across many nodes without introducing scaling issues such as slow
convergence or false positive node failures.

The control plane is highly secure, providing confidentiality, integrity, and

authentication through encrypted channels. It is also scoped per network which
greatly reduces the updates that any given host will receive.


It is composed of several components that work together to achieve fast convergence

across large scale networks. The distributed nature of the control plane ensures
that cluster controller failures don't affect network performance.

The Docker network control plane components are as follows:

- **Message Dissemination** updates nodes in a peer-to-peer fashion fanning out the

information in each exchange to a larger group of nodes. Fixed intervals and size
of peer groups ensures that network usage is constant even as the size of the
cluster scales. Exponential information propagation across peers ensures that
convergence is fast and bounded across any cluster size.
- **Failure Detection** utilizes direct and indirect hello messages to rule out
network congestion and specific paths from causing false positive node failures.
- **Full State Syncs** occur periodically to achieve consistency faster and resolve
network partitions.
- **Topology Aware** algorithms understand the relative latency between themselves
and other peers. This is used to optimize the peer groups which makes convergence
faster and more efficient.
- **Control Plane Encryption** protects against man in the middle and other attacks
that could compromise network security.

> The Docker Network Control Plane is a component of

[Swarm](https://docs.docker.com/engine/swarm/) and requires a Swarm cluster to
operate.

Next: **[Docker Bridge Network Driver Architecture](05-bridge-networks.md)**

## <a name="drivers"></a>Docker Bridge Network Driver Architecture

This section explains the default Docker bridge network as well as user-defined
bridge networks.

### Default Docker Bridge Network

On any host running Docker Engine, there will, by default, be a local Docker
network named `bridge`. This network is created using a `bridge` network driver
which instantiates a Linuxbridge called `docker0`. This may sound confusing.

- `bridge` is the name of the Docker network

- `bridge` is the network driver, or template, from which this network is created
- `docker0` is the name of the Linux bridge that is the kernel building block used
to implement this network

On a standalone Docker host, `bridge` is the default network that containers will
connect to if no other network is specified. In the following example a container
is created with no network parameters. Docker Engine connects it to the `bridge`
network by default. Inside the container we can see `eth0` which is created by the
`bridge` driver and given an address bythe Docker built-in IPAM driver.
```bash
#Create a busybox container named "c1" and show its IP addresses
host$ docker run -it --name c1 busybox sh
c1 # ip address
4: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue
link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.2/16 scope global eth0
...
```
> A container interface's MAC address is dynamically generated and embeds the IP
address to avoid collision. Here `ac:11:00:02` corresponds to `172.17.0.2`.

By using the tool `brctl` on the host, we show the Linux bridges that exist in the
host network namespace. It shows a single bridge called `docker0`. `docker0` has
one interface, `vetha3788c4`, which provides connectivity from the bridge to the
`eth0` interface inside container `c1`.

```
host$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242504b5200 no vethb64e8b8
```

Inside container `c1` we can see the container routing table that directs traffic
to `eth0` of the container and thus the `docker0` bridge.

```bash
c1# ip route
default via 172.17.0.1 dev eth0
172.17.0.0/16 dev eth0 src 172.17.0.2
```
A container can have zero to many interfaces depending on how many networks it is
connected to. Each Docker network can only have a single interface per container.


When we peek into the host routing table we can see the IP interfaces in the global
network namespace that now includes `docker0`. The host routing table provides
connectivity between`docker0` and `eth0` on the external network, completing the
path from inside the container to the external network.

```bash
host$ ip route
default via 172.31.16.1 dev eth0
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.42.1
172.31.16.0/20 dev eth0 proto kernel scope link src 172.31.16.102
```

By default `bridge` will be assigned one subnet from the ranges 172.[17-31].0.0/16
or 192.168.[0-240].0/20 which does not overlap with any existing host interface.
The default `bridge` network can be also be configured to use user-supplied address
ranges. Also, an existing Linux bridge can be used for the `bridge` network rather
than Docker creating one. Go to the [Docker Engine
docs](https://docs.docker.com/engine/userguide/networking/default_network/custom-
docker0/) for more information about customizing `bridge`.

> The default `bridge` network is the only network that supports legacy [links]
(https://docs.docker.com/engine/userguide/networking/default_network/dockerlinks/).
Name-based service discovery and user-provided IP addresses are __not__ supported
by the default `bridge` network.

### <a name="userdefined"></a>User-Defined Bridge Networks

In addition to the default networks, users can create their own networks called
**user-defined networks** of any network driver type. In the case of user-defined
`bridge` networks, Docker will create a new Linux bridge on the host. Unlike the
default `bridge` network, user-defined networks supports manual IP address and
subnet assignment. If an assignment isn't given, then Docker's default IPAM driver
will assign the next subnet available in the private IP space.


Below we are creating a user-defined `bridge` network and attaching two containers
to it. We specify a subnet and call the network `my_bridge`. One container is not
given IP parameters, so the IPAM driver assigns it the next available IP in the
subnet. The other container has its IP specified.

```
$ docker network create -d bridge my_bridge
$ docker run -itd --name c2 --net my_bridge busybox sh
$ docker run -itd --name c3 --net my_bridge --ip 10.0.0.254 busybox sh
```

`brctl` now shows a second Linux bridge on the host. The name of the Linux bridge,
`br-4bcc22f5e5b9`, matches the Network ID of the `my_bridge` network. `my_bridge`
also has two `veth` interfaces connected to containers `c2` and `c3`.

```
$ brctl show
bridge name bridge id STP enabled interfaces
br-b5db4578d8c9 8000.02428d936bb1 no vethc9b3282

vethf3ba8b5
docker0 8000.0242504b5200 no vethb64e8b8

$ docker network ls
NETWORK ID NAME DRIVER SCOPE
b5db4578d8c9 my_bridge bridge local
e1cac9da3116 bridge bridge local
...
```

Listing the global network namespace interfaces shows the Linux networking
circuitry that's been instantiated by Docker Engine. Each `veth` and Linux bridge
interface appears as a link between one of the Linux bridges and the container
network namespaces.

```bash
$ ip link

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536

2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001
3: docker0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
5: vethb64e8b8@if4: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
6: br-b5db4578d8c9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
8: vethc9b3282@if7: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
10: vethf3ba8b5@if9: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500
...
```

### External and Internal Connectivity

By default all containers on the same `bridge` driver network will have
connectivity with each other without extra configuration. This is an aspect of most
types of Docker networks. By virtue of the Docker network the containers are able
to communicate across their network namespaces and (for multi-host drivers) across
external networks as well. **Communication between different Docker networks is
firewalled by default.** This is a fundamental security aspect that allows us to
provide network policy using Docker networks. For example, in the figure above
containers `c2` and `c3` have reachability but they cannot reach `c1`.

Docker `bridge` networks are not exposed on the external (underlay) host network by
default. Container interfaces are given IPs on the private subnets of the bridge
network. Containers communicating with the external network are port mapped or
masqueraded so that their traffic uses an IP address of the host. The example below
shows outbound and inbound container traffic passing between the host interface and
a user-defined `bridge` network.


Outbound (egress) container traffic is allowed by default. Egress connections

initiated by containers are masqueraded/SNATed to an ephemeral port (_typically in
the range of 32768 to 60999_). Return traffic on this connection is allowed, and
thus the container uses the best routable IP address of the host on the ephemeral
port.

Ingress container access is provided by explicitly exposing ports. This port

mapping is done by Docker Engine and can be controlled through UCP or the Engine
CLI. A specific or randomly chosen port can be configured to expose a service or
container. The port can be set to listen on a specific (or all) host interfaces,
and all traffic will be mapped from this port to a port and interface inside the
container.

This previous diagram shows how port mapping and masquerading takes place on a
host. Container `C2` is connected to the `my_bridge` network and has an IP address
of `10.0.0.2`. When it initiates outbound traffic the traffic will be masqueraded
so that it is sourced from ephemeral port `32768` on the host interface
`192.168.0.2`. Return traffic will use the same IP address and port for its
destination and will be masqueraded internally back to the container address:port
`10.0.0.2:33920`.

Exposed ports can be configured using `--publish` in the Docker CLI or UCP. The
diagram shows an exposed port with the container port `80` mapped to the host
interface on port `5000`.The exposed container would be advertised at
`192.168.0.2:5000`, and all traffic going to this interface:port would be sent to
the container at `10.0.0.2:80`.

Next: **[Overlay Driver Network Architecture](06-overlay-networks.md)**

## <a name="overlaydriver"></a>Overlay Driver Network Architecture

The built-in Docker `overlay` network driver radically simplifies many of the
challenges in multi-host networking. With the `overlay` driver, multi-host networks
are first-class citizens inside Docker without external provisioning or components.
`overlay` uses the Swarm-distributed control plane to provide centralized
management, stability, and security across verylarge scale clusters.

### VXLAN Data Plane

The `overlay` driver utilizes an industry-standard VXLAN data plane that decouples
the container network from the underlying physical network (the _underlay_). The
Docker overlay network encapsulates container traffic in a VXLAN header which
allows the traffic to traverse the physical Layer 2 or Layer 3 network. The overlay
makes network segmentation dynamic and easy to control no matter what the
underlying physical topology. Use of the standard IETF VXLAN header promotes
standard tooling to inspect and analyze network traffic.

> VXLAN has been a part of the Linux kernel since version 3.7, and Docker uses the
native VXLAN features of the kernel to create overlay networks. The Docker overlay
datapath is entirely in kernel space. This results in fewer context switches, less
CPU overhead, and a low-latency, direct traffic path between applications and the
physical NIC.

IETF VXLAN ([RFC 7348](https://datatracker.ietf.org/doc/rfc7348/)) is a data-layer

encapsulation format that overlays Layer 2 segments over Layer 3 networks. VXLAN is
designed to be used in standard IP networks and can support large-scale, multi-
tenant designs on shared physical network infrastructure. Existing on-premises and
cloud-based networks can support VXLANtransparently.

VXLAN is defined as a MAC-in-UDP encapsulation that places container Layer 2 frames

inside an underlay IP/UDP header. The underlay IP/UDP header provides the transport
between hosts on the underlay network. The overlay is the stateless VXLAN tunnel
that exists as point-to-multipoint connections between each host participating in a
given overlay network. Because theoverlay is independent of the underlay topology,
applications become more portable. Thus, network policy and connectivity can be
transported with the application whether it is on-premises, on a developer desktop,
or in a public cloud.


In this diagram we see the packet flow on an overlay network. Here are the steps
that take place when `c1` sends `c2` packets across their shared overlay network:

- `c1` does a DNS lookup for `c2`. Since both containers are on the same overlay
network the Docker Engine local DNS server resolves `c2` to its overlay IP address
`10.0.0.3`.
- An overlay network is a L2 segment so `c1` generates an L2 frame destined for the
MAC address of `c2`.
- The frame is encapsulated with a VXLAN header by the `overlay` network driver.
The distributed overlay control plane manages the locations and state of each VXLAN
tunnel endpoint soit knows that `c2` resides on `host-B` at the physical address of
`192.168.1.3`. That address becomes the destination address of the underlay IP
header.
- Once encapsulated the packet is sent. The physical network is responsible of
routing or bridging the VXLAN packet to the correct host.
- The packet arrives at the `eth0` interface of `host-B` and is decapsulated by the
`overlay` network driver. The original L2 frame from `c1` is passed to the `c2`'s
`eth0` interface and up to the listening application.

### Overlay Driver Internal Architecture

The Docker Swarm control plane automates all of the provisioning for an overlay
network. No VXLAN configuration or Linux networking configuration is required.
Data-plane encryption, an optional feature of overlays, is also automatically
configured by the overlay driver as networks are created. The user or network
operator only has to define the network (`docker network create -d overlay ...`)
and attach containers to that network.


During overlay network creation, Docker Engine creates the network infrastructure
required for overlays on each host. A Linux bridge is created per overlay along
with its associated VXLAN interfaces. The Docker Engine intelligently instantiates
overlay networks on hosts only when a container attached to that network is
scheduled on the host. This prevents sprawl ofoverlay networks where connected
containers do not exist.

In the following example we create an overlay network and attach a container to

that network. We'll then see that Docker Swarm/UCP automatically creates the
overlay network.

```bash
#Create an overlay named "ovnet" with the overlay driver
$ docker network create -d overlay ovnet

#Create a service from an nginx image and connect it to the "ovnet" overlay network
$ docker service create --network ovnet --name container nginx
```

When the overlay network is created, you will notice that several interfaces and
bridges are created inside the host.

```bash
# Run the "ifconfig" command inside the nginx container
$ docker exec -it container ifconfig

#docker_gwbridge network
eth1 Link encap:Ethernet HWaddr 02:42:AC:12:00:04
inet addr:172.18.0.4 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe12:4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:648 (648.0 B) TX bytes:648 (648.0 B)

#overlay network
eth2 Link encap:Ethernet HWaddr 02:42:0A:00:00:07
inet addr:10.0.0.7 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::42:aff:fe00:7/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1
RX packets:8 errors:0 dropped:0 overruns:0 frame:0
TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:648 (648.0 B) TX bytes:648 (648.0 B)

#container loopback
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:48 errors:0 dropped:0 overruns:0 frame:0
 TX packets:48 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:4032 (3.9 KiB) TX bytes:4032 (3.9 KiB)
```

Two interfaces have been created inside the container that correspond to two
bridges that now exist on the host. On overlay networks, each container will have
at least two interfaces that connect it to the `overlay` and the `docker_gwbridge`.

| Bridge | Purpose |
|:------:|------|
| **overlay** | The ingress and egress point to the overlay network that VXLAN
encapsulates and (optionally) encrypts traffic going between containers on the same
overlay network. It extends the overlay across all hosts participating in this
particular overlay. One will exist per overlay subnet on a host, and it will have
the same name that a particular overlay network is given. |
| **docker_gwbridge** | The egress bridge for traffic leaving the cluster. Only
one `docker_gwbridge` will exist per host. Container-to-Container traffic is
blocked on this bridge allowing ingress/egress traffic flows only. |

> The Docker Overlay driver has existed since Docker Engine 1.9, and an external
K/V store was required to manage state for the network. Docker 1.12 integrated the
control plane stateinto Docker Engine so that an external store is no longer
required. 1.12 also introduced several new features including encryption and
service load balancing. Networking features thatare introduced require a Docker
Engine version that supports them, and using these features with older versions of
Docker Engine is not supported.

Next: **[MACVLAN](07-macvlan.md)**
## <a name="macvlandriver"></a>MACVLAN

The `macvlan` driver is a new implementation of the tried and true network
virtualization technique. The Linux implementations are extremely lightweight
because rather than using a Linux bridge for isolation, they are simply associated
with a Linux Ethernet interface or sub-interface to enforce separation between
networks and connectivity to the physical network.

MACVLAN offers a number of unique features and capabilities. It has positive

performance implications by virtue of having a very simple and lightweight
architecture. Rather than port mapping, the MACVLAN driver provides direct access
between containers and the physical network. It also allows containers to receive
routable IP addresses that are on the subnet of thephysical network.

The `macvlan` driver uses the concept of a parent interface. This interface can be
a physical interface such as `eth0`, a sub-interface for 802.1q VLAN tagging like
`eth0.10` (`.10` representing `VLAN 10`), or even a bonded host adaptor which
bundle two Ethernet interfaces into a single logical interface.

A gateway address is required during MACVLAN network configuration. The gateway

must be external to the host provided by the network infrastructure. MACVLAN
networks allow access between container on the same network. Access between
different MACVLAN networks on the same host is not possible without routing outside
the host.


In this example, we bind a MACVLAN network to `eth0` on the host. We attach two
containers to the `mvnet` MACVLAN network and show that they can ping between
themselves. Each container has an address on the `192.168.0.0/24` physical network
subnet and their default gateway is an interface in the physical network.

```bash
#Creation of MACVLAN network "mvnet" bound to eth0 on the host
$ docker network create -d macvlan --subnet 192.168.0.0/24 --gateway 192.168.0.1 -o
parent=eth0 mvnet

#Creation of containers on the "mvnet" network

$ docker run -itd --name c1 --net mvnet --ip 192.168.0.3 busybox sh
$ docker run -it --name c2 --net mvnet --ip 192.168.0.4 busybox sh
/ # ping 192.168.0.3
PING 127.0.0.1 (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: icmp_seq=0 ttl=64 time=0.052 ms
```

As you can see in this diagram, `c1` and `c2` are attached via the MACVLAN network
called `macvlan` attached to `eth0` on the host.

### VLAN Trunking with MACVLAN

Trunking 802.1q to a Linux host is notoriously painful for many in operations. It

requires configuration file changes in order to be persistent through a reboot. If
a bridge is involved, a physical NIC needs to be moved into the bridge, and the
bridge then gets the IP address. The `macvlan` driver completely manages sub-
interfaces and other components of the MACVLAN network through creation,
destruction, and host reboots.


When the `macvlan` driver is instantiated with sub-interfaces it allows VLAN

trunking to the host and segments containers at L2. The `macvlan` driver
automatically creates the sub-interfaces and connects them to the container
interfaces. As a result each container will be in a different VLAN, and
communication will not be possible between them unless traffic is routed in the
physical network.

```bash
#Creation of macvlan10 network that will be in VLAN 10
$ docker network create -d macvlan --subnet 192.168.10.0/24 --gateway 192.168.10.1
-o parent=eth0.10macvlan10

#Creation of macvlan20 network that will be in VLAN 20

$ docker network create -d macvlan --subnet 192.168.20.0/24 --gateway 192.168.20.1
-o parent=eth0.20 macvlan20

#Creation of containers on separate MACVLAN networks

$ docker run -itd --name c1--net macvlan10 --ip 192.168.10.2 busybox sh
$ docker run -it --name c2--net macvlan20 --ip 192.168.20.2 busybox sh
```

In the preceding configuration we've created two separate networks using the
`macvlan` driver that are configured to use a sub-interface as their parent
interface. The `macvlan` driver creates the sub-interfaces and connects them
between the host's `eth0` and the container interfaces. The host interface and
upstream switch must be set to `switchport mode trunk` so that VLANs are tagged
going across the interface. One or more containers can be connected to a given
MACVLAN network to create complex network policies that are segmented via L2.

> Because multiple MAC addresses are living behind a single host interface you
might need to enable promiscuous mode on the interface depending on the NIC's
support for MAC filtering.

Next: **[Host (Native) Network Driver](08-host-networking.md)**

## <a name="hostdriver"></a>Host (Native) Network Driver

The `host` network driver connects a container directly to the host networking
stack. Containers using the `host` driver reside in the same network namespace as
the host itself. Thus,containers will have native bare-metal network performance at
the cost of namespace isolation.

```bash
#Create a container attached to the host network namespace and print its network
interfaces
$ docker run -it --net host --name c1 busybox ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:19:5F:BC:F7
inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

eth0 Link encap:Ethernet HWaddr 08:00:27:85:8E:95

inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe85:8e95/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:190780 errors:0 dropped:0 overruns:0 frame:0
TX packets:58407 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:189367384 (180.5 MiB) TX bytes:3714724 (3.5 MiB)
...

#Display the interfaces on the host

$ ifconfig
docker0 Link encap:Ethernet HWaddr 02:42:19:5f:bc:f7
inet addr:172.17.0.1 Bcast:0.0.0.0 Mask:255.255.0.0
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

eth0 Link encap:Ethernet HWaddr 08:00:27:85:8e:95

inet addr:10.0.2.15 Bcast:10.0.2.255 Mask:255.255.255.0
inet6 addr: fe80::a00:27ff:fe85:8e95/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:190812 errors:0 dropped:0 overruns:0 frame:0
TX packets:58425 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:189369886 (189.3 MB) TX bytes:3716346 (3.7 MB)
...
```

In this example we can see that the host and container `c1` share the same
interfaces. This has some interesting implications. Traffic passes directly from
the container to the host interfaces.

With the `host` driver, Docker does not manage any portion of the container
networking stack such as port mapping or routing rules. This means that common
networking flags like `-p` and `--icc` have no meaning for the `host` driver. They
will be ignored. If the network admin wishes to provide access and policy to
containers then this will have to be self-managed onthe host or managed by another
tool.

Every container using the `host` network will all share the same host interfaces.
This makes `host` ill suited for multi-tenant or highly secure applications. `host`
containers will have access to every other container on the host.

Full host access and no automated policy management may make the `host` driver a
difficult fit as a general network driver. However, `host` does have some
interesting properties that may be applicable for use cases such as ultra high
performance applications, troubleshooting, or monitoring.

## <a name="nonedriver"></a>None (Isolated) Network Driver

Similar to the `host` network driver, the `none` network driver is essentially an
unmanaged networking option. Docker Engine will not create interfaces inside the
container, establishport mapping, or install routes for connectivity. A container
using `--net=none` will be completely isolated from other containers and the host.
The networking admin or external toolsmust be responsible for providing this
plumbing. In the following example we see that a container using `none` only has a
loopback interface and no other interfaces.

```bash
#Create a container using --net=none and display its interfaces
$ docker run -it --net none busybox ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
```

Unlike the `host` driver, the `none` driver will create a separate namespace for
each container. This guarantees container network isolation between any containers
and the host.

> Containers using `--net=none` or `--net=host` cannot be connected to any other

Docker networks.

Next: **[Physical Network Design Requirements](09-physical-networking.md)**

## <a name="requirements"></a>Physical Network Design Requirements
Docker Datacenter and Docker networking are designed to run over common data center
network infrastructure and topologies. Its centralized controller and fault-
tolerant cluster guarantee compatibility across a wide range of network
environments. The components that provide networking functionality (network
provisioning, MAC learning, overlay encryption) are either a part of Docker Engine,
UCP, or the Linux kernel itself. No extra components or special networking features
are required to run any of the built-in Docker networking drivers.
More specifically, the Docker built-in network drivers have NO requirements for:

- Multicast
- External key-value stores
- Specific routing protocols
- Layer 2 adjacencies between hosts
- Specific topologies such as spine & leaf, traditional 3-tier, and PoD designs.
Any of these topologies are supported.

This is in line with the Container Networking Model which promotes application
portability across all environments while still achieving the performance and
policy required of applications.

## <a name="sd"></a>Service Discovery Design Considerations

Docker uses embedded DNS to provide service discovery for containers running on a
single Docker Engine and `tasks` running in a Docker Swarm. Docker Engine has an
internal DNS server that provides name resolution to all of the containers on the
host in user-defined bridge, overlay, and MACVLAN networks. Each Docker container (
or `task` in Swarm mode) has a DNS resolver that forwards DNS queries to Docker
Engine, which acts as a DNS server. Docker Engine then checks if the DNS query
belongs to a container or `service` on network(s) that the requesting container
belongs to. If it does, then Docker Engine looks up the IP address that matches a
container, `task`, or`service`'s **name** in its key-value store and returns that
IP or `service` Virtual IP (VIP) back to the requester.

Service discovery is _network-scoped_, meaning only containers or tasks that are on

the same network can use the embedded DNS functionality. Containers not on the same
network cannot resolve each other's addresses. Additionally, only the nodes that
have containers or tasks on a particular network store that network's DNS entries.
This promotes security and performance.

If the destination container or `service` does not belong on same network(s) as

source container, then Docker Engine forwards the DNS query to the configured
default DNS server.


In this example there is a service of two containers called `myservice`. A second

service (`client`) exists on the same network. The `client` executes two `curl`
operations for `docker.com` and `myservice`. These are the resulting actions:

- DNS queries are initiated by `client` for `docker.com` and `myservice`.

- The container's built in resolver intercepts the DNS queries on `127.0.0.11:53`
and sends them to Docker Engine's DNS server.
- `myservice` resolves to the Virtual IP (VIP) of that service which is internally
load balanced to the individual task IP addresses. Container names will be resolved
as well, albeitdirectly to their IP address.
- `docker.com` does not exist as a service name in the `mynet` network and so the
request is forwarded to the configured default DNS server.

Next: **[Load Balancing Design Considerations](10-load-balancing.md)**

## <a name="lb"></a>Load Balancing Design Considerations

Load balancing is a major requirement in modern, distributed applications. Docker

Swarm mode introduced in 1.12 comes with a native internal and external load
balancing functionalities that utilize both `iptables` and `ipvs`, a transport-
layer load balancing inside the Linux kernel.

### Internal Load Balancing

When services are created in a Docker Swarm cluster, they are automatically
assigned a Virtual IP (VIP) that is part of the service's network. The VIP is
returned when resolving the service's name. Traffic to that VIP will be
automatically sent to all healthy tasks of that service across the overlay network.
This approach avoids any client-side load balancing because only a single IP is
returned to the client. Docker takes care of routing and equally distributing the
traffic across the healthy service tasks.


To see the VIP, run a `docker service inspect my_service` as follows:

```
# Create an overlay network called mynet
$ docker network create -d overlay mynet
a59umzkdj2r0ua7x8jxd84dhr

# Create myservice with 2 replicas as part of that network

$ docker service create --network mynet --name myservice --replicas 2 busybox ping
localhost
8t5r8cr0f0h6k2c3k7ih4l6f5

# See the VIP that was created for that service

$ docker service inspect myservice
...

"VirtualIPs": [
{
"NetworkID": "a59umzkdj2r0ua7x8jxd84dhr",
"Addr": "10.0.0.3/24"
},
]

```

> DNS round robin (DNS RR) load balancing is another load balancing option for
services (configured with `--endpoint-mode`). In DNS RR mode a VIP is not created
for each service. The Docker DNS server resolves a service name to individual
container IPs in round robin fashion.

###External Load Balancing (Docker Routing Mesh)

You can expose services externally by using the `--publish` flag when creating or
updating the service. Publishing ports in Docker Swarm mode means that every node
in your cluster will be listening on that port. But what happens if the service's
task isn't on the node that is listening on that port?

This is where routing mesh comes into play. Routing mesh is a feature introduced in
Docker 1.12 that combines `ipvs` and `iptables` to create a powerful cluster-wide
transport-layer (L4) load balancer. It allows all the Swarm nodes to accept
connections on the services' published ports. When any Swarm node receives traffic
destined to the published TCP/UDP port of a running `service`, it forwards it to
service's VIP using a pre-defined overlay network called `ingress`. The `ingress`
network behaves similarly to other overlay networks but its sole purpose is to
transport mesh routing traffic from external clients to cluster services. It uses
the same VIP-based internal load balancing as described in the previous section.

Once you launch services, you can create an external DNS record for your
applications and map it to any or all Docker Swarm nodes. You do not need to worry
about where your container is running as all nodes in your cluster look as one with
the routing mesh routing feature.

```
#Create a service with two replicas and export port 8000 on the cluster
$ docker service create --name app --replicas 2 --network appnet -p 8000:80 nginx
```


This diagram illustrates how the Routing Mesh works.

- A service is created with two replicas, and it is port mapped externally to port
`8000`.
- The routing mesh exposes port `8000` on each host in the cluster.
- Traffic destined for the `app` can enter on any host. In this case the external
LB sends the traffic to a host without a service replica.
- The kernel's IPVS load balancer redirects traffic on the `ingress` overlay
network to a healthy service replica.

Next: **[Network Security and Encryption Design Considerations](11-security.md)**

## <a name="security"></a>Network Security and Encryption Design Considerations

Network security is a top-of-mind consideration when designing and implementing

containerized workloads with Docker. In this section, we will go over three key
design considerations that are typically raised around Docker network security and
how you can utilize Docker features and best practices to address them.

### Container Networking Segmentation

Docker allows you to create an isolated network per application using the `overlay`
driver. By default different Docker networks are firewalled from eachother. This
approach provides a true network isolation at Layer 3. No malicious container can
communicate with your application's container unless it's on the same network or
your applications' containers expose services on the host port. Therefore, creating
networks for each applications adds another layer of security. The principles of
"Defense in Depth" still recommends application-level security to protect at L3 and
L7.

### Securing the Control Plane

Docker Swarm comes with integrated PKI. All managers and nodes in the Swarm have a
cryptographically signed identify in the form of a signed certificate. All manager-
to-manager and manager-to-node control communication is secured out of the box with
TLS. No need to generate certs externally or set up any CAs manually to get end-to-
end control plane traffic secured in Docker Swarm mode. Certificates are
periodically and automatically rotated.

### Securing the Data Plane

In Docker Swarm mode the data path (e.g. application traffic) can be encrypted out-
of-the-box. This feature uses IPSec tunnels to encrypt network traffic as it leaves
the source container and decrypts it as it enters the destination container. This
ensure that your application traffic is highly secure when it's in transit
regardless of the underlying networks. In a hybrid, multi-tenant, or multi-cloud
environment, it is crucial to ensure data is secure as it traverses networks you
might not have control over.

This diagram illustrates how to secure communication between two containers running
on different hosts in a Docker Swarm.


This feature works with the `overlay` driver in Swarm mode only and can be enabled
per network at the time of creation by adding the `--opt encrypted=true` option
(e.g `docker networkcreate -d overlay --opt encrypted=true <NETWORK_NAME>`). After
the network gets created, you can launch services on that network (e.g `docker
service create --network <NETWORK_NAME> <IMAGE> <COMMAND>`). When two tasks of the
same services are created on two different hosts, an IPsec tunnel is created
between them and traffic gets encrypted as it leaves the source host and gets
decrypted as it enters the destination host.

The Swarm leader periodically regenerates a symmetrical key and distributes it

securely to all cluster nodes. This key is used by IPsec to encrypt and decrypt
data plane traffic. The encryption is implemented via IPSec in host-to-host
transport mode using AES-GCM.

Next: **[IP Address Management](12-ipaddress-management.md)**

## <a name="ipam"></a>IP Address Management

The Container Networking Model (CNM) provides flexibility in how IP addresses are
managed. There are two methods for IP address management.

- CNM has a built-in IPAM driver that does simple allocation of IP addresses
globally for a cluster and prevents overlapping allocations. The built-in IPAM
driver is what is used by default if no other driver is specified.
- CNM has interfaces to use plug-in IPAM drivers from other vendors and the
community. These drivers can provide integration into existing vendor or self-built
IPAM tools.

Manual configuration of container IP addresses and network subnets can be done

using UCP, the CLI, or Docker APIs. The address request will go through the chosen
driver which will decide how to process the request.

Subnet size and design is largely dependent on a given application and the specific
network driver. IP address space design is covered in more depth for each [Network
Deployment Model](#models) in the next section. The uses of port mapping, overlays,
and MACVLAN all have implications on how IP addressing is arranged. In general,
container addressing falls into two buckets. Internal container networks (bridge
and overlay) address containers with IP addresses that are not routable on the
physical network by default. MACVLAN networks provide IP addresses to containers
that are on the subnet of the physical network. Thus, traffic from container
interfaces can be routable on the physical network. It is important to note that
subnets for internal networks (bridge, overlay) should not conflict with the IP
space of the physical underlay network. Overlapping address space can cause traffic
to not reach its destination.

Next: **[Network Troubleshooting](13-troubleshooting.md)**

## <a name="tshoot"></a>Network Troubleshooting

Docker network troubleshooting can be difficult for devops and network engineers.
With proper understanding of how Docker networking works and the right set of
tools, you can troubleshoot and resolve these network issues. One recommended way
is to use the [netshoot](https://github.com/nicolaka/netshoot) container to
troubleshoot network problems. The `netshoot` container has a set of powerful
networking troubleshooting tools that can be used to troubleshoot Docker network
issues.

Next: **[Network Deployment Models](14-network-models.md)**

## <a name="models"></a>Network Deployment Models
Docker Engine and community provide multiple drivers to use. These drivers can be
configured in multiple ways, and the physical network design and configuration will
also affect network behavior. This section looks at different configurations and
how they interoperate with the application and the physical network. This is not an
exhaustive list but a description ofcommon methods of deployment.


Back to [Concepts](README.md)
or
On to [Tutorials](../tutorials.md)
## <a name="challenges"></a>Challenges of Networking Containers and Microservices

Microservices practices have increased the scale of applications which has put even
more importance on the methods of connectivity and isolation that we provide to
applications. The Docker networking philosophy is application driven. It aims to
provide options and flexibility to the network operators as well as the right level
of abstraction to the application developers.

Like any design, network design is a balancing act. __Docker Datacenter__ and the
Docker ecosystem provides multiple tools to network engineers to achieve the best
balance for their applications and environments. Each option provides different
benefits and tradeoffs. The remainder of this guide details each of these choices
so network engineers can understand what might be best for their environments.

Docker has developed a new way of delivering applications, and with that,
containers have also changed some aspects of how we approach networking. The
following topics are common design themes for containerized applications:

- __Portability__
- _How do I guarantee maximum portability across diverse network
environments while taking advantage of unique network characteristics?_

- __Service Discovery__
- _How do I know where services are living as they are scaled up and down?_

- __Load Balancing__
- _How do I share load across services as services themselves are brought
up and scaled?_

- __Security__
- _How do I segment to prevent the right containers from accessing each
other?_
- _How do I guarantee that a container with application and cluster control
traffic is secure?_

- __Performance__
- _How do I provide advanced network services while minimizing latency and
maximizing bandwidth?_

- __Scalability__
- _How do I ensure that none of these characteristics are sacrificed when
scaling applications across many hosts?_

### <a name="concepts"></a>Concepts

This section contains 14 different short networking concept chapters. Feel free to
skip right to the [tutorials](../tutorials.md) if you feel you are ready and come
back here if you need a refresher. The concept chapters are:

1. [The Container Networking Model](01-cnm.md)

1. [Drivers](02-drivers.md)

1. [Linux Networking Fundamentals](03-linux-networking.md)

1. [Docker Network Control Plane](04-docker-network-cp.md)

1. [Bridge Networks](05-bridge-networks.md)

1. [Overlay Networks](06-overlay-networks.md)

1. [MACVLAN](07-macvlan.md)

1. [Host (Native) Network Driver](08-host-networking.md)

1. [Physical Network Design Requirements](09-physical-networking.md)

1. [Load Balancing Design Considerations](10-load-balancing.md)

1. [Security](11-security.md)

1. [IP Address Management](12-ipaddress-management.md)

1. [Troubleshooting](13-troubleshooting.md)

1. [Network Deployment Models](14-network-models.md)

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
||||||||||||

LABORATORIOS

[node1] (local) [email protected] ~/labs/networking

$ ls
A1-network-basics.md A3-overlay-networking.md README.md img
tutorials.md
A2-bridge-networking.md A4-HTTP Routing Mesh.md concepts
scratch.md
[node1] (local) [email protected] ~/labs/networking
$ cat A*.md
# Docker Networking Basics

# Lab Meta
> **Difficulty**: Beginner

> **Time**: Approximately 10 minutes

In this lab you'll look at the most basic networking components that come with a
fresh installation of Docker.

You will complete the following steps as part of this lab.

- [Step 1 - The `docker network` command](#docker_network)- [Step 2 - List

networks](#list_networks)- [Step 3 - Inspect a network](#inspect)- [Step 4 - List
network driver plugins](#list_drivers)

# Prerequisites

You will need all of the following to complete this lab:

- A Linux-based Docker Host running Docker 1.12 or higher

# <a name="docker_network"></a>Step 1: The `docker network` command

The `docker network` command is the main command for configuring and managing
container networks.
Run a simple `docker network` command from any of your lab machines.
```
$ docker network

Usage: docker network COMMAND

Manage Docker networks

Options:
--help Print usage

Commands:
connect Connect a container to a network
create Create a network
disconnect Disconnect a container from a network
inspect Display detailed information on one or more networks
ls List networks
rm Remove one or more networks

Run 'docker network COMMAND --help' for more information on a command.

```

The command output shows how to use the command as well as all of the `docker
network` sub-commands. As you can see from the output, the `docker network` command
allows you to create new networks, list existing networks, inspect networks, and
remove networks. It also allows you to connect and disconnect containers from
networks.

# <a name="list_networks"></a>Step 2: List networks

Run a `docker network ls` command to view existing container networks on the
current Docker host.

```
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
1befe23acd58 bridge bridge local
726ead8f4e6b host host local
ef4896538cc7 none null local
```

The output above shows the container networks that are created as part of a
standard installation of Docker.

New networks that you create will also show up in the output of the `docker network
ls` command.

You can see that each network gets a unique `ID` and `NAME`. Each network is also
associated with a single driver. Notice that the "bridge" network and the "host"
network have the same name as their respective drivers.

# <a name="inspect"></a>Step 3: Inspect a network

The `docker network inspect` command is used to view network configuration details.
These details include; name, ID, driver, IPAM driver, subnet info, connected
containers, and more.

Use `docker network inspect` to view configuration details of the container

networks on your Docker host. The command below shows the details of the network
called `bridge`.

```
$ docker network inspect bridge
[
{
"Name": "bridge",
"Id": "1befe23acd58cbda7290c45f6d1f5c37a3b43de645d48de6c1ffebd985c8af4b",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "172.17.0.0/16",
"Gateway": "172.17.0.1"
}
]
},
"Internal": false,
"Containers": {},
"Options": {
"com.docker.network.bridge.default_bridge": "true",
"com.docker.network.bridge.enable_icc": "true",
"com.docker.network.bridge.enable_ip_masquerade": "true",
"com.docker.network.bridge.host_binding_ipv4": "0.0.0.0",
"com.docker.network.bridge.name": "docker0",
"com.docker.network.driver.mtu": "1500"
},
"Labels": {}
}
]
```
> **NOTE:** The syntax of the `docker network inspect` command is `docker network
inspect <network>`, where `<network>` can be either network name or network ID. In
the example above we are showing the configuration details for the network called
"bridge". Do not confuse this with the "bridge" driver.

# <a name="list_drivers"></a>Step 4: List network driver plugins

The `docker info` command shows a lot of interesting information about a Docker
installation.

Run a `docker info` command on any of your Docker hosts and locate the list of
network plugins.

```
$ docker info
Containers: 0
Running: 0
Paused: 0
Stopped: 0
Images: 0
Server Version: 1.12.3
Storage Driver: aufs
<Snip>
Plugins:
Volume: local
Network: bridge host null overlay <<<<<<<<
Swarm: inactive
Runtimes: runc
<Snip>
```

The output above shows the **bridge**, **host**, **null**, and **overlay** drivers.
# Bridge networking

# Lab Meta

> **Difficulty**: Intermediate

> **Time**: Approximately 15 minutes

In this lab you'll learn how to build, manage, and use **bridge** networks.

You will complete the following steps as part of this lab.

- [Step 1 - The default **bridge** network](#default_bridge)

- [Step 2 - Connect a container to the default *bridge* network]
(#connect_container)
- [Step 3 - Test the network connectivity](#ping_local)
- [Step 4 - Configure NAT for external access](#nat)

# Prerequisites

You will need all of the following to complete this lab:

- A Linux-based Docker host running Docker 1.12 or higher

- The lab was built and tested using Ubuntu 16.04
# <a name="default_bridge"></a>Step 1: The default **bridge** network

Every clean installation of Docker comes with a pre-built network called

**bridge**. Verify this with the `docker network ls` command.

```
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
1befe23acd58 bridge bridge local
726ead8f4e6b host host local
ef4896538cc7 none null local
```

The output above shows that the **bridge** network is associated with the *bridge*
driver. It's important to note that the network and the driver are connected, but
they are not the same. In this example the network and the driver have the same
name - but they are not the same thing!

The output above also shows that the **bridge** network is scoped locally. This
means that the network only exists on this Docker host. This is true of all
networks using the *bridge*driver - the *bridge* driver provides single-host
networking.

All networks created with the *bridge* driver are based on a Linux bridge (a.k.a. a
virtual switch).

Install the `brctl` command and use it to list the Linux bridges on your Docker
host.

```
# Install the brctl tools

$ apt-get install bridge-utils

<Snip>

# List the bridges on your Docker host

$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242f17f89a6 no
```

The output above shows a single Linux bridge called **docker0**. This is the bridge
that was automatically created for the **bridge** network. You can see that it has
no interfaces currently connected to it.

You can also use the `ip` command to view details of the **docker0** bridge.

```
$ ip a
<Snip>
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
group default
link/ether 02:42:f1:7f:89:a6 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
inet6 fe80::42:f1ff:fe7f:89a6/64 scope link
valid_lft forever preferred_lft forever
```
# <a name="connect-container"></a>Step 2: Connect a container

The **bridge** network is the default network for new containers. This means that
unless you specify a different network, all new containers will be connected to the
**bridge** network.

Create a new container.

```
$ docker run -dt ubuntu sleep infinity
6dd93d6cdc806df6c7812b6202f6096e43d9a013e56e5e638ee4bfb4ae8779ce
```

This command will create a new container based on the `ubuntu:latest` image and
will run the `sleep` command to keep the container running in the background. As no
network was specified on the `docker run` command, the container will be added to
the **bridge** network.

Run the `brctl show` command again.

```
$ brctl show
bridge name bridge id STP enabled interfaces
docker0 8000.0242f17f89a6 no veth3a080f
```

Notice how the **docker0** bridge now has an interface connected. This interface
connects the **docker0** bridge to the new container just created.

Inspect the **bridge** network again to see the new container attached to it.

```
$ docker network inspect bridge
<Snip>
"Containers": {
"6dd93d6cdc806df6c7812b6202f6096e43d9a013e56e5e638ee4bfb4ae8779ce": {
"Name": "reverent_dubinsky",
"EndpointID":
"dda76da5577960b30492fdf1526c7dd7924725e5d654bed57b44e1a6e85e956c",
"MacAddress": "02:42:ac:11:00:02",
"IPv4Address": "172.17.0.2/16",
"IPv6Address": ""
}
},
<Snip>
```

# <a name="ping_local"></a>Step 3: Test network connectivity

The output to the previous `docker network inspect` command shows the IP address of
the new container. In the previous example it is "172.17.0.2" but yours might be
different.

Ping the IP address of the container from the shell prompt of your Docker host.
Remember to use the IP of the container in **your** environment.

```
$ ping 172.17.0.2
64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.069 ms
64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.052 ms
64 bytes from 172.17.0.2: icmp_seq=3 ttl=64 time=0.050 ms
64 bytes from 172.17.0.2: icmp_seq=4 ttl=64 time=0.049 ms
64 bytes from 172.17.0.2: icmp_seq=5 ttl=64 time=0.049 ms
^C
--- 172.17.0.2 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3999ms
rtt min/avg/max/mdev = 0.049/0.053/0.069/0.012 ms
```

Press `Ctrl-C` to stop the ping. The replies above show that the Docker host can
ping the container over the **bridge** network.

Log in to the container, install the `ping`

program and ping `www.dockercon.com`.

```
# Get the ID of the container started in the previous step.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS NAMES
6dd93d6cdc80 ubuntu "sleep infinity" 5 mins Up reverent_dubinsky

# Exec into the container

$ docker exec -it 6dd93d6cdc80 /bin/bash

# Update APT package lists and install the iputils-ping package

root@6dd93d6cdc80:/# apt-get update
<Snip>

apt-get install iputils-ping

Reading package lists... Done
<Snip>

# Ping www.dockercon.com from within the container

root@6dd93d6cdc80:/# ping www.dockercon.com
PING www.dockercon.com (104.239.220.248) 56(84) bytes of data.
64 bytes from 104.239.220.248: icmp_seq=1 ttl=39 time=93.9 ms
64 bytes from 104.239.220.248: icmp_seq=2 ttl=39 time=93.8 ms
64 bytes from 104.239.220.248: icmp_seq=3 ttl=39 time=93.8 ms
^C
--- www.dockercon.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 93.878/93.895/93.928/0.251 ms
```

This shows that the new container can ping the internet and therefore has a valid
and working network configuration.

# <a name="nat"></a>Step 4: Configure NAT for external connectivity

In this step we'll start a new **NGINX** container and map port 8080 on the Docker
host to port 80 inside of the container. This means that traffic that hits the
Docker host on port 8080 will be passed on to port 80 inside the container.

> **NOTE:** If you start a new container from the official NGINX image without
specifying a command to run, the container will run a basic web server on port 80.
Start a new container based off the official NGINX image.

```
$ docker run --name web1 -d -p 8080:80 nginx
Unable to find image 'nginx:latest' locally
latest: Pulling from library/nginx
386a066cd84a: Pull complete
7bdb4b002d7f: Pull complete
49b006ddea70: Pull complete
Digest: sha256:9038d5645fa5fcca445d12e1b8979c87f46ca42cfb17beb1e5e093785991a639
Status: Downloaded newer image for nginx:latest
b747d43fa277ec5da4e904b932db2a3fe4047991007c2d3649e3f0c615961038
```

Check that the container is running and view the port mapping.

```
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED
STATUS PORTS NAMES
b747d43fa277 nginx "nginx -g 'daemon off" 3 seconds ago Up
2 seconds 443/tcp, 0.0.0.0:8080->80/tcp web1
6dd93d6cdc80 ubuntu "sleep infinity" About an hour ago
Up About an hour reverent_dubinsky
```

There are two containers listed in the output above. The top line shows the new
**web1** container running NGINX. Take note of the command the container is running
as well as the portmapping - `0.0.0.0:8080->80/tcp` maps port 8080 on all host
interfaces to port 80 inside the **web1** container. This port mapping is what
effectively makes the containers web serviceaccessible from external sources (via
the Docker hosts IP address on port 8080).

Now that the container is running and mapped to a port on a host interface you can
test connectivity to the NGINX web server.

To complete the following task you will need the IP address of your Docker host.
This will need to be an IP address that you can reach (e.g. if your lab is in AWS
this will need to bethe instance's Public IP).

Point your web browser to the IP and port 8080 of your Docker host. The following
example shows a web browser pointed to `52.213.169.69:8080`


If you try connecting to the same IP address on a different port number it will
fail.

If for some reason you cannot open a session from a web broswer, you can connect
from your Docker host using the `curl` command.

```
$ curl 127.0.0.1:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<Snip>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
```

If you try and curl the IP address on a different port number it will fail.

> **NOTE:** The port mapping is actually port address translation (PAT).
# Overlay networking and service discovery

# Lab Meta

> **Difficulty**: Intermediate

> **Time**: Approximately 20 minutes

In this lab you'll learn how to build, manage, and use an **overlay** network with
a *service* in *Swarm mode*.

You will complete the following steps as part of this lab.

- [Step 1 - Create a new Swarm](#swarm_init)

- [Step 2 - Create an overlay network](#create_network)
- [Step 3 - Create a service](#create_service)
- [Step 4 - Test the network](#test)
- [Step 5 - Test service discovery](#discover)

# Prerequisites

You will need all of the following to complete this lab:

- Two Linux-based Docker hosts running **Docker 1.12** or higher in Engine mode
(i.e. not yet configured for Swarm mode). You should use **node1** and **node2**
from your lab.

# <a name="swarm_init"></a>Step 1: Create a new Swarm

In this step you'll initialize a new Swarm, join a single worker node, and verify
the operations worked.

1. Execute the following command on **node1**.

```
node1$ docker swarm init
Swarm initialized: current node (cw6jpk7pqfg0jkilff5hr8z42) is now a manager.
To add a worker to this swarm, run the following command:

docker swarm join \

--token SWMTKN-1-3n2iuzpj8jynx0zd8axr0ouoagvy0o75uk5aqjrn0297j4uaz7-
63eslya31oza2ob78b88zg5xe \
172.31.34.123:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and
follow the instructions.
```

2. Copy the entire `docker swarm join` command that is displayed as part of the
output from the command.
3. Paste the copied command into the terminal of **node2**.

```
node2$ docker swarm join \
> --token SWMTKN-1-3n2iuzpj8jynx0zd8axr0ouoagvy0o75uk5aqjrn0297j4uaz7-
63eslya31oza2ob78b88zg5xe \
> 172.31.34.123:2377

This node joined a swarm as a worker.

```

4. Run a `docker node ls` on **node1** to verify that both nodes are part of the
Swarm.

```
node1$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER
STATUS
4nb02fhvhy8sb0ygcvwya9skr ip-172-31-43-74 Ready Active
cw6jpk7pqfg0jkilff5hr8z42 * ip-172-31-34-123 Ready Active Leader
```

The `ID` and `HOSTNAME` values may be different in your lab. The important
thing to check is that both nodes have joined the Swarm and are *ready* and
*active*.

# <a name="create_network"></a>Step 2: Create an overlay network

Now that you have a Swarm initialized it's time to create an **overlay** network.

1. Create a new overlay network called "overnet" by executing the following command
on **node1**.

```
node1$ docker network create -d overlay overnet
0cihm9yiolp0s9kcczchqorhb
```

2. Use the `docker network ls` command to verify the network was created
successfully.

```
node1$ docker network ls
NETWORK ID NAME DRIVER SCOPE
1befe23acd58 bridge bridge local
0ea6066635df docker_gwbridge bridge local
726ead8f4e6b host host local
8eqnahrmp9lv ingress overlay swarm
ef4896538cc7 none null local
0cihm9yiolp0 overnet overlay swarm
```

The new "overnet" network is shown on the last line of the output above. Notice
how it is associated with the **overlay** driver and is scoped to the entire Swarm.

> **NOTE:** The other new networks (ingress and docker_gwbridge) were created
automatically when the Swarm cluster was created.

3. Run the same `docker network ls` command from **node2**

 ```
node2$ docker network ls
NETWORK ID NAME DRIVER SCOPE
b76635120433 bridge bridge local
ea13f975a254 docker_gwbridge bridge local
73edc8c0cc70 host host local
8eqnahrmp9lv ingress overlay swarm
c4fb141606ca none null local
```

Notice that the "overnet" network does not appear in the list. This is because
Docker only extends overlay networks to hosts when they are needed. This is usually
when a host runsa task from a service that is created on the network. We will see
this shortly.

4. Use the `docker network inspect` command to view more detailed information about
the "overnet" network. You will need to run this command from **node1**.

```
node1$ docker network inspect overnet
[
{
"Name": "overnet",
"Id": "0cihm9yiolp0s9kcczchqorhb",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": []
},
"Internal": false,
"Containers": null,
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "257"
},
"Labels": null
}
]
```

# <a name="create_service"></a>Step 3: Create a service

Now that you have a Swarm initialized and an overlay network, it's time to create a
service that uses the network.

1. Execute the following command from **node1** to create a new service called
*myservice* on the *overnet* network with two tasks/replicas.

```
node1$ docker service create --name myservice \
--network overnet \
--replicas 2 \
ubuntu sleep infinity

e9xu03wsxhub3bij2tqyjey5t
```
2. Verify that the service is created and both replicas are up.

```
node1$ docker service ls
ID NAME REPLICAS IMAGE COMMAND
e9xu03wsxhub myservice 2/2 ubuntu sleep infinity
```

The `2/2` in the `REPLICAS` column shows that both tasks in the service are up
and running.

3. Verify that a single task (replica) is running on each of the two nodes in the
Swarm.

```
node1$ docker service ps myservice
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR
5t4wh...fsvz myservice.1 ubuntu node1 Running Running 2 mins
8d9b4...te27 myservice.2 ubuntu node2 Running Running 2 mins
```

The `ID` and `NODE` values might be different in your output. The important
thing to note is that each task/replica is running on a different node.

4. Now that **node2** is running a task on the "overnet" network it will be able to
see the "overnet" network. Run the following command from **node2** to verify this.

```
node2$ docker network ls
NETWORK ID NAME DRIVER SCOPE
b76635120433 bridge bridge local
ea13f975a254 docker_gwbridge bridge local
73edc8c0cc70 host host local
8eqnahrmp9lv ingress overlay swarm
c4fb141606ca none null local
0cihm9yiolp0 overnet overlay swarm
```

5. Run the following command on **node2** to get more detailed information about
the "overnet" network and obtain the IP address of the task running on **node2**.

```
node2$ docker network inspect overnet
[
{
"Name": "overnet",
"Id": "0cihm9yiolp0s9kcczchqorhb",
"Scope": "swarm",
"Driver": "overlay",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": null,
"Config": [
{
"Subnet": "10.0.0.0/24",
"Gateway": "10.0.0.1"
}
 ]
},
"Internal": false,
"Containers": {
"286d2e98c764...37f5870c868": {
"Name": "myservice.1.5t4wh7ngrzt9va3zlqxbmfsvz",
"EndpointID": "43590b5453a...4d641c0c913841d657",
"MacAddress": "02:42:0a:00:00:04",
"IPv4Address": "10.0.0.4/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "257"
},
"Labels": {}
}
]
```

You should note that as of Docker 1.12, `docker network inspect` only shows
containers/tasks running on the local node. This means that `10.0.0.4` is the IPv4
address of the containerrunning on **node2**. Make a note of this IP address for
the next step (the IP address in your lab might be different than the one shown
here in the lab guide).

# <a name="test"></a>Step 4: Test the network

To complete this step you will need the IP address of the service task running on
**node2** that you saw in the previous step.

1. Execute the following commands from **node1**.

```
node1$ docker network inspect overnet
[
{
"Name": "overnet",
"Id": "0cihm9yiolp0s9kcczchqorhb",
"Scope": "swarm",
"Driver": "overlay",
"Containers": {
"053abaa...e874f82d346c23a7a": {
"Name": "myservice.2.8d9b4i6vnm4hf6gdhxt40te27",
"EndpointID": "25d4d5...faf6abd60dba7ff9b5fff6",
"MacAddress": "02:42:0a:00:00:03",
"IPv4Address": "10.0.0.3/24",
"IPv6Address": ""
}
},
"Options": {
"com.docker.network.driver.overlay.vxlanid_list": "257"
},
"Labels": {}
}
]
```

Notice that the IP address listed for the service task (container) running on
**node1** is different to the IP address for the service task running on **node2**.
Note also that they are one the sane "overnet" network.

2. Run a `docker ps` command to get the ID of the service task on **node1** so that
you can log in to it in the next step.

```
node1$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS
NAMES
053abaac4f93 ubuntu:latest "sleep infinity" 19 mins ago Up 19 mins
myservice.2.8d9b4i6vnm4hf6gdhxt40te27
<Snip>
```

3. Log on to the service task. Be sure to use the container `ID` from your
environment as it will be different from the example shown below.

```
node1$ docker exec -it 053abaac4f93 /bin/bash
root@053abaac4f93:/#
```

4. Install the ping command and ping the service task running on **node2**.

```
root@053abaac4f93:/# apt-get update && apt-get install iputils-ping
<Snip>
root@053abaac4f93:/#
root@053abaac4f93:/#
root@053abaac4f93:/# ping 10.0.0.4
PING 10.0.0.4 (10.0.0.4) 56(84) bytes of data.
64 bytes from 10.0.0.4: icmp_seq=1 ttl=64 time=0.726 ms
64 bytes from 10.0.0.4: icmp_seq=2 ttl=64 time=0.647 ms
^C
--- 10.0.0.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.647/0.686/0.726/0.047 ms
```

The output above shows that both tasks from the **myservice** service are on
the same overlay network spanning both nodes and that they can use this network to
communicate.

# <a name="discover"></a>Step 5: Test service discovery

Now that you have a working service using an overlay network, let's test service
discovery.

If you are not still inside of the container on **node1**, log back into it with
the `docker exec` command.

1. Run the following command form inside of the container on **node1**.

```
root@053abaac4f93:/# cat /etc/resolv.conf
search eu-west-1.compute.internal
nameserver 127.0.0.11
options ndots:0
 ```

The value that we are interested in is the `nameserver 127.0.0.11`. This value
sends all DNS queries from the container to an embedded DNS resolver running inside
the container listening on 127.0.0.11:53. All Docker container run an embedded DNS
server at this address.

> **NOTE:** Some of the other values in your file may be different to those
shown in this guide.

2. Try and ping the `myservice` name from within the container.

```
root@053abaac4f93:/# ping myservice
PING myservice (10.0.0.2) 56(84) bytes of data.
64 bytes from ip-10-0-0-2.eu-west-1.compute.internal (10.0.0.2): icmp_seq=1
ttl=64 time=0.020 ms
64 bytes from ip-10-0-0-2.eu-west-1.compute.internal (10.0.0.2): icmp_seq=2
ttl=64 time=0.041 ms
64 bytes from ip-10-0-0-2.eu-west-1.compute.internal (10.0.0.2): icmp_seq=3
ttl=64 time=0.039 ms
^C
--- myservice ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 0.020/0.033/0.041/0.010 ms
```

The output clearly shows that the container can ping the `myservice` service by
name. Notice that the IP address returned is `10.0.0.2`. In the next few steps
we'll verify that this address is the virtual IP (VIP) assigned to the `myservice`
service.

3. Type the `exit` command to leave the `exec` container session and return to the
shell prompt of your **node1** Docker host.

4. Inspect the configuration of the `myservice` service and verify that the VIP
value matches the value returned by the previous `ping myservice` command.

```
node1$ docker service inspect myservice
[
{
"ID": "e9xu03wsxhub3bij2tqyjey5t",
"Version": {
"Index": 20
},
"CreatedAt": "2016-11-23T09:28:57.888561605Z",
"UpdatedAt": "2016-11-23T09:28:57.890326642Z",
"Spec": {
"Name": "myservice",
"TaskTemplate": {
"ContainerSpec": {
"Image": "ubuntu",
"Args": [
"sleep",
"infinity"
]
},
<Snip>
 "Endpoint": {
"Spec": {
"Mode": "vip"
},
"VirtualIPs": [
{
"NetworkID": "0cihm9yiolp0s9kcczchqorhb",
"Addr": "10.0.0.2/24"
}
<Snip>
```

Towards the bottom of the output you will see the VIP of the service listed.
The VIP in the output above is `10.0.0.2` but the value may be different in your
setup. The important point to note is that the VIP listed here matches the value
returned by the `ping myservice` command.

Feel free to create a new `docker exec` session to the service task (container)
running on **node2** and perform the same `ping service` command. You will get a
response form the sameVIP.
# HTTP Routing Mesh (HRM)

> **NOTE:** This lab assumes two things.

>1. You have configured DNS name resolution for red.example.com and
white.example.com to point to a load balancer. This name resolution is required for
your laptop/desktop and notthe Docker nodes that will make up your UCP cluster.
Therefore, it can be as simple as a couple of entries in the local `hosts` file of
your laptop or desktop. As long as your web browser can resolve red.example.com and
white.example.com to a load balancer in front of your Swarm this lab will work.
>2. You have configured an external load balancer to accept connections for the
two DNS names above and to load balance across all nodes in a UCP cluster.

# Lab Meta

> **Difficulty**: Intermediate

> **Time**: Approximately 15 minutes

In this lab you'll learn how to configure and use the *HTTP Routing Mesh* with
*Docker Datacenter*.

You will complete the following steps as part of this lab.

- [Step 1 - Enable the HTTP Routing Mesh (HRM)](#enable_hrm)

- [Step 2 - Verify the HRM](#verify_hrm)
- [Step 3 - Create the RED service](#create_red)
- [Step 4 - Create the WHITE service](#create_white)
- [Step 5 - Test the configuration](#test)

# Prerequisites

You will need all of the following to complete this lab:

- A UCP Cluster running **Docker 1.12** or higher

- Credentials to log in to UCP, create services, and enable the HRM
- Name resolution configured for two DNS names (your lab instructor will give you
these)
Your instructor will provide you with the details you require.

> **NOTE:** Throughout this guide we will use *red.example.com* and

*white.example.com*. As per the note above, you will need to configure this
yourself. You can also substitute othernames if you like.That would mean that any
time you see *red.example.com* and *white.example.com* you will need to substitute
these for *red.* and *white.<your-domain-goes-here>*.

# <a name="enable_hrm"></a>Step 1: Enable the HTTP Routing Mesh (HRM)

1. Use a web browser to connect to the Login page of your UCP cluster

2. Enter your credentials as supplied by your lab instructor

3. Navigate to `Admin Settings` > `Routing Mesh` and enable the HTTP Routing Mesh
(HRM) on port 80.


The HRM is now configured and ready to use.

# <a name="verify_hrm"></a>Step 2: Verify the HRM

Enabling the HRM creates a new *service* called `ucp-hrm` and a new network called
`ucp-hrm`. In this step we'll confirm that both of these constructs have been
created correctly.

Execute the following steps in the UCP web UI.

1. Navigate to `Resources` > `Networks` and check for the presence of the `ucp-hrm`
network. You may have to `search` for it.


The network shows as an overlay network scoped to the entire Swarm cluster.

2. Navigate to `Resources` > `Services` and click the checkbox to `Show system
services`.


The image above shows the `ucp-hrm` service up and running.

You have now verified that the HRM was configured successfully.

In the next two steps you'll create two services. Each service will based off the
same `ehazlett/docker-demo:latest` image, and runs a web server that counts
containers and requests. You will configure each service with a different number of
tasks and each with a different value in the `TITLE` variable.

# <a name="create_red"></a>Step 3: Create the RED service

In this step you'll create a new service called **RED**, and configure it to use
the HRM.

1. In `DDC` click `Resources` > `Services` and then `+Create Service`.

2. Configure the service as follows (leave all other options as default and
remember to substitute "red.example.com" with the DNS name from your environment):
- Name: `RED`
- Image: `ehazlett/docker-demo:latest`
- Scale: `10`
- Arguments: `-close-conn`
- Published port: Port = `8080/tcp`, Public Port = `5000`
- Attached Networks: `ucp-hrm`
- Labels: `com.docker.ucp.mesh.http` = `8080=http://red.example.com`
- Environment Variables: `TITLE` = `RED`

It will take a few minutes for this service to pull down the image and start.
Continue with the next step to create the **WHITE** service.

# <a name="create_white"></a>Step 4: Create the WHITE service

In this step you'll create a new service called **WHITE**. The service will be very
similar to the **RED** service created in the previous step.

1. In `DDC` click `Resources` > `Services` and then `+Create Service`.

2. Configure the service as follows (leave all other options as default and
remember to substitute "red.example.com" with the DNS name from your environment):
- Name: `RWHITE`
- Image: `ehazlett/docker-demo:latest`
- Scale: `5`
- Arguments: `-close-conn`
- Published port: Port = `8080/tcp`, Public Port = `5001`
- Attached Networks: `ucp-hrm`
- Labels: `com.docker.ucp.mesh.http` = `8080=http://white.example.com`
- Environment Variables: `TITLE` = `WHITE`

This service will start instantaneously as the image is already pulled on every
host in your UCP cluster.

3. Verify that both services are up and running by clicking `Resources` >
`Services` and checking that both services are running as shown below.


You now have two services running. Both are connected to the `ucp-hrm` network and
both have the `com.docker.ucp.mesh.http` label. The **RED** service is associated
with HTTP requestsfor `red.example.com` and the **WHITE** service is associated
with HTTP requests for `white.example.com`. This mapping of labels to URLs is
leveraged by the `ucp-hrm` service which ispublished on port 80.

# <a name="test"></a>Step 5: Test the configuration

> **NOTE: DNS name resolution is required for this step. This can obviously be via
the local hosts file, but this step will not work unless the URLs specified in the
`com.docker.ucp.mesh.http` labels resolve to the UCP cluster nodes (probably via a
load balancer).**

In this step you will use your web browser to issue HTTP requests to
`red.example.com` and `white.example.com`. DNS name resolution is configured so
that these URLs resolve to a load balancer which in turn balances requests across
all nodes in the UCP cluster.

> Remember to substitute `example.com` with the domain supplied by your lab
instructor.

1. Open a web browser tab and point it to `red.example.com`.


The text below the whale saying "RED" indicates that this request was answered by
the **RED** service. This is because the `TITLE` environment variable for the
**RED** service was configured to display "RED" here. You also know it is the
**RED** service as this was the service configured with 10 replicas (containers).

2. Open another tab to `white.example.com`.


The output above shows that this request was routed to the **WHITE** service as
it displays "WHITE" below the whale and only has 5 replicas (containers).

Congratulations. You configured two services in the same Swarm (UCP cluster) to
respond to requests on port 80. Traffic to each service is routed based on the URL
included in the `host` field of the HTTP header.

Requests arrive to the Swarm on port 80 and are forwarded to the `ucp-hrm` system
service. The `ucp-hrm` service inspects the HTTP headers of requests and routes
them to the service with the matching `com.docker.ucp.mesh.http` label.

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|||||||||||||||||||||||||||||||||

##Service Discovery & Load Balancing

Service discovery is increasingly important in a containerized world. To scale apps
developers have broken them in to smaller pieces that can be distributed across
different machines t
o provide load balancing and fault tolerance. This presents challenges from a
networking perspective. The challenge is mapping a container to its location (the
container's IP address).
Containers may be created and destroyed frequently and scheduled across different
hosts dynamically. Containers must be able to register themsevles with a mapping
authority and other
services must be able to query this authority to find the location of those
services.
When a new service (container) is created it should be able register itself with a
service discovery authority. The ```IP address:port``` of the container will then
be registered to th
at service. The mapping of services to containers should dynamically be updated
when containers are added or removed from a service. This diagram shows an example
of the service discovery process. A container is created, it is registered to a
specific service, and then a load balancer is updated with the containers location
and service. Converseley when a container
is stopped or becomes unhealthy then it should be removed from the service and load
balancer.
<br>
<p align="center">
<img src="./img/servicediscovery.png" width=100%>
</p>

The mechanisms that provide service discovery and load balancing can take many
forms. They can be external service or provided natively within Docker without
extra infrastructure. In D
ocker Swarm Mode automatic service discovery and load balancing is provided right
out of the box. A service can be defined and traffic is load balanced via DNS to
containers. DNS load
balancing is covered later in this guide.

External solutions for service discovery and/or load balancing is also possible and
may be desired to levarage existing infrastructure or to take advantage of special
features. Common
external service discovery mechanisms include Consul, etcd, and Zookeeper. Common
external load balancers include HAproxy, Nginx, F5, and many more.
<br><br>
<br>
<br><br>
<br>
<br>
<br>

#####Underlay and Overlay Networks

The default behavior of the Docker bridge and overlay driver is to provide an
internal subnet for each bridge or overlay network. Bridge networks only have local
significance on each h
ost and they provide port mapping to allow containers to reach outside of the host.
Overlay networks span across hosts and use VXLAN tunneling to keep its IP namespace
separate from ot
her networks.

In this context we call the physical network (comprised of the host network
adapters and upstream switches & routers) the underlay network. Between port
mapping and overlay tunneling,containers only receive private IP addresses and are
not part of the underlay network. This has many advantages.

* Portability is increased because applications are not tied to the design of the
physical network
* Agility is improved because new networks can be created and destroyed without
having to reconfigure physical infrastrture

There are scenarios where it may be more desirable to place containers directly on
the underlay network so that they receive an IP address that is on the underlay
subnet. These scenarios include:

- Legacy Applications - Some legacy applications may use hard-coded IP addresses or

ports. Applications that require themselves to be advertised on a certain port can
cause difficulties with port-contention and may not be suitable to exist on a
private bridge or overlay network.
- Protocol & Application Incompatibilities - Some protocols and applications are
incompatible with NAT and port mapping.

#####Attaching Containers Directly to Underlay Networks

Docker networking also has the ability to place containers directly on the underlay
network with the overlay and bridge drivers. To achieve this port-mapping and NAT
are turned out and containers are given IP addresses that are on the physical
network subnet.

<br>
<br>

#####Policy with Docker Networks

Docker networks can easily be created dynamically to define the policies defined by
applications. In this example we use a combination of overlay and bridge networks
to connect multiple containers. We deploy three networks:

- `backnet` for communication between the `web` and `db` containers. It is an

overlay network that can span multiple hosts. Purposefully, we will not provide any
connectivity between'backnet' and the outside world. It is purely an internal
network.
- `monitor-net` for communication between the `monitor` container and all other
containers on a given host. It is a bridge network as the monitoring communications
are only local to each host.
- `docker0` is the default bridge network. In this case it's not used for
container to container communication. It's purely a network created to provide
outside access to the `web` containers.

<p align="center">
<img src="./img/app-policy.png" width=70%>
</p>

##Docker Network Deployment Models

#####Default Bridge Mode (Bridge + NAT/Port Mapping)
This is the default deployment of the Docker bridge driver. Each bridge network
`br0` is local to the host. Traffic leaving or entering the host is NAT/port-mapped
to the IP address of the network interface of the host.

```
$ docker network create -d bridge --subnet 10.0.0.0/24 --gateway 10.0.0.1 br0
```

<p align="center">
<img src="./img/bridgenat.png" width=100%>
</p>

#####Bridge Mapped to Underlay

In the case that a container needs to have an ip address on the subnet of the
physical/underlay network then this option can be used.

```
host A
$ docker network create -d bridge --subnet 192.168.1.0/24 --ip-range 192.168.1.0/28
--gateway 192.168.1.100 --aux-address DefaultGatewayIPv4=192.168.1.1 -o
com.docker.network.bridge.name=brnet brnet

host B
$ docker network create -d bridge --subnet 192.168.1.0/24 --ip-range
192.168.1.32/28 --gateway 192.168.1.101 --aux-address
DefaultGatewayIPv4=192.168.1.1 -o com.docker.network.bridge.name=brnet brnet

```

<p align="center">
<img src="./img/bridgetounderlay.png" width=100%>
</p>

#####Default Overlay Mode (Overlay + NAT/Port Mapping)

The default mode of the overlay network is to use the overlay network for container
to container communication and `gw_bridge` NAT/Port-Mapping for external
communication.

<p align="center">
<img src="./img/overlaydef.png" width=100%>
</p>

#####MACVLAN

##Docker Network Troubleshooting and Tools

#####docker network
#####ip route/address
#####brctl
#####nsenter

---------------
####My Questions
- Possible to connect container to two networks in swarm mode?
- How does DNS work when a container is connectected to two different networks?
- Possible to connect both bridge and overlay network to container at same time?

####Questions to Ask your Application

- What segmentation and access is required of the applications?
- How granular does the segmentation need to be?
- What are the different tiers or environments that exist?

####Management Network
- What Docker engine/swarm/ucp/dtr traffic constitutes as management traffic?
- What are the network policies for docker management traffic?

####Best Practices for IP Address Management

- How are addresses reserved and released?
- How does the IPAM driver work?
- What 3rd party IPAM drivers exist?
- How to align host subnet layout with swarm labels/host environments?
####Network Security
- Should the underlay network be subnetted to provide application isolation?
Pros/cons
- Should the docker cluster be one flat IP space with use of overlay networks to
provide segmentation? Pros/cons
- Map container IPs to underlay? Pros/cons
- Port forwarding/NAt to reach underlay? Pros/cons
- Methods to connect separate interal overlay networks (different app tiers)
- attach multiple overlay networks to intermediary containers
- send traffic outside of overlay to gateway router

####Swarm/UCP Network Architecture

- Can a UCP or swarm instance stretch across multiple AZ/data centers?
- Ports and network access required for UCP and swarm infrastructure?

Notes
No more than 1 bridge when plumbing directly to underlay
Discuss docker events -> service discovery
Mention port mapping with port IP, not insecure because port is only expose on that
IP

Networking Challenges

|||||||||||||||||||||||||||||||||||||||||||||||||||||||

cat tutorials.md

### <a name="pets"></a>Tutorial Application: The Pets App

In the following example, we will use a fictional app called

**[Pets](https://github.com/mark-church/pets)** to illustrate the __Network
Deployment Models__. It serves up images of pets on a web page while counting the
number of hits to the page in a backend database. It is configurable via two
environment variables, `DB` and `ROLE`.

- `DB` specifies the hostname:port or IP:port of the `db` container for the web
front end to use.
- `ROLE` specifies the "tenant" of the application and whether it serves pictures
of dogs or cat.

It consists of `web`, a Python flask container, and `db`, a redis container. Its
architecture and required network policy is described below.


We will run this application on different network deployment models to show how we
can instantiate connectivity and network policy. Each deployment model exhibits
different characteristics that may be advantageous to your application and
environment.

We will explore the following network deployment models in this section:

- Bridge Driver
- Overlay Driver
- MACVLAN Bridge Mode Driver

### <a name="bridgemodel"></a>Tutorial App: Bridge Driver

This model is the default behavior of the built-in Docker `bridge` network driver.
The `bridge` driver creates a private network internal to the host and provides an
external port mapping on a host interface for external connectivity.

```bash
#Create a user-defined bridge network for our application
$ docker network create -d bridge catnet

#Instantiate the backend DB on the catnet network

$ docker run -d --net catnet --name cat-db redis

#Instantiate the web frontend on the catnet network and configure it to connect to
a container named `cat-db`
$ docker run -d --net catnet -p 8000:5000 -e 'DB=cat-db' -e 'ROLE=cat' chrch/web
```

> When an IP address is not specified, port mapping will be exposed on all
interfaces of a host. In this case the container's application is exposed on
`0.0.0.0:8000`. We can specify a specific IP address to advertise on only a single
IP interface with the flag `-p IP:host_port:container_port`. More options to expose
ports can be found in the [Docker
docs](https://docs.docker.com/engine/reference/run/#/expose-incoming-ports).


The `web` container takes some environment variables to determine which backend it
needs to connect to. Above we supply it with `cat-db` which is the name of our
`redis` service. The Docker Engine's built-in DNS will resolve a container's name
to its location in any user-defined network. Thus, on a network, a container or
service can always be referenced by its name.

With the above commands we have deployed our application on a single host. The
Docker bridge network provides connectivity and name resolution amongst the
containers on the same bridge while exposing our frontend container externally.

```
$ docker network inspect catnet
[
{
"Name": "catnet",
"Id": "81e45d3e3bf4f989abe87c42c8db63273f9bf30c1f5a593bae4667d5f0e33145",
"Scope": "local",
"Driver": "bridge",
"EnableIPv6": false,
"IPAM": {
"Driver": "default",
"Options": {},
"Config": [
 {
"Subnet": "172.19.0.0/16",
"Gateway": "172.19.0.1/16"
}
]
},
"Internal": false,
"Attachable": false,
"Containers": {
"2a23faa18fb33b5d07eb4e0affb5da36449a78eeb196c944a5af3aaffe1ada37": {
"Name": "backstabbing_pike",
"EndpointID":
"9039dae3218c47739ae327a30c9d9b219159deb1c0a6274c6d994d37baf2f7e3",
"MacAddress": "02:42:ac:13:00:03",
"IPv4Address": "172.19.0.3/16",
"IPv6Address": ""
},
"dbf7f59187801e1bcd2b0a7d4731ca5f0a95236dbc4b4157af01697f295d4528": {
"Name": "cat-db",
"EndpointID":
"7f7c51a0468acd849fd575adeadbcb5310c5987195555620d60ee3ffad37c680",
"MacAddress": "02:42:ac:13:00:02",
"IPv4Address": "172.19.0.2/16",
"IPv6Address": ""
}
},
"Options": {},
"Labels": {}
}
]
```
In this output, we can see that our two containers have automatically been given ip
addresses from the `172.19.0.0/16` subnet. This is the subnet of the local `catnet`
bridge, and it will provide all connected containers a subnet from this range
unless they are statically configured.

### Tutorial App: Multi-Host Bridge Driver Deployment

Deploying a multi-host application requires some additional configuration so that

distributed components can connect with each other. In the following example we
explicitly tell the `web` container the location of `redis` with the environment
variable `DB=hostB:8001`. Another change is that we are port mapping port `6379`
inside the`redis` container to port `8001` on the `hostB`. Without the port
mapping, `redis` would only be accessible on its connected networks (the default
`bridge` in this case).

```
host-A $ docker run -d -p 8000:5000 -e 'DB=host-B:8001' -e 'ROLE=cat' --name cat-
web chrch/web
host-B $ docker run -d -p 8001:6379 redis
```


> In this example we don't specify a network to use, so the default Docker `bridge`
network will exist on every host.

When we configure the location of `redis` at `host-B:8001`, we are creating a form

of **service discovery**. We are configuring one service to be able to discover
another service. In the single host example, this was done automatically because
Docker Engine provided built-in DNS resolution for the container names. In this
multi-host example we are doing this manually.

- `cat-web` makes a request to the `redis` service at `host-B:8001`

- On `host-A` the `host-B` hostname is resolved to `host-B`'s IP address by the
infrastructure's DNS
- The request from `cat-web` is masqueraded to use the `host-A` IP address.
- Traffic is routed or bridged by the external network to `host-B` where port
`8001` is exposed.
- Traffic to port `8001` is NATed and routed on `host-B` to port `6379` on the
`cat-db` container.

The hardcoding of application location is not typically recommended. Service

discovery tools exist that provide these mappings dynamically as containers are
created and destroyed in acluster. The `overlay` driver provides global service
discovery across a cluster. External tools such as [Consul](https://www.consul.io/)
and [etcd](https://coreos.com/etcd/) also provide service discovery as an external
service.

In the overlay driver example we will see that multi-host service discovery is
provided out of the box, which is a major advantage of the overlay deployment
model.

#### Bridge Driver Benefits and Use-Cases

- Very simple architecture promotes easy understanding and troubleshooting

- Widely deployed in current production environments
- Simple to deploy in any environment from developer laptops to production data
center

### <a name="overlaymodel"></a>Tutorial App: Overlay Driver

This model utilizes the built-in `overlay` driver to provide multi-host

connectivity out of the box. The default settings of the overlay driver will
provide external connectivity to the outside world as well as internal connectivity
and service discovery within a container application. The [Overlay Driver
Architecture](#overlayarch) section reviews the internals ofthe Overlay driver
which you should review before reading this section.

In this example we are re-using the previous Pets application. Prior to this
example we already set up a Docker Swarm. For instructions on how to set up a Swarm
read the [Docker docs](https://docs.docker.com/engine/swarm/swarm-tutorial/create-
swarm/). When the Swarm is set up, we can use the `docker service create` command
to create containers and networks that will be managed by the Swarm.

The following shows how to inspect your Swarm, create an overlay network, and then
provision some services on that overlay network. All of these commands are run on a
UCP/swarm controller node.

```bash
#Display the nodes participating in this swarm cluster
$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS
a8dwuh6gy5898z3yeuvxaetjo host-B Ready Active
elgt0bfuikjrntv3c33hr0752 * host-A Ready Active Leader

#Create the dognet overlay network

$ docker network create -d overlay --subnet 10.1.0.0/24 --gateway 10.1.0.1 dognet

#Create the backend service and place it on the dognet network

$ docker service create --network dognet --name dog-db redis

#Create the frontend service and expose it on port 8000 externally

$ docker service create --network dognet -p 8000:5000 -e 'DB=dog-db' -e 'ROLE=dog'
--name dog-web chrch/web
```


We pass in `DB=dog-db` as an environment variable to the web container. The overlay

driver will resolve the service name `dog-db` and load balance it to containers in
that service. Itis not required to expose the `redis` container on an external port
because the overlay network will resolve and provide connectivity within the
network.

> Inside overlay and bridge networks, all TCP and UDP ports to containers are open
and accessible to all other containers attached to the overlay network.

The `dog-web` service is exposed on port `8000`, but in this case the __routing
mesh__ will expose port `8000` on every host in the Swarm. We can test to see if
the application is working by going to `<host-A>:8000` or `<host-B>:8000` in the
browser.

Complex network policies can easily be achieved with overlay networks. In the
following configuration, we add the `cat` tenant to our existing application. This
will represent two applications using the same cluster but requirE network micro-
segmentation. We add a second overlay network with a second pair of `web` and
`redis` containers. We also add an `admin` container that needs to have access to
_both_ tenants.

To accomplish this policy we create a second overlay network, `catnet`, and attach
the new containers to it. We also create the `admin` service and attach it to both
networks.

```
$ docker network create -d overlay --subnet 10.2.0.0/24 --gateway 10.2.0.1 catnet
$ docker service create --network catnet --name cat-db redis
$ docker service create --network catnet -p 9000:5000 -e 'DB=cat-db' -e 'ROLE=cat'
--name cat-web chrch/web
$ docker service create --network dognet --network catnet -p 7000:5000 -e 'DB1=dog-
db' -e 'DB2=cat-db' --name admin chrch/admin
```

This example uses the following logical topology:

- `dog-web` and `dog-db` can communicate with each other, but not with the `cat`
service.
- `cat-web` and `cat-db` can communicate with each other, but not with the `dog`
service.
- `admin` is connected to both networks and has reachability to all containers.


#### Overlay Benefits and Use Cases

- Very simple multi-host connectivity for small and large deployments

- Provides service discovery and load balancing with no extra configuration or
components
- Useful for east-west micro-segmentation via encrypted overlays
- Routing mesh can be used to advertise a service across an entire cluster

### <a name="macvlanmodel"></a>Tutorial App: MACVLAN Bridge Mode

There may be cases where the application or network environment requires containers
to have routable IP addresses that are a part of the underlay subnets. The MACVLAN
driver provides an implementation that makes this possible. As described in the
[MACVLAN Architecture section](#macvlan), a MACVLAN network binds itself to a host
interface. This can be a physical interface, a logical sub-interface, or a bonded
logical interface. It acts as a virtual switch and provides communication between
containers on the same MACVLAN network. Each container receives a unique MAC
address and an IP address of the physical network that the node is attached to.


In this example, the Pets application is deployed on to `host-A` and `host-B`.

```bash
#Creation of local macvlan network on both hosts
host-A $ docker network create -d macvlan --subnet 192.168.0.0/24 --gateway
192.168.0.1 -o parent=eth0 macvlan
host-B $ docker network create -d macvlan --subnet 192.168.0.0/24 --gateway
192.168.0.1 -o parent=eth0 macvlan

#Creation of web container on host-A

host-A $ docker run -it --net macvlan --ip 192.168.0.4 -e 'DB=dog-db' -e 'ROLE=dog'
--name dog-web chrch/web

#Creation of db container on host-B

host-B $ docker run -it --net macvlan --ip 192.168.0.5 --name dog-db redis
```

When `dog-web` communicates with `dog-db`, the physical network will route or
switch the packet using the source and destination addresses of the containers.
This can simplify networkvisibility as the packet headers can be linked directly to
specific containers. At the same time application portability is decreased as
container IPAM is tied to the physical network. Container addressing must adhere to
the physical location of container placement in addition to preventing overlapping
address assignment. Because of this, care must be taken to manage IPAM externally
to a MACVLAN network. Overlapping IP addressing or incorrect subnets can lead to
loss of container connectivity.

#### MACVLAN Benefits and Use Cases

- Very low latency applications can benefit from the `macvlan` driver because it
does not utilize NAT.
- MACVLAN can provide an IP per container, which may be a requirement in some
environments.
- More careful consideration for IPAM must be taken in to account.

## Conclusion

Docker is a quickly evolving technology, and the networking options are growing to
satisfy more and more use cases every day. Incumbent networking vendors, pure-play
SDN vendors, and Docker itself are all contributors to this space. Tighter
integration with the physical network, network monitoring, and encryption are all
areas of much interest and innovation.

This document detailed some but not all of the possible deployments and CNM network
drivers that exist. While there are many individual drivers and even more ways to
configure those drivers, we hope you can see that there are only a few common
models routinely deployed. Understanding the tradeoffs with each model is key to
long term success.