Deploying (micro)services with Disnix

Deploying (micro)services with Disnix
Sander van der Burg
Nov 14, 2015
Sander van der Burg Deploying (micro)services with Disnix

Microservice architectures
Microservice architectures have become quite popular:

Microservice architectures
There are even books available:

Microservice architectures: What are they?
In short, the microservice architectural style is an
approach to developing a single application as a suite of
small services, each running in its own process and
communicating with lightweight mechanisms, often an
HTTP resource API. These services are built around
business capabilities and independently deployable by
fully automated deployment machinery. There is a bare
minimum of centralized management of these services,
which may be written in diﬀerent programming languages
and use diﬀerent data storage technologies.
– Martin Fowler

Microservice architectures: are they a revolution?

Service oriented architectures
Services are autonomous, platform-independent
entities that can be described, published, discov-
ered, and loosely coupled in novel ways. They per-
form functions that range from answering simple
requests to executing sophisticated business pro-
cesses requiring peer-to-peer relationships among
multiple layers of service consumers and providers.
Any piece of code and any application component
deployed on a system can be reused and trans-
formed into a network-available service.

“A software component is a unit of composition with
contractually speciﬁed interfaces and explicit context
dependencies only. A software component can be
deployed independently and is subject to composition by
third parties.”

Let’s implement an example system!

Let’s implement an example system!
Staﬀtracker
Let’s maintain a collection of staﬀ members and their
rooms
From the room number I can determine the zipcode
From the zipcode I can determine the street and city
Divide the system into processes communicating
through “light weight” network protocols (HTTP,
TCP, ...)

Designing an architecture

Pros
Strong Module Boundaries:
You can build teams around each service
Independent Deployment:
Less likely that a failure crashes the whole system
A small component is typically easier to deploy (e.g.
fewer dependencies)
Technology Diversity:
One service can be implemented in JavaScript,
another in Python etc.
Any storage means possible (e.g. databases)
We can use many operating systems, hardware
architectures etc.
(source: http://martinfowler.com/microservices)

Cons
Distribution:
Network links are slow and subject to failure
Eventual Consistency:
Diﬃcult to guarantee that data remains consistent
Operational Complexity:
...
(source: http://martinfowler.com/microservices)

Operational Complexity
All services can be deployed to one machine, but each service can
also run on a dedicated machine:
Worst case: 8 machines need to be managed

Diverse technology imposes many kinds of deployment procedures:
Deploying a Node.js package is different than deploying a
Python package
Different operating systems, different dependencies, many
variants

How to update the deployment frequently?
How not to break the system while upgrading?
How to minimize downtimes?
How to roll back in case of a failure?

The Nix project

The Nix project
Automated deployment using declarative speciﬁcations with the
following properties:
Generic. Can be used with many programming languages,
component technologies, and operating systems.
Reproducible. (Almost) no impurities – if inputs are the same,
result should be the same regardless of its location
Reliable. Dependency completeness, (almost) atomic
upgrades and rollbacks.
Eﬃcient. Only the required deployment activities are
executed.

The Nix project: Tools
Nix – declarative specification of package build procedures
(including their dependencies)
NixOS – declarative specification of system aspects
Hydra – declarative specification of jobs
NixOps – declarative specification of a network of machines
Deployment activities are implicit – a user writes or adapts a deploy-
ment specification, and the tools will carry out the required activities,
such as building, transfering, activating, and deactivating.

Deploying (micro)services
How to deploy (micro)service(-oriented) systems the “Nix-way”?

Disnix
$ disnix-env -s services.nix -i infrastructure.nix -d distribution.nix

Disnix: Modeling a service build
{stdenv, inetutils}:
let
makeFlags = "PREFIX=$out "+
"inetutils=${inetutils}";
in
stdenv.mkDerivation {
name = "hello-world-client";
src = ../../../services/hello-world-client;
buildPhase = "make ${makeFlags}";
installPhase = "make ${makeFlags} install";
}
Services are speciﬁed in a similar way as “ordinary” Nix packages:
Deﬁne a function taking local (intra-dependencies) as a
parameters
Body describes how to build package from source and its
dependencies

{stdenv, inetutils}:
{hello world server}:
let
makeFlags = "PREFIX=$out "+
"helloWorldHostname=${hello world server.target.hostname} "+
"helloWorldPort=${toString (hello world server.port)} "+
"inetutils=${inetutils}";
in
stdenv.mkDerivation {
name = "hello-world-client";
src = ../../../services/hello-world-client;
buildPhase = "make ${makeFlags}";
installPhase = "make ${makeFlags} install";
}
Disnix expressions also take inter-dependencies into account

Composing services locally
{system, pkgs}:
let
callPackage = pkgs.lib.callPackageWith (pkgs // self);
self = {
hello_world_server = callPackage ../pkgs/hello-world-server { };
hello_world_client = callPackage ../pkgs/hello-world-client { };
};
in
self
As with ordinary Nix packages, we need to compose their intra-
dependencies by providing them as function parameters.

Services model
{system, distribution, invDistribution, pkgs}:
let
# Imports the intra-dependency composition shown previously
customPkgs = import ../top-level/all-packages.nix { inherit system pkgs; };
in rec {
hello_world_server = rec {
name = "hello_world_server";
pkg = customPkgs.hello_world_server { inherit port; };
port = 3000;
type = "wrapper"; # Specifies how to activate and deactivate a service
};
hello_world_client = {
name = "hello_world_client";
# Refers to the intra-dependency closure of the client
pkg = customPkgs.hello_world_client;
dependsOn = { # Inter-dependency composition
inherit hello_world_server;
};
type = "echo";
};
}

Infrastructure model
{
test1 = { # x86 Linux machine (Kubuntu) reachable with SSH
hostname = "192.168.56.101";
system = "i686-linux";
targetProperty = "hostname";
clientInterface = "disnix-ssh-client";
};
test2 = { # x86-64 Linux machine (NixOS) reachable with SOAP/HTTP
hostname = "192.168.56.102";
system = "x86_64-linux";
targetEPR = http://192.168.56.102:8080/DisnixWebService/...;
targetProperty = "targetEPR";
clientInterface = "disnix-soap-client";
};
test3 = { # x86-64 Windows machine (Windows 7) reachable with SSH
hostname = "192.168.56.103";
system = "x86_64-cygwin";
targetProperty = "hostname";
clientInterface = "disnix-ssh-client";
};
}

Distribution model
{infrastructure}:
{
hello_world_server = [ infrastructure.test2 ];
hello_world_client = [ infrastructure.test1 ];
}
Maps services in the services model to target machines in the infras-
tructure model

Demo

Demo
Initial deployment:
Adding a second client (by adding test2):
Moving the server from test2 to test3 and building on targets:
--build-on-targets

Disnix: Communication ﬂow
disnix-env communicates with target machines through a custom
connection client to remotely execute deployment activities:
SSH (the default)
SOAP/HTTP through DisnixWebService
(http://github.com/svanderburg/DisnixWebService)
NixOps through DisnixOS
(http://github.com/svanderburg/disnixos)

Disnix: Communication ﬂow
Two deployment tools are invoked to carry out remote deployment
activities:
Nix (http://nixos.org/nix): package manager responsible
for building, distributing
Dysnomia (http://github.com/svanderburg/dysnomia):
plugin system responsible for activating, deactivating, locking,
snapshotting, restoring

Disnix: Process decomposition of disnix-env
Each deployment activity is implemented by a command-line tool.

Disnix: Process decomposition of disnix-env
Two additional tools are used when building on targets has been
enabled.

How can we deploy our example system?

How can we deploy our example system?
Implementation details
Each database: a MongoDB database instance
Each service: a process with REST API implemented
in Node.js
Web application front-end: A web application
implemented in Node.js
nginx proxy

Infrastructure deployment
Before we can use Disnix, we need machines ﬁrst! With the
following characteristics:
A Disnix service instance for executing deployment steps
remotely
MongoDB DBMS
Machines can be deployed with various kinds of (external)
solutions (including: Disnix itself).

Infrastructure deployment
NixOps can be used to automatically instantiate and deploy a
network of NixOS machines having the required characteristics:
{
test1 = {
{ pkgs, ...}:
{
networking.firewall.enable = false;
# Enable the Disnix service
services.disnix.enable = true;
# Enabling MongoDB also configures Disnix
# to use the Dysnomia plugin that can activate it
services.mongodb.enable = true;
services.mongodb.bind_ip = "0.0.0.0";
}
test2 = ...
test3 = ...
}

Integrating service and infrastructure deployment
Disnix’s service deployment and NixOps’ infrastructure deployment
facilities can be combined with an extension toolset called:
DisnixOS.
Major difference with the basic toolset: DisnixOS uses
networked NixOS configuration files instead of an
infrastructure model

Demo
Deploying a network of NixOS machines with NixOps:
$ nixops create ./network.nix ./network-virtualbox.nix -d test
$ nixops deploy -d test
Deploying services to machines with DisnixOS:
$ export NIXOPS DEPLOYMENT=test
$ disnixos-env -s services.nix
-n network.nix -n network-virtualbox.nix
-d distribution.nix --use-nixops

A real world example: Conference Compass

Conference Compass provides a service to improve the way
people experience events
Most visible part of the service: apps for conference attendees
Each customer basically gets “their own” app.
We have a product-line using a Nix-based build infrastructure,
including Hydra

The app’s contents is customizable with a configurator service
allowing organizers to create and update their content
Apps connect to a configurator to retrieve the data to be
displayed and other configuration settings
Integration with third party information systems is also
possible

Architectural decisions
Each app connects to its own dedicated configurator service:
No single point of failure
More scalable – slow configurator of one app does not affect
another
More flexible – we can easily move configurators around
Multiple version support – we can add new features for a new
app, without breaking old apps
Implemented in Node.js
Each third party integration (called: channel) is a seperate
program:
They may consume a lot of system resources
Implemented in Python, because we find that more appropriate
for data conversion
Processes communicate through REST/HTTP APIs

Deploying services
We deploy four kinds of services with Disnix:
Configurators (Node.js programs)
Channels (Python programs)
Mongo databases (one for each configurator)
nginx proxies (one for each machine)
Responsible for forwarding requests to configurators and
caching expensive requests

Deploying services
A simple deployment scenario:
(Diagram generated with disnix-visualize)

Deploying services
We might want to move services to more powerful machines when
an event goes live (because more users of an app imply that more
system resources are needed, e.g. network bandwidth):

Deploying services
For really big events, we may want to deploy redundant instances
conﬁgurators and channels to handle all the load:

Some statistics
We have been using Disnix + NixOps for deploying our
production environment since January 2015
Between 6-11 Amazon EC2 machines are managed by NixOps
±80 conﬁgurators have been deployed with Disnix including
their inter-dependencies
Translates to ±200 services managed by Disnix
Production environment is frequently updated. Sometimes
three times a day.
Transition phase duration is ±5 minutes in worst case. Most
simple upgrades only bring a few seconds of downtime.

Conclusions
I have explained the relevance of componentized distributed
system architectures (e.g. microservices)
I have explained how to do service deployment with Disnix
I have explained how to integrate infrastructure deployment
and service deployment
I have shown a real-life usage scenario

References
There is much more you can do with Disnix! This presentation
only covers a few basic aspects!
Disnix homepage: http://nixos.org/disnix
TCP proxy example: https:
//github.com/svanderburg/disnix-proxy-example
StaﬀTracker examples:
Java/WebServices/MySQL version: https://github.com/
svanderburg/disnix-stafftracker-java-example
PHP/MySQL version: https://github.com/svanderburg/
disnix-stafftracker-php-example
Node.js/MongoDB version: https://github.com/
svanderburg/disnix-stafftracker-nodejs-example
Disnix should be considered an advanced prototype tool!

References

Questions

Deploying (micro)services with Disnix

More Related Content

What's hot (20)

Similar to Deploying (micro)services with Disnix (20)

More from Sander van der Burg (16)

Recently uploaded (20)

Deploying (micro)services with Disnix