•

Public Review for •

•
•
Rethinking Virtual Network Embedding: Substrate •
•
Support for Path Splitting and Migration •
•
Minlan Yuy, Yung Yiz, Jennifer Rexfordy and Mung Chiang •
•
•
•
The attractions of network virtualisation are clear: a flexible means for supporting networks of diverse nature, and for •
experimenting with network architectures, protocols, and user interactions, over a common physical infrastructure. •
This paper covers new ground in this young and challenging area by focussing attention on strategies the underlying •
•
substrate network can adopt to make the embedding of `virtual network requests’ more flexible, and therefore more •
efficient, seeking “..to make the substrate network more supportive of the VN embedding problem.” At this high level, •
•
an analogy can be made with an approach common in statistics, namely moving from a non-parametric framework, •
where the space of possible behaviours is too large to allow for universally optimal solutions, to a semi-parametric one, •
where broad decisions are made in advance which restrict this range, allowing typical variations to be accomodated •
•
simply by parameter fitting. This is a sensible approach when dealing with overwhelming possibilities and seems well •
suited to the embedding problem. •
•
The reviewers were unanimous in their finding that there was worthwhile novel content in the paper, specifically the •
two specific substrate-network strategies that were discussed in detail: path splitting and path migration. This formed •
the basis of the success of the paper for CCR: the ideas are new and although there are many question left unanswered,
•
•
the paper clearly demonstrated that they are worth considering. Another strength is the fact that several pertinent prob- •
lems are tackled together, namely an `on-line' process of request arrivals and departures, subject to admission control, •
•
and with consideration for both node and link constraints. This set the problem in a somewhat realistic context, albeit •
highly simplified. •
•
The bulk of the paper concerns path splitting in the substrate network. Based on small random networks, examples were •
given illustrating the potential for path splitting to find solutions for virtual network requests which would have other- •
wise been forbidden. This reflects the intuition that a new embedding requesting large chunks of CPU resources at •
•
nodes and/or link bandwidth may be impossible to directly satisfy for the substrate at a given time, yet possible at a •
finer granularity. Essentially, the jug fits more H2O as water than as large chunks of ice. The examples show that •
•
large performance gains are possible, however as the reviewers pointed out, there are significant costs involved in split- •
ting which are not taken into account here. Furthermore, larger networks may effectively increase the severity of con- •
straints, making it difficult to find suitable split paths. Much more work is needed to determine under what circum- •
•
stances splitting wins the cost-benefit battle. •
As a problem class embedding strategies lies on top of, and hence its success ultimately relies upon, adequate solutions •
•
to underlying traffic engineering problems which have defied general solution for a long time. If we still do not know •
how to design a single network marrying the benefits of IP networks as we know them today with controlled quality •
•
of service, even for a set of predefined static traffic classes, then doing so when virtualising arbitrary networks carry-
•
ing arbitrary services is ambitious to say the least. The danger is that we may end up studying and solving some of the •
new problems specific to virtualisation, assuming solutions for underlying performance issues which do not in fact •
•
exist. Of course, this does not mean that the attempt should not be made, and this paper is a worthy contribution to •
that end. •
•
Public review written by •
Darryl Veitch •
University of Melbourne, Australia •
•
•
•
acm sigcomm •
•
•
•

ACM SIGCOMM Computer Communication Review 17 Volume 38, Number 2, April 2008
 Rethinking Virtual Network Embedding:
Substrate Support for Path Splitting and Migration

Minlan Yu, Yung Yi, Jennifer Rexford, Mung Chiang

Princeton University Princeton, NJ
{minlanyu,yyi,jrex,chiangm}@princeton.edu

ABSTRACT offer customized end-to-end services over a common phys-

Network virtualization is a powerful way to run multiple ical infrastructure [14, 27]. For example, Voice over IP
architectures or experiments simultaneously on a shared in- (VoIP) could run on a virtual network that provides pre-
frastructure. However, making efficient use of the underly- dictable performance (by allocating dedicated resources and
ing resources requires effective techniques for virtual network employing routing protocols that ensure fast recovery from
embedding—mapping each virtual network to specific nodes equipment failures), whereas online banking could run on a
and links in the substrate network. Since the general embed- virtual network that provides security guarantees (through
ding problem is computationally intractable, past research self-certifying addresses and secure routing protocols).
restricted the problem space to allow efficient solutions, or Making efficient use of the substrate resources requires
focused on designing heuristic algorithms. In this paper, we effective techniques for virtual network (VN) embedding—
advocate a different approach: rethinking the design of the mapping a new virtual network, with constraints on the vir-
substrate network to enable simpler embedding algorithms tual nodes and links, on to specific physical nodes and links
and more efficient use of resources, without restricting the in the substrate network. However, the VN embedding prob-
problem space. In particular, we simplify virtual link em- lem is extremely challenging, for four main practical reasons:
bedding by: i) allowing the substrate network to split a Node and link constraints. Each VN request has re-
virtual link over multiple substrate paths and ii) employing source constraints, such as processing resources on the nodes
path migration to periodically re-optimize the utilization of and bandwidth resources on the links, that the embedding
the substrate network. We also explore node-mapping al- must satisfy. For example, to run a controlled experiment,
gorithms that are customized to common classes of virtual- a researcher may need 1 GHz of CPU for each virtual node
network topologies. Our simulation experiments show that and 10 Mbps for each virtual link. In addition, the VN
path splitting, path migration, and customized embedding may impose additional constraints on node location or link
algorithms enable a substrate network to satisfy a much propagation delay. For example, a commercial gaming ser-
larger mix of virtual networks. vice may need virtual nodes in several major cities, as well
as virtual links with propagation delays less than 50 msec.
The combination of node and link constraints make the em-
Categories and Subject Descriptors bedding problem computationally difficult to solve.
C.2.5 [Computer-Communication Networks]: Local and Admission control. Since the substrate resources are
Wide-Area Networks; G.1.6 [Numerical Analysis]: Opti- limited, some VN requests must be rejected or postponed to
mization avoid violating the resource guarantees for existing virtual
networks. That is, the substrate must reserve node and link
resources, and perform admission control on new requests to
General Terms ensure that sufficient resources are available. For example, a
Algorithms; Design virtual network that requires 1 GHz of CPU for each virtual
node may be rejected if no physical nodes have enough unal-
located processing capacity. Once accepted, the virtual net-
Keywords works receive their guaranteed resources through scheduling
Virtual Network Embedding; Path Splitting; Path Migra- techniques for sharing the node and link resources.
tion; Network Virtualization; Optimization Online requests. The VN requests are not known in ad-
vance, and may arrive dynamically and stay in the network
for an arbitrary period of time before departing. For exam-
1. INTRODUCTION ple, a researcher may start a new experiment at any time, to
Network virtualization has emerged as a powerful way to run for some duration based on the needs of the experiment.
allow multiple network architectures, each customized to a Similarly, a service provider may deploy a new service at any
particular application or user community, to run on a com- time, and continue supporting the service indefinitely, possi-
mon substrate. For example, virtualization can enable mul- bly discontinuing the service when it is no longer profitable.
tiple researchers to evaluate new network protocols simulta- To be practical, the embedding algorithm must handle VN
neously on a shared experimental facility [3, 7, 2, 10]. In ad- requests as they arrive, rather than handling a large collec-
dition, network virtualization could serve as the foundation tion of requests at once. Online problems are typically much
of a future Internet that allows multiple service providers to

ACM SIGCOMM Computer Communication Review 19 Volume 38, Number 2, April 2008
more difficult to solve, because the embedding algorithm has
little (if any) visibility into the future request arrivals. Table 1: Notations of VN Embedding Problem
Diverse topologies. The virtual networks may have di- Gs Substrate network
verse topologies. For example, researchers may run experi- Ns Nodes of substrate network
ments under a variety of topologies to explore how their pro- Ls Links of substrate network
tocol performs in different settings. Also, a service provider
may tailor the virtual-network topology to the application, AsN Node attribute of substrate network
such as a hub-and-spoke to provide customers with access to AsL Link attribute of substrate network
a centralized server, or a tree to distribute streaming video P s
Paths on substrate network
to a group of receivers. Although virtual networks may have v
a wide variety of structures, certain kinds of topologies— G Virtual network
such as a hub-and-spoke or a tree—may be especially com- Nv Nodes of virtual network
mon in practice. Handling arbitrary topologies, while effi- Lv Links of virtual network
ciently supporting the most common topologies, introduces v
CN Node constraint of substrate network
an additional challenge for the embedding algorithm.
These four properties make the VN embedding problem CLv Link constraint of substrate network
very difficult. In fact, the problem is computationally in- RN Resources allocated for virtual network nodes
tractable, even if some of these four properties are ignored.
RL Resources allocated for virtual network links
Due to the combination of node and link constraints, the VN
embedding problem is NP-hard, even in the offline case. (For VN request 1 Substrate network
example, assigning virtual nodes to the substrate network 10 40 60
without violating bandwidth constraints can be reduced to 20
b
B
15
C
the multiway separator problem, which is NP-hard [6].) Even 10 a
20 15 15
5
if the locations of the virtual nodes are pre-determined, em- c
b
10 20 100
bedding the virtual links with bandwidth constraints is still 70 A 40
D
40
E 80
VN request 2 20
NP-hard, as discussed in more detail in Section 3. The on- 5 5
a 10
10
20 60
line problem is even more difficult to solve. Traditional d 10
e c F 60 G
10
H
10
I 60
techniques for solving online problems (e.g., dynamic pro- Atlanta New Jersey 40 d e
10
gramming) are impractical here because the properties of
incoming VN requests are generally unpredictable and the
search space is prohibitively huge when the substrate net- Figure 1: An example of VN embedding
work is large.
to satisfy the first three of the four challenges listed above.
Previous research has addressed these computational chal-
To address the fourth challenge, we introduce customized
lenges by restricting the problem space in one or more di-
node-embedding algorithms for common topologies like a
mensions to enable efficient heuristics [31, 13, 23, 26], at the
hub-and-spoke.
expense of limiting the practical applicability of the solu-
The remainder of this paper is organized as follows. In
tions. For example, the papers either solve an offline vari-
Section 2, we define the VN embedding problem and present
ant of the problem [31, 23], consider only bandwidth con-
a simple embedding algorithm that does not assume any spe-
straints [13, 23], or do not perform admission control [31,
cial capabilities from the substrate. This algorithm is similar
23, 13]. In this paper, we take a different approach—we
to the techniques proposed in previous research, extended
reconsider the capabilities of the underlying substrate net-
to support admission control and online requests. The algo-
work, to make the substrate network more supportive of the
rithm serves as a basis of comparison for our new algorithm
VN embedding problem. This allows us to create simpler
for substrates that support path splitting and migration, as
embedding algorithms that make more efficient use of the
discussed in Section 3. Section 4 presents simulations that
substrate resources, without compromising on the four chal-
evaluate our algorithm and quantify the benefits of a more
lenges listed above. Since network virtualization is still in its
flexible substrate. In Section 5, we present our customized
infancy, we believe it is important to explore how to design
node-embedding algorithm for hub-and-spoke topologies, and
the substrate to best satisfy its goals.
promising initial simulation results. Section 6 compares our
In particular, we investigate how to simplify the problem
algorithms with related work, and Section 7 concludes the
of virtual-link embedding by allowing the substrate network
paper.
to map a virtual link to multiple substrate paths with a flex-
ible path-splitting ratio. In addition, for efficient handling
of online requests, we allow the substrate to periodically 2. VIRTUAL NETWORK EMBEDDING
re-optimize the mapping of existing virtual links, either by In this section, we first describe the general VN embed-
selecting new underlying paths or adapting the splitting ra- ding problem. Then, we present an original solution to this
tios for the existing paths. Flexible path splitting is realiz- problem without assuming that the substrate can split a
able in practice without disrupting the basic properties of virtual link over multiple underlying paths.
a link—such as predictable bandwidth, propagation delay,
and in-order packet delivery, as discussed in more detail in 2.1 Virtual Network Embedding Problem
Section 3.3. Flexible path splitting allows us to map vir-
Substrate network. We denote the substrate network by
tual links to the substrate in polynomial time, while making
an undirected graph Gs = (N s , Ls , AsN , AsL ), where N s and
much more efficient use of substrate bandwidth and increas-
Ls refer to the set of nodes and links, respectively. We use
ing robustness to substrate failures. This feature allows us
superscript to refer to substrate or virtual network, and use

ACM SIGCOMM Computer Communication Review 20 Volume 38, Number 2, April 2008
subscript to refer to nodes or links, unless otherwise speci- Req. Queue new Reqs. time window
fied. Substrate nodes and links are associated with their at-
tributes, denoted by AsN and AsL , respectively. In this paper, Req leaves
Req comes
we consider CPU capacity and location for node attributes,
and bandwidth capacity for link attributes. We also denote
by P s the set of all loop-free paths in the substrate network.
The right side of Figure 1 shows a substrate network. The
numbers near the links represent available bandwidths and Reqs postponed
Node Mapping
the numbers in rectangles are the available CPU resources
Reqs postponed fixed nodes
at the nodes.
Virtual network request. We denote by an undirected Link Mapping

graph Gv = (N v , Lv , CN v
, CLv ) a virtual network request. A
VN request typically has link and node constraints that are Embedding output for this time window
specified in terms of attributes of the substrate network. We
denote by CLv and CN v
the set of link and node constraints, Figure 2: The baseline algorithm overview
respectively. Figure 1 depicts two VN requests: the VN re-
quest 1 requires the bandwidth 20 over the links (a, b) and a balance between the relative costs of the two classes of
(a, c), and the CPU resource 10 at all nodes, a, b, and c; the resources.
VN request 2 is: “connect two nodes d, e ∈ N v with con- Thus, for a VN request Gv , we define its revenue R(Gv (t))
straints that node d should be in Atlanta (where substrate at any particular time t that the virtual network Gv is run-
nodes D and G are located), and node e should be in New ning as:
Jersey (where substrate nodes E and I are located), with ten X X
units of bandwidth on the virtual link between them.” R(Gv (t)) = bw(lv ) + α CPU(nv ), (2)
lv ∈Lv nv ∈N v
VN embedding. A virtual network embedding for a VN
request is defined as a mapping M from Gv to a subset of where bw(lv ) and CPU(nv ) are the bandwidth and CPU
Gs , such that the constraints in Gv are satisfied, i.e., requirements for the virtual link lv and the virtual node
M : Gv → (N  , P  , RN , RL ), nv , respectively. We note that the bandwidth revenue (i.e.,
the first term in Equation (2)) is not affected by the sub-
where N  ⊂ N s and P  ⊂ P s , and RN and RL are the node strate paths that the virtual links are mapped to, in partic-
and link resources allocated for the VN requests. The VN ular, physical distance or the number of hops of the mapped
network embedding can be naturally decomposed into node paths. This seems to be reasonable, since VN requests only
and link mapping as follows: care about the satisfiability of their constraints in the sub-
Node Mapping: MN : (N v , CNv
) → (N  , RN ), strate network and will not pay for longer distance.
To achieve the goal in Equation (1), it is crucial to embed
Link Mapping: M : (L , CL ) → (P  , RL ).
L v v
incoming VN requests efficiently, such that the substrate
The right side of Figure 1 shows the VN embedding solu- resource is minimally occupied. This is because an inefficient
tions for the two VN requests. For example, the nodes a, b, embedding of a virtual network at time t may restrict the
and c in VN request 1 are mapped to the substrate nodes A, substrate’s ability to accept future requests. Section 4.2
E, and F, and the virtual links (a, b) and (a, c) are mapped shows the relationship of revenue and efficiency.
to the substrate paths (A,D,E) and (A,D,F) with the CPU
and bandwidth constraints all satisfied. A similar mapping 2.2 Baseline VN Embedding Algorithm
occurs for VN request 2. In this subsection, we propose a simple embedding algo-
Objectives. Our main interest is to propose an efficient rithm that does not exploit any special capabilities from
embedding algorithm for the online problem, where VN re- the substrate network. The algorithm is motivated by the
quests arrive and depart over time. From the substrate net- techniques proposed in related work (e.g., [31, 26]) with ex-
work provider’s point of view, a natural objective of an on- tensions to perform admission control and handle online re-
line embedding algorithm would be to maximize the revenue. quests. Figure 2 depicts our algorithm.
We introduce the notion of revenue that corresponds to the Our algorithm collects a group of incoming requests dur-
economic benefit of accepting VN requests. We denote by ing a time window and then tries to allocate substrate re-
R(Gv (t)) the revenue of serving the VN requests at time t. sources to satisfy the constraints required by the requests.
Then, our objective is to maximize the long-term average Some requests may be deferred due to lack of bandwidth or
revenue, given by the following: CPU resources in the substrate network, and returned to
PT v the request queue. The requests in the queue are dropped
t=0 R(G (t)) if they cannot be served within some delay, which, specified
lim . (1)
T →∞ T by a request, corresponds to the time that a request is will-
The revenue can be defined in various ways according to ing to wait. The requests in the request queue are processed
economic models. In this paper, we focus on bandwidth again in the subsequent time windows.
and CPU as the main substrate network resources. Then, We process all VN requests arriving within the time win-
a natural choice of the revenue for a VN request would be dow as well as in the request queue, in decreasing order of
the weighted sum of revenues for bandwidth and CPU, each their revenues. We map virtual nodes onto the substrate for
of which is proportional to the amount of the requested re- all the considered VN requests, and then map the virtual
sources. Similar to the work in [31], we introduce a tun- links for the requests that successfully finish the node map-
able weight α that allows the substrate provider to strike ping stage. An alternative way, which we do not choose,

ACM SIGCOMM Computer Communication Review 21 Volume 38, Number 2, April 2008
Algorithm 1 Greedy Node Mapping Algorithm Algorithm 2 Link Mapping Algorithm
Step 1 Sort the requests according to their revenues. Step 1 Sort the requests that successfully completed the
Step 2 If no requests left, stop. node-mapping stage by their revenues.
Step 3 Take one request with the largest revenue. Step 2 If no requests left, stop.
Step 4 Find the subset S of substrate nodes that sat- Step 3 Take one request with the largest revenue.
isfy restrictions and available CPU capacity (larger Step 4 For each virtual link of the request, we search
than that specified by the request.) If S == ∅, store the k-shortest paths for increasing k, and stop the
this request in the queue, and GOTO Step 2. search if we can find one with enough bandwidth
Step 5 For each virtual node, find the substrate node in capacity.
S with the “maximum available resources” H (de- Step 5 If fail in Step 3 for some virtual link, then defer
fined in Equation (3)), and GOTO Step 2. this request, and store it in the request queue.
Step 6 GOTO Step 2.
is to map the nodes and links of one request first, before
mapping the other requests. In the baseline VN embedding CPU(ns ) is the remaining CPU resource of ns , and bw(ls )
algorithm, both methods produce similar mapping results, is the unoccupied bandwidth resource for the substrate link
but our method is more efficient because of batch processing ls . The definition in (3) is similar to that in [31] with slight
in the node/link mapping stage. difference that the number of virtual links and nodes are
The optimal embedding algorithm is computationally in- used to measure the resources, not the actual amount of
tractable as discussed in Section 1. Our baseline VN em- CPU and bandwidth resources. With this definition, for a
bedding algorithm heuristically tries to achieve the goal in virtual node, we find the substrate node with the maximum
Equation (1) over each time window. Indeed, the algorithm available resources (Step 5).
contributes to instantaneous revenue maximization by giv-
ing higher priority to the requests with more revenue and 2.2.2 Link Mapping Algorithm
accepting as many requests as possible in the node mapping. When the substrate nodes are selected for mapping, we
Additionally, the algorithm tends to make efficient utiliza- map the virtual links to specific substrate links. Finding an
tion of the substrate bandwidth resources by mapping vir- optimal mapping from a virtual link to a single substrate
tual links to shortest paths in the substrate network, leaving path with fixed node mapping reduces to the Unsplittable
more resources for future requests. Flow Problem (UFP), which is NP-hard [21, 22]. Therefore,
we use the k-shortest path algorithm as an approximation
2.2.1 Node Mapping Algorithm approach in order to minimize bandwidth consumption by
We employ a “greedy” node mapping algorithm, since it the virtual network.
is computationally too expensive to employ other strategies, We search the k-shortest paths for increasing values of k,
such as iterative methods [23] and simulated annealing [13, until we find a path which has enough bandwidth to map
20]. The motivation of the greedy algorithm is to map the the corresponding virtual link. Our k-shortest-path link-
virtual nodes to the substrate nodes with the maximum sub- mapping algorithm can be solved in O(M +N log N +k) time
strate resources so as to minimize the use of the resources at in a substrate network with N nodes and M links [12]. Both
the bottleneck nodes/links [31]. This is beneficial to future for computational efficiency and efficient use of substrate
requests which require specific substrate nodes with scarce resources, k should be kept small.
resources.
In our algorithm, we collect all outstanding requests, and 3. PATH SPLITTING AND MIGRATION
then map all the virtual nodes in these requests to the Restricting each virtual link to a single substrate path
substrate nodes. VN requests sometimes impose some re- makes the link-embedding problem computationally intractable,
strictions on their nodes. The examples of node restric- and the resulting embeddings inefficient. In this section, we
tions include geographic location and special functionality first argue that the substrate network should support flex-
at the substrate node. These node restrictions are quite ible splitting of virtual links over multiple substrate paths,
common in practice, e.g., servers near their customers in and present a new link-embedding algorithm that capital-
content-delivery service, programmable routers, and a node izes on the flexibility. Next, we describe how to periodically
with Internet-2 network connectivity. Requests with restric- re-optimize the mapping of existing virtual links to allow
tions reduce the search space for placing the virtual nodes the substrate network to accept more new requests. Finally,
(Step 4). For example, location-specific requests usually we explain how substrate support for path splitting and mi-
limit their virtual nodes to particular geographic regions. gration can be implemented in practice.
Then, we keep track of the available node/link resources of
the substrate network. Note that for a substrate node ns ∈ 3.1 Path Splitting
N s , we do not use CPU(ns ) alone as the metric of available
resource, because we not only want to make sure that there is 3.1.1 Motivation for Flexible Path Splitting
enough CPU capacity available, but also consider bandwidth To motivate substrate support for path splitting, consider
capacity to prepare for the subsequent link mapping stage. the example in Figure 3. Initially the substrate network
Therefore, we define the amount of available resources for a runs a single virtual network with three virtual nodes and
substrate node ns by: two virtual links that each require 20 units of bandwidth.
X The virtual nodes are mapped to physical nodes A, E, and F,
H(ns ) = CPU(ns ) bw(ls ), (3)
and the two virtual links are mapped to the paths (A,D,E)
ls ∈L(ns )
and (A,D,F), as shown in the lower left part of the figure.
where L(ns ) is the set of all adjacent substrate links of ns , Now, suppose a new VN request arrives with a single virtual

ACM SIGCOMM Computer Communication Review 22 Volume 38, Number 2, April 2008
 b New arriving Req. Queue
20 VN Req 2
Without time window
new Reqs.
Existing
a
path splitting
VN Req 1 d 30 Reject
20 e
c
Req leaves
Path splitting Req comes
Accept
15 15
B C B C

15 15 15
5 15 5
b b Reqs postponed
20 20 40 40 Node Mapping
40 40 A D E
A D E
20 20 fixed nodes
a 20 10
a 10
10 10
20 10 20 Reqs postponed
10 10 10 10 Link Mapping Unsplittable
c c F G H I
F G H I
Reqs postponed
Link Mapping Splittable
Substrate network at time t-1 Substrate network at time t
Node Remapping
Figure 3: Illustration of the benefit of path splitting
Path Migration
link that requires 30 units of bandwidth. Unfortunately, no
one path in the substrate network can accommodate the new Embedding output for this time window

request. However, the new VN could be mapped to nodes D

and E, if the substrate could allocate 20 units of bandwidth Figure 4: Algorithm for path splitting and migration
on the path (D, E) and 10 on the path (D, G, H, I, E). That
is, directing two-thirds of the traffic over the (D, E) path respectively, by the node-mapping algorithm in Section 2.2.
and one-third over the (D, G, H, I, E) path would allow the The pair of substrate nodes (ns1 , ns2 ) is a commodity, and
substrate to accept the second request. finding multiple substrate paths for lv is equivalent to find-
Path splitting enables better resource utilization by har- ing flows from source ns1 to destination ns2 in the substrate
nessing the small pieces of available bandwidth, allowing the network with available bandwidth on the substrate links.
substrate to accept more VN requests. In addition, flexible Thus, a group of, say r, virtual links generates a group of
path splitting makes the link-embedding problem compu- r commodities. The algorithm tries to find all the paths for
tationally tractable. A virtual link l with some capacity r commodities based on the following linear constraints:
constraint, say Cl , is mapped into multiple paths in the
substrate network, such that the sum of reserved end-to- X
r
∀ls ∈ Ls , f (ci , ls ) ≤ bw(ls ) (4)
end bandwidth along the multiple paths is equal to Cl . The i=1
division of traffic over the substrate paths is specified as
a splitting ratio, such as a ratio of 2:1 in the example in where f (ci , ls ) is the bandwidth on the substrate link ls
Figure 3. Under flexible splitting over multiple paths, the that we allocate to commodity ci , or its corresponding vir-
link-embedding problem can be reduced to the Multicom- tual link. After generating the r commodities and the lin-
modity Flow Problem (MFP) [5], which can be solved in ear constraints, we solve the resulting multicommodity flow
polynomial time. problem (Step 2).
The benefits of having multiple paths have been estab- Even with flexible path splitting, the MFP problem may
lished in other contexts, such as load balancing and relia- not have a feasible solution because one or more substrate
bility. In fact, even having just two paths can significantly links do not have enough available capacity. The algorithm
reduce the maximum load on a network, compared to solu- revisits the node-mapping decisions for these virtual links
tions that limit the traffic flow to a single path [24, 19]. Hav- (Steps 4, 5, and 6). The failure in the MFP computation
ing multiple paths also enables faster recovery from network implies that one or more substrate links violate the linear
failures. For example, if a link or node fails, the network constraints in Equation (4). Fortunately, the MFP algo-
can quickly switch the affected traffic to other paths simply rithm can easily output the substrate links that violate the
by changing the splitting ratios. In contrast, in a single- constraint, as well as the extent of the violation. The node-
path setting, a failure requires establishing a new end-to- remapping stage focuses its attention on the substrate link
end path, leading to a more severe service disruption. Due with the largest violation P (bottleneck link) , i.e., the ls ∈ Ls
to the computational, performance, and reliability benefits, with the highest value of ri=1 f (ci , ls ) − bw(ls ).
we believe flexible path splitting should be a key feature We randomly choose one virtual link that is originally
in future virtualized network infrastructures, and the rest of mapped to the path including the bottleneck link, and map
this paper will provide the algorithmic and simulation-based one end of this virtual link to another substrate node with
evidence to support this view. maximum remaining resource H (defined in Equation (3)),
in order to avoid occupying this bottleneck link. The node
3.1.2 Link Mapping Algorithm with Path Splitting remapping revisits the node mapping decision for the new
We describe the link mapping algorithm supporting path requests. However, the remapping process does not change
splitting to enable efficient solutions in Algorithm 3. In Step the resource allocation for virtual networks already running
1, we first construct linear constraints for the virtual links. in the substrate. We try this node remapping for a pre-
For simplicity, consider a request with only one link lv with defined number of times Ttry and make sure each time we
the capacity constraint C, where two end nodes of lv are choose a different bottleneck link. If the MFP is still infea-
denoted by nv1 and nv2 . We denote by MN (nv1 ) = ns1 and sible after Ttry trials, we defer the request that requires the
MN (nv2 ) = ns2 the substrate nodes chosen for nv1 and nv2 , most bandwidth on the bottleneck substrate link and return

ACM SIGCOMM Computer Communication Review 23 Volume 38, Number 2, April 2008
 New arriving
Algorithm 3 Link Mapping Algorithm for Requests with b VN Req 2
Without
20
Path Splitting Existing 30 migration
a d e Reject
VN Req 1 20
MFP Computation: c
With
Step 1 For all requests with splittability, construct linear Accept migration
constraint on the commodities for each substrate
15
link. B C B
15
C
Step 2 Solve MFP (Multicommodity Flow Problem). 15 15 15 5 15
5 10
Step 3 If feasible, stop. b d b
20 40 40 10
Node Remapping: A D E A
40
D
40
E e
Step 4 If infeasible, find the “bottleneck” substrate link. a 20 a 20 30
20 20
Step 5 Randomly choose one virtual link that is originally
mapped at the bottleneck link, pick one end of the c F c F
virtual link and map it to another substrate node
with maximum remaining resource H (defined in Substrate network at time t-1 Substrate network at time t
Equation (3)). Then GOTO Step 2 with new linear
constraints. Figure 5: Illustration of the benefit of migration
Step 6 If remapping of virtual nodes for Ttry times does not
produce a feasible solution, eliminate one of the VN chance of accepting future requests. In particular, the ability
requests having the “largest” impact on infeasibil- to migrate virtual links to different substrate paths—while
ity. Then, construct the linear constrains only with keeping the node mapping intact—can further improve the
the remaining requests, and GOTO Step 2. substrate’s ability to accept future requests.
To motivate substrate support for path migration, con-
it to the request queue, and then try to solve the MFP with sider the example in Figure 5. Initially the substrate net-
the remaining requests again. Larger values of Ttry increase works runs a single virtual network with three nodes (mapped
the computational overhead but improve the likelihood of to physical nodes A, E, and F) and two virtual links that
finding a successful embedding. each require 20 units of bandwidth (mapped to the paths
In practice, some virtual networks may have strict require- (A,D,E) and (A,D,F)). Now, suppose a new VN request
ments that preclude path splitting As such, we envision our arrives with a single virtual link that requires 30 units of
algorithm would handle a mix of both kinds of VN requests. bandwidth. Unfortunately, no pair of nodes in the sub-
As illustrated in Figure 4, we first apply Algorithm 2 for strate network can accommodate the new request, even if
requests that do not allow path splitting, before applying path splitting is permitted. For example, in the left side of
Algorithm 3 for requests that allow path splitting. Unfor- Figure 5, nodes D and E have a path (D,E) with 20 units
tunately, node-remapping is difficult to perform for the un- of bandwidth, a path (D,B,C,E) with 5 units of bandwidth,
splittable virtual links, since the embedding algorithm pro- and a path (D, A, B, C, E) with 0 units of bandwidth—
cesses one virtual link at a time. For the collection of virtual not enough to support a virtual link requiring 30 units of
links that fail to find a suitable path, we cannot easily iden- bandwidth. However, migrating some of the traffic for the
tify the most congested substrate link in a computationally first virtual network to a different path would enable the
efficient manner. To maintain computational simplicity, we substrate to accept the new request. In particular, the sub-
do not consider node-remapping for these requests. In the strate could carry half of the traffic for virtual link (a,b) on
evaluation, we quantify the benefits of path splitting, with a new path (A, B, C, E) to free up additional capacity on
and without the node-remapping step. the substrate link (D, E). Then, the second virtual network
can have link (d,e) mapped to substrate path (D,E).
3.2 Path Migration
3.2.2 Migration Algorithm
To deal with the online nature of the VN embedding prob-
lem, we introduce the idea of path migration, i.e., changing In the migration algorithm described in Algorithm 4, we
the route or splitting ratio of a virtual link. This turns out to fix the node mapping of the virtual networks already running
be another advantage of allowing multipath in the substrate on the substrate. We perform path migration by rerunning
network. the link-mapping algorithm with requests that allow path
splitting (Algorithm 3). Path migration is performed by
3.2.1 Motivation for Path Migration either changing the splitting ratios for the existing paths or
selecting new underlying paths.
Since VN requests arrive and depart over time, the sub-
If only adjusting the splitting ratios is allowable rather
strate network can easily drift into an inefficient configura-
tion, where resources are increasingly fragmented, forcing than setting up any new paths, we have to make sure that
the substrate to reject future requests or route new virtual the flows coming from a commodity only traverse the sub-
links over more expensive (longer) paths. Theoretically, one strate paths originally taken in the link mapping stage. Thus,
could try to address these challenges with predictive mod- we add the following linear constraints to the constraints in
Equation (4) (Step 2):
els of future requests, coupled with mathematical techniques
like dynamic programming. However, the arrival and depar- f (ci , ls ) = 0, ∀ls ∈ Ls , ∀ci , ls ∈
/ P s (ci ), (5)
ture of requests is unpredictable and the underlying search
s
space is too large for dynamic programming to be practical. where P (ci ) is the set of original substrate paths the virtual
Instead, we argue the substrate network should be able to link (or corresponding commodity ci ) was mapped to. Then
“rebalance” the mapping of virtual networks to make more we solve the MFP problem again with the new constraints
efficient use of the substrate resources and to maximize the of both Equations (4) and (5). If we are allowed to select

ACM SIGCOMM Computer Communication Review 24 Volume 38, Number 2, April 2008
Algorithm 4 Path Migration Algorithm ply hashing to the packets based on their header bits and
For all the served requests, forward the packets to the corresponding substrate path.
Step 1 Select the request set S whose durations are larger This hash-based scheme is efficient and, in fact, is widely
than a threshold Tdur . used in IP networks to split traffic evenly over equal-cost
Step 2 If only changing splitting ratio is allowed, add linear multipath [15, 8]. For those non-IP packets, the virtual
constraints (Equation (5)), so that each virtual link network would need to tell the substrate which bits in the
is forced to be mapped to the paths it originally header indicate packets in the same flow, so that the hash-
take in the link mapping step. If setting up new ing can be based on those fields. There are also techniques
path is also allowed, skip this step. for more generalized, enhanced multipath routing to realize
Step 3 Rerun the link mapping algorithm with path split- path splitting, see e.g., [18].
ting, and migrate the related paths. Path migration is closely related to path splitting, and is
easily implemented either by selecting new underlying paths
new underlying paths, we rerun the link-mapping algorithm or adapting the splitting ratios for the existing paths. In
with only the constraints in Equation (4). addition, path migration will not cause significant service
Path migration allows us to (periodically) treat the online disruptions for two reasons: (i) we only need a slight change
embedding problem as an offline problem, to capitalize on of flow splitting ratio for the already-existing paths; (ii) we
the efficiency gains that are possible when handling a large can create the new path in advance before moving the traf-
collection of requests together. As such, we expect the ben- fic to avoid service disruption. The substrate router can
efits of path migration to be highest when the time window use consistent hashing to minimize the fraction of flows that
(for grouping requests) is small, and less significant as the must change paths when the splitting ratio changes or new
window grows larger. paths are created [9]. Therefore, path migration should not
In practice, migrating paths introduces overhead to estab- unduly influence the performance experienced by the virtual
lish new paths, switch the traffic onto the new paths, and network.
tear down the old paths. As such, the benefits of path migra- In our current work, we have focused on path migration
tion should be weighed against the overheads. To illustrate while keeping the node-mapping intact, to minimize the dis-
this, we expect that VN requests would be quite diverse in ruption experienced by the virtual networks. However, in
their durations, which corresponds to their running time in ongoing work we plan to explore node migration to provide
the substrate network, ranging from a few months to sev- even greater flexibility in handling new VN requests. We
eral hours. As an example, a content distribution network believe node migration should be feasible for several rea-
like Akamai [1] may run infinitely, whereas an impromptu sons. First, long-running services usually have their own
conference or video game may last for a few hours. The maintenance windows, where they drain traffic off a server
algorithm should not migrate short-lived virtual networks to upgrade the software. These maintenance windows can
that are likely to exit the system soon after the migration be used for node migration. Second, with ample warning
completes. Thus, our algorithm only considers the requests and prior planning, we can minimize the negative effects of
whose durations are larger than some threshold Tdur (Step node migration on an ongoing service. Node migration can
1). Fortunately, migrating long-running virtual networks be done quite quickly in practice, e.g., within a few sec-
should offer ample benefits in practice, since many short- onds [29], and the virtual node can continue running in the
lived virtual networks will come and go while they run. old location until the migration completes.
Virtual-network requests would indicate their likely dura-
tion, or we can infer that a virtual network that has run 4. PERFORMANCE EVALUATION
for a long time is likely to continue running for a long time, In this section, we first describe the performance evalu-
analogous to previous research on migration in the context ation environment, and then present our main evaluation
of job scheduling [17]. results. Our evaluation focuses primarily on quantifying the
benefits of substrate support for flexible path splitting and
3.3 Implementation Issues migration in the VN embedding problem.
Path splitting can be implemented in the substrate net-
work without significant overhead. When the virtual node 4.1 Evaluation Environment
directs a packet over the virtual link, the substrate sends the We implemented a VN embedding simulator (publicly avail-
packet over one of the paths based on the target splitting able at [4]) to evaluate our embedding algorithm and the
ratio. Path splitting may cause out-of-order packet delivery. advantages of path splitting and migration.
Some virtual networks do not care about out-of-order pack- The actual characteristics of substrate and virtual net-
ets; or they can reorder the out-of-order packets by them- works are not well understood since network virtualization
selves, e.g. those applications with only UDP flows. We can is still an open field. Therefore, we use synthetic networks to
also make the virtual networks oblivious to the traffic split- study the trends and quantify the benefits of path splitting
ting by preventing the disruptions of out-of-order packets in and migration.
the substrate, e.g., using hash-based splitting. Substrate network. We use the GT-ITM tool [30] to
Out-of-order delivery is a primary concern for packets in generate a substrate network topology. The GT-ITM tool
the same flow —a group of packets between the same end has been popularly used in research that requires practi-
hosts or part of the same transport-level connection. Hash- cal network topology generation. The substrate network is
based splitting prevents out-of-order delivery by directing all configured to have 100 nodes and around 500 links, a scale
packets from the same flow to the same path. The substrate that corresponds to a medium-sized ISP The CPU resources
router first divides the hash space into weighted partitions at nodes and the link bandwidths at links follow a uniform
that each correspond to one substrate path. Then, we ap- distribution from 0 to 100 units.

ACM SIGCOMM Computer Communication Review 25 Volume 38, Number 2, April 2008
 12 12 12
E[bw] = 50, Baseline Algorithm
Average Revenue Over Time
E[bw] = 50, Path Splitting Baseline Algorithm

Average Revenue Over Time

Average Revenue Over Time

E[bw] = 50, Path Splitting & Migration Path Splitting
Ttry = 0
10 E[bw] = 25 10 Path Splitting & Migration 10 Ttry = 1
Ttry = 2

8 8 8

6 6 6

4 4 4
0 20 40 60 80 100 0 20 40 60 80 100 0 20 40 60 80 100
Splitting Ratio (%) Splitting Ratio (%) Splitting Ratio (%)

Figure 6: Revenue changes with Figure 7: Revenue changes with Figure 8: Influence of Node
RPS(%) (E[CPU]:0, DELAY:3, α:0, RPS(%) (E[BW]:50, E[CPU]:0, DELAY:6, Remapping (E[BW]:50, E[CPU]:0, DE-
Ttry :1) α:0, Ttry :1) LAY:3, α:0)

Virtual network request. In one VN request, the num- at most 3 time windows in the queue (DELAY=3) and node
ber of VN nodes is randomly determined by a uniform distri- remapping in Algorithm 3 is tried just once, i.e., Ttry = 1.
bution between 2 and 10 in Figures 6–12, following a similar In Figure 6, the performance of the baseline algorithm
setup to previous work [31]. We also test larger requests in Section 2 does not depend on RPS(%), since the base-
whose number of are chosen uniformly between 2 and 20 in line algorithm maps each virtual link into a single path in
Figure 11. the substrate network. However, with more requests allow-
Each pair of virtual nodes are randomly connected with ing path splitting, the substrate network resources are effi-
probability 0.5. This means that for a n-node virtual net- ciently utilized at current time window, which enables the
work, we have n(n − 1)/4 links on average. The arrivals of system to accept more requests, leading to an increase in
VN requests are modeled by a Poisson process with mean the average revenue. When all the requests allow path split-
five requests per time window. The duration of the requests ting, our algorithm achieves about 120% revenue increase
follows an exponential distribution with 10 time windows on over the baseline algorithm. Even with half of the requests
average. We run all of our simulations for 500 time windows, permitting path splitting, we still gain about 65% revenue
which corresponds to about 2500 requests on average in one increase.
instance of simulation. Figure 6 also shows that path migration further increases
The parameters and their symbols that we vary in all our the revenue. For example, when RPS(%)=100, our algorithm
simulations are summarized in the following table: with path migration achieves additional 15% revenue in-
crease over the algorithm only with path splitting. This im-
E[CPU] average CPU requirement on a virtual node
plies that path splitting is a dominant factor in the revenue
E[BW] average bandwidth requirement on a virtual link increase, and path migration further builds on path split-
RPS(%) percentage of the requests allowing path splitting ting to adapt to the online VN embedding problem more
DELAY time a request is willing to wait (see Section 2.2) flexibly. More benefits are expected to be obtained by node
α weight constant in revenue function (Equation (2)) migration, at the expense of more service disruption.
Ttry number of rounds in node remapping When the bandwidth requirement is low (E[BW]=25) and
substrate resources are ample, we can accept all the requests
Comparison method. Comparing our algorithm with for both algorithms. Naturally, the revenue remains the
previous work is difficult because these earlier embedding same, whether the requests allow path splitting or not. We
algorithms do not start with the same problem formulation. will show later in Figure 12 that in this case, our algorithm
They do not handle one or more of the first three challenges reduces cost more than the baseline algorithm. Note that
in Section 1 (i.e., combined node and link constraints, ad- in Figure 6, the revenues differ when E = 25 and E =50
mission control, or online requests). Instead, we use the due to its dependence on the amount of required (average)
algorithm in Section 2.2, which embodies many of the key bandwidth in the requests.
ideas from prior work, as a baseline for comparison.
(2) Path splitting still increases revenue when CPU
4.2 Evaluation Results requirements are considered. Figure 9 shows the long-
term revenues with both CPU and bandwidth requirements,
Our evaluation results quantify the benefits of path split- where the average CPU requirement is set to be 25 and other
ting and migration in various environments. We present our parameters are the same as those in Figure 6.
simulation results by summarizing the key observations. We observe a similar increase in revenue from path split-
(1) More requests allowing path splitting leads to larger ting. We achieve more than 100% revenue increase over
revenues, which is further improved by path migra- the baseline algorithm, when RPS(%)=100; and about 50%
tion. Figures 6 shows the long-term average revenue with when RPS(%)=50. However, the benefits from migration are
increasing percentages of the requests permitting path split- less dramatic. This is due to the fact that we only employ
ting for different average link bandwidth requirements. In path migration, which does not offer any benefits when the
these experiments, we remove the influence of CPU in the node CPU resource is the bottleneck. Note that revenue in-
constraint and the revenue (i.e., E[CPU]=0 and α=0). Each crease with the CPU requirement is less than that without
request which cannot be served immediately will wait for the CPU requirement. This is anticipated, because when

ACM SIGCOMM Computer Communication Review 26 Volume 38, Number 2, April 2008
 3 16
Baseline Algorithm Baseline Algorithm
2

Average Revenue Over Time

Average Revenue Over Time

Path Splitting
Average Revenue Over Time
Path Splitting 2.5 14
Path Splitting & Migration Path Splitting & Migration
1.5 2 12

1.5 10
1
1 8
Baseline Algorithm
0.5 Path Splitting
0.5 6
Path Splitting & Migration
0 0 4
0 20 40 60 80 100 0 0.2 0.4 0.6 0.8 1 0 20 40 60 80 100
Splitting Ratio (%) Weight Constant in Revenue (a) Splitting Ratio (%)

Figure 9: Effect of CPU require- Figure 10: Effect of α (RPS(%):100, Figure 11: Effect of larger virtual
ment (E[BW]:50, E[CPU]:25, DELAY:3, E[BW]:50, E[CPU]:25, DELAY:3, Ttry :1) networks (nodes:2-20, E[BW]:25,
α:0, Ttry :1) E[CPU]:0, α:0, Ttry :1)

CPU requirements tends to reduce the number of accepted without change of total resource requirement, larger requests
requests. lead to more links and thus less bandwidth per link. In other
To further evaluate effects of CPU resource constraints, experiments when we increase the scale of requests without
in Figure 10, we have tested different weight constants α in changing the average bandwidth per link, then we are able
the revenue definition (Equation (2)) while keeping all the to achieve more benefits. We omit this result due to space
other parameters the same. The benefits of path splitting limitation.
over the baseline algorithm decrease as α increases, since
path splitting and migration only improve the bandwidth
resource utilization in the link mapping stage. For example,
when CPU and bandwidth are almost equally evaluated in 17
Average Bandwidth Cost Over Time
the revenue function, we achieve more than 100% of revenue Baseline Algorithm
Path Splitting
increase over the baseline algorithm; when CPU resource 16 Path Splitting & Migration
becomes the main factor (α=1), our algorithm with path
splitting still achieves around 60% more revenue than the 15
baseline algorithm.
(3) Node remapping contributes modestly to revenue 14
increase. Revenue is not significantly influenced by
the delay we choose. All previous experiments were made 13
with Ttry = 1, i.e., we ran one round of node remapping in
Algorithm 3. In Figure 8, we show the result of the path
12
splitting algorithm without node remapping (i.e., Ttry = 0 20 40 60 80 100
Splitting Ratio (%)
0), where the revenue only decreases by 4%. This implies
that the revenue increase shown in earlier simulations mainly
comes from path splitting itself. With increasing values of Figure 12: Effect on cost (E[BW]:25, E[CPU]:0, DELAY:3,
Ttry , we could achieve more substantial increases in revenue α:0, Ttry :1)
at the expense of computation time, because we must rerun
the link mapping stage for Ttry times more than the path
splitting solution without node remapping.
Figure 7 shows that our benefits of path splitting and mi- (5) Without admission control, path splitting and mi-
gration are not influenced by delay we choose (DELAY=3), gration reduces cost. In case when the substrate network
since the result for DELAY=6 is similar to Figure 6 where DE- resources are sufficient, but the number of incoming VN re-
LAY=3. This is because the substrate resources are almost quests per time-window is small, we can probably service all
fully used with requests coming and departing over time, the requests, irrespective of using of path splitting or not,
so that the deferred requests cannot be accepted even if it i.e., the long-term revenue achieved will be the same for the
waits for more time. baseline algorithm and the algorithm with path splitting.
However, the algorithm allowing path splittability saves sub-
(4) Path splitting and migration can still help improve
strate network resources.
revenue for larger requests. Figure 11 examines the ben-
First, we discuss notions of cost to quantify efficiency in
efits of path splitting and migration for larger requests, i.e.,
resource utilization. The bandwidth cost for a VN request
the number of nodes in each VN request uniformly ranges
should be defined to reflect the entire amount of bandwidth
between 2 and 20. We first keep the total amount of re-
used to map the request to the physical substrate network.
source requirement the same as that in the earlier experi-
For a single virtual link in the request, it would be natural
ments. Figure 11 shows that with all the requests allowing
to use its required bandwidth multiplied by the length of
path splitting, the revenue increases by 50% with path split-
the substrate path that the virtual link is mapped to. With
ting. This benefit is less than the 120% increase of revenue
path splitting, we count the bandwidth allocated on each
in Figure 6 with E[BW] = 50. This is due to the fact that
path of the virtual link and sum them up. Thus, we define

ACM SIGCOMM Computer Communication Review 27 Volume 38, Number 2, April 2008
Algorithm 5 Customized Node Mapping Algorithm for Re- nodes (Step 4.1), and the spoke nodes are mapped into
quests with Hub-and-spoke the substrate nodes that have the shortest path to the sub-
Steps 1, 2, and 3: Same as in Greedy Node Mapping strate node hosting the virtual hub node (Step 4.2). This
(Algorithm 1). is motivated by the fact that the hub node handles much
Step 4 If the request has hub-and-spoke topology, more traffic than the individual spokes. In Step 4.2, we
4.1 For each hub node, find the substrate node also achieve significant cost reduction, since cost is generally
with the maximum available resource in S. proportional to the distance (i.e., number of hops), whereas
4.2 For each spoke node, find the shortest path be- the greedy algorithm allocates large substrate resource to
tween a substrate node in S and the substrate “unimportant nodes” (i.e., the spokes). The wasted re-
node mapped to the corresponding hub node. sources keep the greedy node mapping algorithm from leav-
else, apply Step 4 in Greedy Node Mapping. ing enough resources available to satisfy the future requests.
Step 5 Same as in Greedy Node Mapping. We did the evaluation on our customized node mapping al-
gorithm with the requests of hub-and-spoke topologies and
the cost of virtual network Gv by: compare it with the greedy node mapping. Our prelimi-
X X nary experiments show that that our customized algorithm
Cbw (Gv ) = hops(p)bw(p, lv ), (6) performs better than the greedy algorithm when the per-
lv ∈Lv p∈P s (lv ) centage of hub-and-spoke topologies among all the requests
increases. By taking advantage of the topology information,
where P s (lv ) is the path(s) the virtual link lv is mapped
our customized algorithm allocates the hub-and-spoke re-
on, hops(p) is the number of hops of path p, and bw(p, lv )
quest more efficiently than the greedy algorithm. Thus, our
is the amount of bandwidth allocated to that virtual link.
algorithm can allow more requests and achieve a higher av-
Similarly, we define the CPU cost of virtual network Gv by:
erage revenue over time. This result is omitted due to page
X
CCPU (Gv ) = CPU(nv ), (7) limit. We are currently exploring and evaluating other algo-
nv ∈N v
rithms that are customized for different common topologies,
like trees.
where CPU(n ) is the amount of CPU virtual node nv re-
v

quires. 6. RELATED WORK

In Figure 12, we have simulated the case where the re-
source requirement of requests is low (E[BW]=25, E[CPU]=0), Previous research has explored how to embed Virtual Pri-
where other parameters are the same as before, i.e., (DE- vate Networks (VPNs) in a shared provider topology [11, 16].
LAY=3, α=1, Ttry =1). VPNs usually has a standard topology, such as full mesh and
We observe that with the increase in percentage of re- hub-and-spoke [25]. The resource constraints in a VPN are
quests permitting path splitting, we reduce the bandwidth typically just bandwidth requirements, specified by a traffic
cost Cbw over the baseline algorithm by making more ef- matrix (i.e., the traffic volume for each pair of nodes), rather
ficient use of the network. When RPS(%)=100, we reduce than node constraints (e.g., processing resources). VN em-
10% cost than the baseline algorithm. Path migration fur- bedding problem is different with VPN design problem in
ther reduces the bandwidth cost by 7%. The CPU cost that VN embedding problem must deal with both node and
CCPU remains the same (CCPU = 3.1) with the increase of link constraints for arbitrary topology.
RPS(%), since all the requests are accepted, whether they Related work on VN embedding addresses the hardness of
allow path splitting or not. the problem by relaxing one or more of the key properties of
the problem. These key properties include (i) whether re-
quests are processed online or not, (ii) whether the requests
5. CUSTOMIZED NODE MAPPING have link constraints, node constraints, or both, (iii) whether
Although virtual networks may have arbitrary topologies, admission control is performed to reject requests when re-
we expect some classes of topologies to be relatively com- sources are insufficient, and (iv) what virtual topologies are
mon, since they meet the needs of the key applications in supported.
network virtualization. For example, a hub-and-spoke topol- Several of the previous studies focus on the offline prob-
ogy is commonly used to connect multiple sites to a central- lem, where all VN requests are known in advance. Zhu and
ized server, e.g., gaming and CDN (Content Distribution Ammar [31] assume that the substrate network resources are
Network), and a tree topology is commonly used to dis- unlimited, and aim at achieving load balancing in the sub-
tribute content efficiently to a large collection of receivers, strate network without the need for admission control. The
e.g. multicast distribution of IPTV. VN-embedding problem for the requests with general topol-
The popularity of a small set of topological structures can ogy is solved by subdividing the requests into multiple star
be leveraged for better solutions to the VN embedding prob- topologies to allocate more substrate resource to the center
lem. In our ongoing work, we present node-mapping tech- of each decomposed star topology. Our greedy node embed-
niques that are customized to specific topologies, starting ding algorithm (Algorithm 1) is based on the key idea of this
with the simple hub-and-spoke topology. paper. Lu and Turner [23] also consider an offline problem
As an example, we propose a customized node mapping for only a single virtual network with a backbone-star topol-
(which is extended from the Greedy Node Mapping in Al- ogy, where their goal is to minimize the cost. They assume
gorithm 1) with hub-and-spoke topologies, as summarized that only bandwidth constraints are imposed, and the sub-
in Algorithm 5. The customized node mapping algorithm strate network resources are unlimited with no admission
differs from the greedy node mapping in that we choose the control needed.
substrate nodes differently for hub and spokes nodes. The In regard to the online problem, Fan and Ammar [13] con-
maximum available resource is allocated only for the hub sider dynamic topology reconfiguration policies for virtual

ACM SIGCOMM Computer Communication Review 28 Volume 38, Number 2, April 2008
networks with dynamic communication requirements, but [10] A. Bavier, N. Feamster, M. Huang, L. Peterson, and
no consideration of the node constraints such as CPU. They J. Rexford. In VINI veritas: Realistic and controlled
also assume that substrate network resources are unlimited network experimentation. In Proc. ACM SIGCOMM,
September 2006.
to accept all requests (i.e., no admission control) and try to
[11] N. G. Duffield, P. Goyal, A. Greenberg, P. Mishra, K. K.
find a strategy to minimize cost. Zhu and Ammar [31] also Ramakrishnan, and J. E. van der Merwe. Resource
solve the online problem by recalculating the whole embed- management with hoses: Point-to-cloud services for virtual
ding solution periodically. The Assign algorithm [26] used private networks. IEEE/ACM Trans. Networking, 2002.
in Emulab testbed considers the online embedding problem [12] D. Eppstein. Finding the k shortest paths. In Proc. IEEE
with the bandwidth constraint. The node constraint in Em- Symposium on Foundations of Computer Science, 1994.
ulab is provided as the exclusive use of nodes, i.e., different [13] J. Fan and M. Ammar. Dynamic topology configuration in
virtual networks cannot share a substrate node. Admission service overlay networks: A study of reconfiguration
policies. In Proc. IEEE INFOCOM, 2006.
control is not explicitly addressed in [26], but it can be in-
[14] N. Feamster, L. Gao, and J. Rexford. How to lease the
ferred that emulab rejects a request if the substrate band- Internet in your spare time. ACM Computer
width/node resources are insufficient. Emulab’s PlanetLab Communication Review, 37(1):61–64, 2007.
portal [28] provides resource allocation service for users to [15] A. Feldmann, A. Greenberg, C. Lund, N. Reingold, and
access to PlanetLab testbed. It searches for the PlanetLab J. Rexford. NetScope: Traffic engineering for IP networks.
nodes with low CPU and memory loads for the users. IEEE Network Magazine, 14(2):11–19, March 2000.
Our work considers all the four challenges outlined in Sec- [16] A. Gupta, J. Kleinberg, A. Kumar, R. Rastogi, and
tion 1. In particular, we improve the link mapping algorithm B. Yener. Provisioning a virtual private network: A
network design problem for multicommodity flow. In Proc.
through substrate support for path splitting and migration. ACM Symposium on Theory of Computing, 2001.
[17] M. Harchol-Balter and A. B. Downey. Exploiting process
7. CONCLUSION lifetime distributions for dynamic load balancing. ACM
Transactions on Computer Systems, 15(3):253–285, 1997.
A key problem in the current study of network virtual- [18] J. He and J. Rexford. Towards Internet-wide multipath
ization, the VN embedding problem, has various constraints routing. In IEEE Network Magazine Special Issue on
that make it computationally intractable. In this paper, Scalability, March 2008.
rather than significantly restrict the problem space to make [19] P. Key, L. Massoulie, and D. Towsley. Path selection and
the problem tractable, we rethink the VN embedding prob- multipath congestion control. In Proc. IEEE INFOCOM,
lem by proposing a more flexible substrate network to better 2007.
support virtual network embedding. This flexibility includes [20] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi.
Optimization by simulated annealing. Science, Number
path splitting and migration. Path splitting (i.e. multipath)
4598, 13 May 1983, 220, 4598:671–680, 1983.
has been a recurring theme in many network research top- [21] J. Kleinberg. Approximation algorithms for disjoint paths
ics, and we demonstrate the power of multipath in substrate problems. PhD thesis, MIT, 1996.
network for more cost-effective virtual network embedding. [22] S. G. Kolliopoulos and C. Stein. Improved approximation
From both the theoretical and engineering perspective, we algorithms for unsplittable flow problems. In Proc. IEEE
show that allowing substrate path splitting and migration Symposium on Foundations of Computer Science, 1997.
would help us to attain better resource utilization. Through [23] J. Lu and J. Turner. Efficient mapping of virtual networks
our publicly available simulator, we demonstrate the bene- onto a shared substrate. Technical Report
WUCSE-2006-35, Washington University, 2006.
fits of these approaches in making the embedding problem
[24] M. Mitzenmacher. The power of two choices in randomized
computationally easier, and the resulting embeddings more load balancing. IEEE Transactions on Parallel and
efficient. Distributed Systems, 12(10):1094–1104, 2001.
[25] S. Raghunath, K. K. Ramakrishnan, S. Kalyanaraman, and
8. REFERENCES C. Chase. Measurement based characterization and
[1] Akamai content distribution network. provisioning of IP VPNs. In Proc. Internet Measurement
http://www.akamai.com/. Conference, 2004.
[2] GENI. http://www.geni.net/. [26] R. Ricci, C. Alfeld, and J. Lepreau. A solver for the
[3] Planetlab. https://www.planet-lab.org/. network testbed mapping problem. ACM Computer
Communication Review, 33(2):65–81, 2003.
[4] Virtual Network Embedding Simulator.
http://www.cs.princeton.edu/~ minlanyu/embed.tar.gz. [27] J. S. Turner and D. E. Taylor. Diversifying the Internet. In
Proc. IEEE GLOBECOM, 2005.
[5] R. K. Ahuja, T. L. Magnanti, and J. B. Orlin. Network
[28] K. Webb, M. Hibler, R. Ricci, A. Clements, and
Flows: Theory, Algorithms, and Applications. Prentice
Hall, 1993. J. Lepreau. Implementing the Emulab-Planetlab portal:
Experience and lessons learned. In Workshop on Real,
[6] D. G. Andersen. Theoretical approaches to node
Large Distributed Systems (WORLDS), 2004.
assignment. Unpublished Manuscript, http:
//www.cs.cmu.edu/~ dga/papers/andersen-assign.ps, [29] T. Wood, P. Shenoy, A. Venkataramani, and M. Yousif.
2002. Black-box and gray-box strategies for virtual machine
migration. In Proc. Networked Systems Design and
[7] T. Anderson, L. Peterson, S. Shenker, and J. Turner. Implementation, 2007.
Overcoming the Internet impasse through virtualization.
IEEE Computer Magazine, 38(4):34–41, 2005. [30] E. W. Zegura, K. L. Calvert, and S. Bhattacharjee. How to
model an internetwork. In Proc. IEEE INFOCOM, 1996.
[8] B. Augustin, X. Cuvellier, B. Orgogozo, F. Viger,
[31] Y. Zhu and M. Ammar. Algorithms for assigning substrate
T. Friedman, M. Latapy, C. Magnien, and R. Teixeira.
Avoiding traceroute anomalies with Paris traceroute. In network resources to virtual network components. In Proc.
Proc. Internet Measurement Conference, 2006. IEEE INFOCOM, 2006.
[9] I. Avramopoulos, D. Syrivelis, J. Rexford, and S. Lalis.
Secure availability monitoring using stealth probes.
Technical Report TR-769-06, Princeton University, 2006.

ACM SIGCOMM Computer Communication Review 29 Volume 38, Number 2, April 2008