Sriov Vs DPDK
Sriov Vs DPDK
Everything is cloud Native from 5G core to RAN, transport and orchestration. Either you
know about it or Nothing about Cloud. In this "FREE" one page infographic poster, I
have made it a QUICK and EASY reference for Cloud Native main concepts which are
otherwise very complex to understand. Plus get notified when important blogs are
published.
Grab it Now !
It is not easy to settle the debate for DPDK vs SR-IOV-the technologies used to optimize
162
packet processing in NFV servers.
For one, you will find supporters on both sides with their claims and arguments.
However although both are used to increase the packet processing performance in servers,
the decision on which one is better comes down to design rather than the technologies
themselves.
So a wrong decision on DPDK vs SR-IOV can really impact the throughput performance as
you will see towards the conclusion of the article.
1. What is DPDK
2. What is SR-IOV
3. How DPDK is different than SR-IOV
4. What are the right use cases for both and how to position them properly?
5. How DPDK/SR-IOV affects throughput performance.
I recommend that you start from the beginning until the end in order to understand the
conclusion in a better way.
What is DPDK?
DPDK stands for Data Plane Development Kit.
In order to understand DPDK , we should know how Linux handles the networking part
By default Linux uses kernel to process packets, this puts pressure on kernel to process
packets faster as the NICs (Network Interface Card) speeds are increasing at fast.
There have been many techniques to bypass kernel to achieve packet efficiency. This
involves processing packets in the userspace instead of kernel space. DPDK is one such
technology.
Kernel space is where the kernel (i.e., the core of the operating system) runs and provides
its services. It sets things up so separate user processes see and manipulate only their
own memory space.
User space is that portion of system memory in which user processes run . Kernel space
can be accessed by user processes only through the use of system calls.
However, the introduction of DPDK (Data Plane Developer Kit), changes the landscape, as
the application can talk directly to the NIC completely bypassing the Linux kernel.
Without DPDK, packet processing is through the kernel network stack which is interrupt-
driven. Each time NIC receives incoming packets, there is a kernel interrupt to process the
packets and a context switch from kernel space to user space. This creates delay.
With the DPDK, there is no need for interrupts, as the processing happens in user space
using Poll mode drivers. These poll mode drivers can poll data directly from NIC, thus
provide fast switching by completely bypassing kernel space. This improves the throughput
rate of data.
Open vSwitch is a production quality, multilayer virtual switch licensed under the open
source Apache 2.0 license. This runs as software in hypervisor and enables virtual
networking of Virtual Machines.
Main components162include:
In fact, OVS was never designed to work in the telco workloads of NFV. The traditional web
applications are not throughput intensive and OVS can get away with it.
Now let’s try to dig deeper into how OVS processes traffic.
OVS, no matter how good it is, faces the same problem as the Linux networking stack
discussed earlier. The forwarding plane of OVS is part of the kernel as shown below,
therefore a potential bottleneck as the throughput speed increases.
Open vSwitch can be combined with DPDK for better performance, resulting in a DPDK-
accelerated OVS (OVS+DPDK). The goal is to replace the standard OVS kernel forwarding
path with a DPDK-based forwarding path, creating a user-space vSwitch on the host, which
uses DPDK internally for its packet forwarding. This increases the performance of OVS
switch as it is entirely running in user space as shown below.
162
DPDK ( OVS + VNF)
It is also possible to run DPDK in VNF instead of OVS. Here the application is taking
advantage of DPDK, instead of standard Linux networking stack as described in the first
section.
While this implementation can be combined with DPDK in OVS but this is another level of
optimization. However, both are not dependent on one another and one can be
implemented without the other.
162
SR-IOV
SR-IOV stands for “Single Root I/O Virtualization”. This takes the performance of the
compute hardware to the next level.
The trick here is to avoid hypervisor altogether and have VNF access NIC directly, thus
enabling almost line throughput.
But to understand this concept properly, let’s introduce an intermediate step, where
hypervisor pass- through is possible even without using SR-IOV.
This is called PCI pass through. It is possible to present a complete NIC to the guest OS
without using a hypervisor. The VM thinks that it is directly connected to NIC. As shown
here there are two NIC cards and two of the VNFs, each has exclusive access to one of the
NIC cards.
However the downside: As the two NICs below are occupied exclusively by the VNF1 and
VNF3. And there is no third dedicated NIC, the VNF2 below is left without any access.
The SR-IOV specification defines a standardized mechanism to virtualize PCIe devices. This
mechanism can virtualize a single PCIe Ethernet controller to appear as multiple PCIe
devices.
By creating virtual slices of PCIe devices, each virtual slice can be assigned to a single
VM/VNF thereby eliminating the issue that happened because of limited NICs
Multiple Virtual Functions ( VFs) are created on a shared NIC. These virtual slices are created
162
and presented to the VNFs.
(The PF stands for Physical function, This is the physical function that supports SR-IOV)
This can be further coupled with DPDK as part of VNF, thus taking combined advantage of
DPDK and SR-IOV.
There is a good study done by intel on DPDK vs SR-IOV; they found out two different
scenarios where one is better than the other.
162
This is clear from this test report of Intel study as shown below the throughput comparison
It is very simple to understand this: If traffic is routed/switched within the server and not
going to the NIC. There is NO advantage of bringing SR-IOV. Rather SR-IOV can become a
bottle neck ( Traffic path can become long and NIC resources utilized) so better to route the
traffic within the server using DPDK.
The following report from the Intel test report clearly shows that SR-IOV throughput wins in
such case
162
It is also easy to interpret this as the traffic has to pass through the NIC anyway so why
involve DPDK based OVS and create more bottlenecks. SR-IOV is a much better solution
here
I will make it very easy. If traffic is switched within a server ( VNFs are within the server),
DPDK is better. If traffic is switched from one server to another server, SR-IOV performs
better.
It is apparent thus that you should know your design and traffic flow. Making a wrong
decision would definitely impact the performance in terms of low throughput as the graphs
above show.
So let say you have a service chaining application for microservices within one server, DPDK
is the solution for you. On the other hand, if you have a service chaining service, where
applications reside on different servers, SR-IOV should be your selection. But don’t forget
that you can always combine SR-IOV with DPDK in VNF ( not the DPDK in OVS case as
explained above) to further optimize the SR-IOV based design.
162
Subscribe
J i th di i
Join the discussion
{} [+]
Name*
Post Comment
Email*
Website
Aman Kaushik
2 years ago
1
Reply
Saroj Panda
2 years ago
Reply to
Aman Kaushik
Excellent write up. Thanks Faisal.
0
Reply
Faisal Khan
2 years ago
Reply to
Saroj Panda
Thanks Saroj, I am afraid I dont have any public info about deployments
0
Reply
Faisal Khan
2 years ago
Reply to
Aman Kaushik
Thanks Aman ! Glad that you liked it
0
Reply
Christopher Adigun
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Christopher Adigun
Interesting Comment Christopher !, I am not aware of this kind of implementation. Yet this
would be interesting to have to get the best of both worlds
0
Reply
Thomas Monjalon
7 months ago
Reply to
Faisal Khan
You can have both paravirtualization and SR-IOV interfaces in the VNF. You just need
to route to the best interface depending your traffic is East-West or North-South.
Even better: the VNF can be a DPDK application: all types of interfaces are supported
in DPDK, and there are some routing libraries.
0
Reply
Faisal
7 months ago
Reply to
Thomas Monjalon
Author
Thanks Thomas for sharing your thoughts. Agreed.
0
Reply
Askar
2 years ago
It appears to me that still there are a lot of details on NFV true implementation challenges
that need to be discussed. Thank you for opening this topic.
0
Reply
Faisal Khan
2 years ago
Reply to
Askar
Thanks Askar for stopping by to read my blog
0
Reply
Adrie T
2 years ago
I just know that there is an implementation of DPDK in VNF for optimizing the VNF in
accessing the virtual network layer. Is this using the same DPDK kit or there is another DPDK
162
for this purpose?
0
Reply
Faisal Khan
2 years ago
Reply to
Adrie T
Thanks for commenting. Yes, this is the same DPDK used in VNF. But as the article shows
that DPDK can be implemented in OVS as well as VNF…
0
Reply
David Korman
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
David Korman
Thanks David, Glad that you liked it
0
Reply
Harish Shah
2 years ago
Reply to
Faisal Khan
A very good read. Thanks for explaining it so nicely.
0
Reply
Faisal Khan
2 years ago
Reply to
Harish Shah
Thanks Harish, Glad that you liked it
0
Reply
Karun
2 years ago
Reply to
Faisal Khan
Amazing explanation and presentation in simple way to understand easily
1. L3VPN
2. L2Gw
3. VxLAN
5. GRE
0
Reply
162
karun
2 years ago
Reply to
Karun
in addition
SD WAN
and orchestration
0
Reply
Faisal Khan
1 year ago
Reply to
Karun
Thanks Karun for stopping by to read my blog. I will consider your feedback for
future blogs….
0
Reply
Kranthi
2 years ago
Request to provide more scenarios on OVS-DPDK, where you use it in real time.
0
Reply
Faisal Khan
2 years ago
Reply to
Kranthi
Thank you Kranthi for your comments !
1
Reply
Faisal Khan
2 years ago
Reply to
Kranthi
Thanks Kranthi, I am not sure if I got your question correctly..
0
Reply
Priyesh
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Priyesh
Thank you Priyesh for visiting and commenting
0
Reply
javed
2 years ago
162
Thanks, very informative
0
Reply
Hossam Abdelmoniem
2 years ago
0
Reply
Samir Dixit
2 years ago
Reply to
Hossam Abdelmoniem
Detailed and yet simple to understand.
1
Reply
Shubhra Srivastav
2 years ago
Excellent explanation, Faisal. Thank you! Can you comment on the limitations of SW based
DPDK and SRIOV in terms of performance and latency and how using an accelerator (FPGA)
can improve both of those.
0
Reply
Luciano
2 years ago
The best article I’ve ever seen about DPDK and SR-IOV. Thanks !
1
Reply
Faisal Khan
2 years ago
Reply to
Luciano
Hi Luciano, thank you…you made my day 🙂
0
Reply
Kunal
2 years ago
Great Explanation.
0
Reply
Faisal Khan
2 years ago
Reply to
Kunal
Thanks Kunal and for your inputs too
162
0
Reply
Mohit
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Mohit
Thanks Mohit for your feedback
0
Reply
Satyam Kumar
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Satyam Kumar
and you made my day 🙂 …thanks Satyam
1
Reply
Jeff Tantsura
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Jeff Tantsura
Thanks for your feedback Jeff Tantsura
0
Reply
budiharto
2 years ago
thanks for the article, very informative. for real NE in telco, it most likely will need to use SR-
IOV for high throughput reason and surely it will sit on top of multeple hosts/phy servers (for
redundancies and capacity), but it comes with the cost: it not easy to do “live” migration
without interrupting the service and less flexible as it require special mapping to phy NIC
(compared to OVS).
the logical choice will be limit SRIOV for telco “NE” and OVS for the rest (EMS, management
etc.)
162
0
Reply
Faisal Khan
2 years ago
Reply to
budiharto
Agreed Budiharto
0
Reply
Sam Gad
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Sam Gad
Thanks Sam, glad that you liked it
0
Reply
SANDEEP
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
SANDEEP
Thank you so much Sandeep
0
Reply
sorfaraz
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
sorfaraz
Salam Sorfaraz, the behaviour should be same irrespective of the use of SDN or not.
0
Reply
Jayaprakash reddy(Jp)
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Jayaprakash reddy(Jp)
Thanks a lot Jayaprakash ! glad that it was of help to you
0
Reply
Ashwani
2 years ago
This is one of the simplest explanation of such a complex topic that I have seen till now. This
article is like a treasure for me. I am thankful to you that you have published it to educate
others.
0
Reply
Faisal Khan
2 years ago
Reply to
Ashwani
Dear Ashwani, Thanks a lot. This is very encouraging comment for me….please spread the
good and share the article to your circle…
0
Reply
Akmal
2 years ago
Excellent review in a very simply and logical steps. Thank you for the great efforts
0
Reply
Faisal Khan
2 years ago
Reply to
Akmal
Thanks Akmal, Glad that you stopped by and liked this piece.
0
Reply
Arvind Kumar
2 years ago
You have dealt with a complex topic in the best possible simple way. Really amazed by your
capability to explain things in such a simple manner.
0
Reply
Faisal Khan
2 years ago
Reply to
Arvind Kumar
Thanks Arvind for stopping by to read the blog…
0
Reply
162
Eslam Seweilam
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Eslam Seweilam
Thanks Eslam for stopping by to read this piece.
0
Reply
Arvind
2 years ago
Excellent post Faisal. You really simplified it for me. My team is in the process of
implementing SR-IOV vs DPDK for a Telco and this is the kind of information I needed to
understand the concepts better.
I will probably come back in a few months with some practical view point that I can add from
our field experience.
0
Reply
Faisal Khan
2 years ago
Reply to
Arvind
Wow that will be wonderful to get the practical viewpoint
0
Reply
Rakesh Shrivastava
2 years ago
0
Reply
Faisal Khan
2 years ago
Reply to
Rakesh Shrivastava
Great to have you here Rakesh !
0
Reply
duy nguyen
2 years ago
This article very clear and easy to understand. Can I translate and add some more info this
article to Vietnamese before share to my Colleages?
Thanks you!
0
Reply
162
Faisal Khan
2 years ago
Reply to
duy nguyen
thanks duy, answered through mail
0
Reply
Karun
2 years ago
Reply to
Faisal Khan
sir, for me vnf and nfv is quite confusing and hardly can differentiate these two, can
you please add some detail to differential both
0
Reply
Thuan T. Nguyen
1 year ago
This post does open my mind. But it is so good if we can show steps to be practical. To see
and trust.
0
Reply
Faisal Khan
1 year ago
Reply to
Thuan T. Nguyen
Thanks Thuan, can you elaborate a little bit
0
Reply
Neetesh
1 year ago
0
Reply
Faisal Khan
1 year ago
Reply to
Neetesh
Thanks Neetesh
0
Reply
Sandeep
1 year ago
Reply to
Faisal Khan
Really, explained in very easy format.
really helpful
0
Reply
Faisal Khan
1 year ago
Reply to
Sandeep
162
Sandeep, thanks for visiting and liking it.
0
Reply
simiter
1 year ago
good report!!
In “If traffic is North-South, SR-IOV wins against DPDK”, who dicide the route to VNF2?
0
Reply
Faisal Khan
1 year ago
Reply to
simiter
Sorry Simiter, not able to get you !
0
Reply
sandeep sharma
1 year ago
0
Reply
Faisal Khan
1 year ago
Reply to
sandeep sharma
Thanks Sandeep, good that you liked it
0
Reply
Ajay Kumar
1 year ago
Many thanks…
0
Reply
Faisal Khan
1 year ago
Reply to
Ajay Kumar
Thanks Ajay for stopping by to read and liking it.
0
Reply
Zahid
1 year ago
Faisal, indeed a great way and simplified one to explain the complex topics together.
0
Reply
162
Faisal Khan
1 year ago
Reply to
Zahid
Zahid, great to know that you liked it
0
Reply
Jessica
1 year ago
0
Reply
Faisal Khan
1 year ago
Reply to
Jessica
Thanks a lot Jessica
0
Reply
Latha Sunkara
1 year ago
Reply to
Jessica
Really explained very well and any one can understand it. Keep posting more and more
pls. to educate rest of the world. Rarely seen such detailed write-ups.
0
Reply
Faisal Khan
1 year ago
Reply to
Latha Sunkara
Thanks Latha for taking to read and commenting.Glad that you liked it
0
Reply
naga
1 year ago
Excellent Sir !!
please share the DPDK and SR-IOV commands for regular operations and debug ?
0
Reply
Faisal Khan
1 year ago
Reply to
naga
Thanks Naga ! I am not sure, can you clarify further
0
Reply
sunil singh
1 year ago
162
simple and superb, as i am fresher to this technology still
Thanks a Lot!!
0
Reply
Faisal Khan
1 year ago
Reply to
sunil singh
Thank you so much Sunil, you made my day 🙂
0
Reply
Simon Chapman
1 year ago
nicely explained.
0
Reply
Faisal Khan
1 year ago
Reply to
Simon Chapman
Thanks Simon
0
Reply
Anand
1 year ago
Hi,
As you explained in DPDK, kernel is bypassed and userspace polls for the packets. Are you
aware of Enhanced Network Stack of VMware? It is a DPDK based stack in which the kernel
itself does the polling.
Do you think it will be a better solution for telcos? Better than simple DPDK or SRIOV?
0
Reply
Faisal Khan
1 year ago
Reply to
Anand
Hi Anand, thanks for commenting. I am not aware of VMWare stack, I will have to check it
up.
0
Reply
Jatin
1 year ago
Thanks Faisal for making the Complex topic so Simple. Kudos to you.
Thanks.
0 162
Reply
Faisal Khan
1 year ago
Reply to
Jatin
Sounds like a good idea ! Thanks Jatin for suggesting.
0
Reply
Syed Asfar
1 year ago
0
Reply
Faisal Khan
1 year ago
Reply to
Syed Asfar
Thanks Syed asfar, for taking time to read and liking it.
0
Reply
Saif
1 year ago
Thanks, Faisal for this excellent write-up. This was really helpful.
0
Reply
Faisal Khan
1 year ago
Reply to
Saif
Thanks Saif, Glad that you liked it
0
Reply
Ajay
1 year ago
can you please correct the typo “DPDK stands for Data Plan Development Kit”.
it is awesome to see how simplified your contents are even though the topic are complex
.thumbs up !!
0
Reply
Faisal Khan
1 year ago
Reply to
Ajay
Good feedback Ajay, this post guests hundreds of visitors every day, you are the first one
to point it out.
0
Reply
Amarinder162
1 year ago
0
Reply
Faisal Khan
1 year ago
Reply to
Amarinder
Thanks Amarinder
0
Reply
Saurabh Mishra
1 year ago
Reply to
Faisal Khan
Very nice explanation .. Awesome Job Faisal
0
Reply
Faisal Khan
1 year ago
Reply to
Saurabh Mishra
Thanks Saurabh, Glad that you liked it
0
Reply
KELING JI
1 year ago
very very good article , but some pic can not be found , could you please to fix it ? many
thanks
0
Reply
Faisal Khan
1 year ago
Reply to
KELING JI
Thanks, keling, Can you clarify, as everything is visible at my end. which browser you are
using, can you change it.
0
Reply
chandra
1 year ago
Wow…Amazing explanation… right on target…So difficulty subject but you made it so easy…
well done
0
Reply
Faisal Khan
1 year ago
Reply to
chandra
Glad that you liked it Chandra, keep visiting back !
0
Reply
162
Eduardo
1 year ago
I have a little doubt, however. As you explained, with SR-IOV the VM can already talk directly
to the server NIC (using a VF). What does DPDK provide for in a SR-IOV+DPDK scenario?
Cheers!
0
Reply
Faisal Khan
1 year ago
Reply to
Eduardo
Hi Eduardo, thanks for your question. Some applications are customized for DPDK ( does
not necessarily mean they have to use OVS for switching ), they can still be used with SR-
IOV, thus taking advantage of SR-IOV fast switching.
0
Reply
rahul
1 year ago
0
Reply
Faisal Khan
1 year ago
Reply to
rahul
Thanks Rahul, its nice to see that you liked it.
0
Reply
Ranjeet
1 year ago
Awesome for any new learner . Basic to deep explanation. Very simple way feeding full meal.
0
Reply
Faisal Khan
1 year ago
Reply to
Ranjeet
Hello Ranjeet, glad that you liked it.
0
Reply
Ishrat Gul
1 year ago
0
Reply
Faisal
162 Khan
1 year ago
Reply to
Ishrat Gul
Thanks Ishrat, Glad that you liked it.
0
Reply
Ashok
1 year ago
Hi,
I see in the DPDK diagram, OVS is included in your explanation but not in SR-IOV.
Is the OVS required for switching traffic? How do you explain that?
Thanks
0
Reply
Faisal Khan
1 year ago
Reply to
Ashok
you are correct Ashok, not needed in case of SR-IOV, but needed for DPDK for switching
traffic.
0
Reply
Ashok Meti
7 months ago
Reply to
Faisal Khan
Can that be corrected from the diagram now? That adds more clarity.
Thanks
0
Reply
Faisal
7 months ago
Reply to
Ashok Meti
Author
Sorry, Ashok , did not get you. Any issue with the diagram ?
0
Reply
Search …
162
Recent Posts
Categories
Edge Cloud
NFV
SD-WAN
SDN
Transport
162