PANEL I
LESSONS LEARNED SO FAR IN OPERATIONALIZING NFV
Panelists
Dave Hughes
VP of Engineering
PCCW Global
Luke Lim
Head of Solutions Engineers, EMEA
SevOne
James Crawshaw
Senior Analyst
Heavy Reading
Ari Banerjee
VP Strategy
Netcracker
Outline
•What progress has been made in operationalizing NFV
and what lessons have been learned?
•What are the greatest challenges in automating
dynamic, virtual networks and how successfully have
these been integrated with legacy physical networks
to provide services that span both domains?
•Has virtualization increased agility, enabled new
business models and reduced opex or simply added to
the complexity of network management and
operations?
Operationalizing SDN and NFV Networks
• While NFV provides an opportunity to reduce opex and improve
customer experience it introduces additional layers of operational
complexity that “put more onus on the operator to integrate
technologies that were traditionally integrated by a vendor” - Irene
Shannon and Jennifer Yates, AT&T, Building the Network of the Future.
• Disaggregation of network management systems, software, operating
systems and underlying hardware (processors) leaves operators with
the fun job of gluing it all together themselves.
Challenges - Onboarding VNFs
• Currently operators are
investigating many different
VNFs from established
suppliers and new vendors.
• Systems integrators can help
with interoperability testing
and onboarding, ensuring VNFs
will work in the specific NFV
environment an operator has
chosen.
• Over time the number of new
VNFs being introduced and
trialled by operators is likely to
diminish making onboarding
less of an issue.
Source: Kyrio (CableLabs test subsidiary)
KPN experience with VNF onboarding
• Test and acceptance: Define a test and release
process to support multiple VNFs in parallel.
Physical and virtual functions are often
combined in test chains.
• VNF catalog: Keep exceptions to a minimum.
Use this process to standardize on VNFs
instead of differentiate.
• Knowledge transfer: Use the application/PNF
owner’s knowledge and learn what is
expected from them as tenants.
Source: KPN, Wendy Ooms
Telefonica - best practice for VNF modelling
• Descriptors should avoid hard-coded values. Parameters are need to
launch an instance with deployment-specific attributes (e.g.
addressing)
• The VIM scheduler must be allowed to choose the resource to assign
to the VNF – physical host parameters (e.g. MAC address) must not
be used
• Don’t bypass the orchestrator by using hidden/hardcoded
parameters, tailored cloud-init files, or a VM console
• Management interface should be reachable after VM boot (should
not require console access to config IP)
• Be clear whether the VIM or the VNF manages IP address allocation
More challenges
• Change management for VNFs - Shannon & Yates state “ONAP
provides a consistent and systematic approach for change
management across all network functions, thereby eliminating the
need for ad hoc scripts”.
• Bridging the IT operations (NFVI) and network operations (VNFM)
domains - there will be greater commonality between the two
domains. The demarcation will probably to shift to those who
manage real-time, customer impacting applications and those that
manage the rest.
Evolution of network management
Day 0 Day 1 Day 2
Traditional
management
• PNF installation
• Initial configuration to
make device reachable
(user, password, network,
etc.)
• License activation
• Injection of
configuration
• Neighbor
configuration
• Network configuration
• Business provisioning (by
BSS)
• Service provisioning (by OSS)
NFV-native
management
• VNF deployment
• Network service
deployment (complex
topology)
• Business provisioning (from
BSS to MANO)
• Service provisioning (by
MANO)
• License activation
• VNF configuration
• Neighbor configuration
Source: Adapted from Telefonica
Further challenges
• DevOps - “Adoption of agile software development techniques
and DevOps principles” are key to rapid development,
certification & deployment of NFV/SDN - Shannon & Yates, AT&T
• Security - one of the advantages of open source is that although
the source code is in the public domain, vulnerabilities are
spotted quickly and patches are swiftly made available.
• Service assurance – see panel 3
Source: clark.comSource: github.com
Swisscom adopts Biz Dev Ops
Source: Swisscom
❑ Automated test has big cost
benefit
❑ Cooperative development (with
vendors) must include testing with
test functions defined and
implemented first
❑ Cross-organizational CI/CD
pipelines need more discussion
between supplier and operator
about
• Toolchain
• Sensitive data handling
• Contractual KPIs
(procurement)
SKtelecom – lessons learned from NFV
• NFV works well: vEPC and vIMS are “definitely feasible”; NFV+SDN can reduce
TCO by 20% and reduce TTM from 12 to 2 weeks but only for large scale
deployments including RAN
• A common MANO within each CSP is essential for a centralized management
framework - avoid vendor & domain-specific MANO
• MANO must support full lifecycle requirements (backup, rollback, etc)
• Standard data models needed for fully automated orchestration
• Open source projects (including OpenStack) are not yet carrier-grade
• For infrastructure performance you need PCI pass-through, SR-IOV, OVS with
DPDK
• For VNF performance these must move from monoliths to independently-
scalable, stateless, infrastructure-independent microservices
• Also need a unified & real-time assurance solution from NFVI to VNF/services
• “Culture shock” - whole operational process from design to run and
debug/maintain must change to support NFV; CTO and CIO offices must merge
DT problems with TeraStream NFV
• 4 years ago DT built a new network called TeraStream which includes an
NFV environment that offloads some traditional router functions (DNS,
DHCP, AFTR, vBRAS, vRouter; caching and streaming) to generic X86
servers (100 Gbps per server expected).
• Initially started with KVM-based VMs and OpenStack
• Most notable issues:
• Hard to keep platform secure (including constant patching of all layers – hosts,
hypervisors, VNFs)
• Extremely high effort to balance VNFs among hosts
• Inconsistency of versions between hosts or different locations
• Very difficult to debug
• Rollbacks are difficult or nearly impossible
• Network performance
DT solution for TeraStream NFV
• Created a Continuous Integration tool chain for development and deployment of
virtual machine images
• VM Factory stores YAML templates to describe hosts and VMs in Git
• Nightly builds via Jenkins ensure up-to-date security patches and other fixes
applied to the images for host OS and VNFs
• Orchestrator (Marlysys) picks up the
new images and start rebuilding nodes
• The result: fast patching, rollbacks
now possible, less dependencies and
coding during run time leading to
fewer errors and better security.
Source: Deutsche Telekom – Tomislav Sukser
CI/CD pipeline for OSM
Source: ETSI
Implementation example - Velcom
• Telekom Austria’s Belarus subsidiary: 80 engineers + 30 from vendor
• Implementation project started Mar 2016
• Core Network equipment and SW delivered May/Jun 2016
• HLR/HSS, SGSN, GGSN, MNP commercial launch Aug-Oct 2016
• MSC/MGW fully migrated Dec 2016
• Fully virtualized Core Network – Jan 2017
• vEPC now live in 2 other countries and being implemented in
remaining across TKA footprint (Austria + 5 East European countries)
• Fully virtualized Core Network across all 7 countries by 2Q19
Turkcell’s priorities for SDN/NFV
Source: Turkcell
Operator positioning re NFV
Source: TBR c/o Telefonica, NFV/SDN Telecom Market Landscape 3Q16 Network Business Quarterly

Lessons learned so far in operationalizing NFV

  • 1.
    PANEL I LESSONS LEARNEDSO FAR IN OPERATIONALIZING NFV
  • 2.
    Panelists Dave Hughes VP ofEngineering PCCW Global Luke Lim Head of Solutions Engineers, EMEA SevOne James Crawshaw Senior Analyst Heavy Reading Ari Banerjee VP Strategy Netcracker
  • 3.
    Outline •What progress hasbeen made in operationalizing NFV and what lessons have been learned? •What are the greatest challenges in automating dynamic, virtual networks and how successfully have these been integrated with legacy physical networks to provide services that span both domains? •Has virtualization increased agility, enabled new business models and reduced opex or simply added to the complexity of network management and operations?
  • 4.
    Operationalizing SDN andNFV Networks • While NFV provides an opportunity to reduce opex and improve customer experience it introduces additional layers of operational complexity that “put more onus on the operator to integrate technologies that were traditionally integrated by a vendor” - Irene Shannon and Jennifer Yates, AT&T, Building the Network of the Future. • Disaggregation of network management systems, software, operating systems and underlying hardware (processors) leaves operators with the fun job of gluing it all together themselves.
  • 5.
    Challenges - OnboardingVNFs • Currently operators are investigating many different VNFs from established suppliers and new vendors. • Systems integrators can help with interoperability testing and onboarding, ensuring VNFs will work in the specific NFV environment an operator has chosen. • Over time the number of new VNFs being introduced and trialled by operators is likely to diminish making onboarding less of an issue. Source: Kyrio (CableLabs test subsidiary)
  • 6.
    KPN experience withVNF onboarding • Test and acceptance: Define a test and release process to support multiple VNFs in parallel. Physical and virtual functions are often combined in test chains. • VNF catalog: Keep exceptions to a minimum. Use this process to standardize on VNFs instead of differentiate. • Knowledge transfer: Use the application/PNF owner’s knowledge and learn what is expected from them as tenants. Source: KPN, Wendy Ooms
  • 7.
    Telefonica - bestpractice for VNF modelling • Descriptors should avoid hard-coded values. Parameters are need to launch an instance with deployment-specific attributes (e.g. addressing) • The VIM scheduler must be allowed to choose the resource to assign to the VNF – physical host parameters (e.g. MAC address) must not be used • Don’t bypass the orchestrator by using hidden/hardcoded parameters, tailored cloud-init files, or a VM console • Management interface should be reachable after VM boot (should not require console access to config IP) • Be clear whether the VIM or the VNF manages IP address allocation
  • 8.
    More challenges • Changemanagement for VNFs - Shannon & Yates state “ONAP provides a consistent and systematic approach for change management across all network functions, thereby eliminating the need for ad hoc scripts”. • Bridging the IT operations (NFVI) and network operations (VNFM) domains - there will be greater commonality between the two domains. The demarcation will probably to shift to those who manage real-time, customer impacting applications and those that manage the rest.
  • 9.
    Evolution of networkmanagement Day 0 Day 1 Day 2 Traditional management • PNF installation • Initial configuration to make device reachable (user, password, network, etc.) • License activation • Injection of configuration • Neighbor configuration • Network configuration • Business provisioning (by BSS) • Service provisioning (by OSS) NFV-native management • VNF deployment • Network service deployment (complex topology) • Business provisioning (from BSS to MANO) • Service provisioning (by MANO) • License activation • VNF configuration • Neighbor configuration Source: Adapted from Telefonica
  • 10.
    Further challenges • DevOps- “Adoption of agile software development techniques and DevOps principles” are key to rapid development, certification & deployment of NFV/SDN - Shannon & Yates, AT&T • Security - one of the advantages of open source is that although the source code is in the public domain, vulnerabilities are spotted quickly and patches are swiftly made available. • Service assurance – see panel 3 Source: clark.comSource: github.com
  • 11.
    Swisscom adopts BizDev Ops Source: Swisscom ❑ Automated test has big cost benefit ❑ Cooperative development (with vendors) must include testing with test functions defined and implemented first ❑ Cross-organizational CI/CD pipelines need more discussion between supplier and operator about • Toolchain • Sensitive data handling • Contractual KPIs (procurement)
  • 12.
    SKtelecom – lessonslearned from NFV • NFV works well: vEPC and vIMS are “definitely feasible”; NFV+SDN can reduce TCO by 20% and reduce TTM from 12 to 2 weeks but only for large scale deployments including RAN • A common MANO within each CSP is essential for a centralized management framework - avoid vendor & domain-specific MANO • MANO must support full lifecycle requirements (backup, rollback, etc) • Standard data models needed for fully automated orchestration • Open source projects (including OpenStack) are not yet carrier-grade • For infrastructure performance you need PCI pass-through, SR-IOV, OVS with DPDK • For VNF performance these must move from monoliths to independently- scalable, stateless, infrastructure-independent microservices • Also need a unified & real-time assurance solution from NFVI to VNF/services • “Culture shock” - whole operational process from design to run and debug/maintain must change to support NFV; CTO and CIO offices must merge
  • 13.
    DT problems withTeraStream NFV • 4 years ago DT built a new network called TeraStream which includes an NFV environment that offloads some traditional router functions (DNS, DHCP, AFTR, vBRAS, vRouter; caching and streaming) to generic X86 servers (100 Gbps per server expected). • Initially started with KVM-based VMs and OpenStack • Most notable issues: • Hard to keep platform secure (including constant patching of all layers – hosts, hypervisors, VNFs) • Extremely high effort to balance VNFs among hosts • Inconsistency of versions between hosts or different locations • Very difficult to debug • Rollbacks are difficult or nearly impossible • Network performance
  • 14.
    DT solution forTeraStream NFV • Created a Continuous Integration tool chain for development and deployment of virtual machine images • VM Factory stores YAML templates to describe hosts and VMs in Git • Nightly builds via Jenkins ensure up-to-date security patches and other fixes applied to the images for host OS and VNFs • Orchestrator (Marlysys) picks up the new images and start rebuilding nodes • The result: fast patching, rollbacks now possible, less dependencies and coding during run time leading to fewer errors and better security. Source: Deutsche Telekom – Tomislav Sukser
  • 15.
    CI/CD pipeline forOSM Source: ETSI
  • 16.
    Implementation example -Velcom • Telekom Austria’s Belarus subsidiary: 80 engineers + 30 from vendor • Implementation project started Mar 2016 • Core Network equipment and SW delivered May/Jun 2016 • HLR/HSS, SGSN, GGSN, MNP commercial launch Aug-Oct 2016 • MSC/MGW fully migrated Dec 2016 • Fully virtualized Core Network – Jan 2017 • vEPC now live in 2 other countries and being implemented in remaining across TKA footprint (Austria + 5 East European countries) • Fully virtualized Core Network across all 7 countries by 2Q19
  • 17.
    Turkcell’s priorities forSDN/NFV Source: Turkcell
  • 18.
    Operator positioning reNFV Source: TBR c/o Telefonica, NFV/SDN Telecom Market Landscape 3Q16 Network Business Quarterly