Automating for a
Smarter Future
Streamline | Empower | Perform
Spirent Automation Platform Technologies
Marc HOREM
Pre-sales Systems Engineer
Velocity Platform Breaks Barriers to Automated
Continuous Test
Managers Developers Testers Lab Operations Customer End Users
Browser Based User Interface
Resource Control and Test Authoring and Abstract Execution and Test Analysis
Automated Workflows Testbed Orchestration Network Connectivity
Dashboards Automation Libraries Reporting and Dashboards
REST API
DevOps Ready
2
Test as a Service
Features and Value Propositions
Turbo-charge automation proficiency Capture and replay
Leverage Python automation for faster Increase quality and predictability with
releases and increased test coverage automated tests matching test procedures
NFVi, VNF and Service Test case abstraction
Level Testing Quickly and efficiently generate reusable
Control life cycle testing tests to validate multiple testbeds
using one single tool
Unified Test Reports Automated data extraction
Streamline root cause Gain actionable information by
analysis using a single report automatically deconstructing
from all testbed devices device responses
Spirent’s LaaS Solution
DevOps Ready Integration
Maximize organizational efficiency
Maximize reuse and test coverage
by plugging into common DevOps applications
by deploying and consuming tests seamlessly
4
Build out Automation with Capture/Replay
Test Engineers Create Automation Following Their Natural Workflow
CLI
Telnet, SSH, CMD, Serial, Process, File, Syslog, TCL Shell, Python Shell,
PowerShell
GUI
Web, Java, Flex, VNC, Selenium/Ranorex, TestPlant
Network Traffic Generator
Spirent (STC, Landslide, Avalanche, CloudStress, Smartbits)
Ixia (IxLoad, IxTrffic, IxNetwork, N2X)
Enterprise, Virtual & Cloud
VMware, OpenStack, Database, Web Services (RESTful/SOAP/ XML-RPC)
Protocol & Communications
SNMP, HTTP, UDP, XMPP, NETCONF, SMTP/Email, POP3, Wireshark
Mobile
ADB, VNC, Ranorex (APPs), TestPlant
Desktop App
Ranorex (Windows), TestPlant (Any GUI), VNC
PROPRIETARY AND CONFIDENTIAL 5
Build out Automation with Capture/Replay
Test Engineers Create Automation Following Their Natural Workflow
Technical Benefit
CAPTURE REPLAY • Capture every action during
manual test
• Replay captured steps into
automated tests
Economic Benefit
• Save time and money
(Domain experts follow
natural workflow)
• Increase quality and
predictability (automated
test cases exactly match
test procedure)
Extend with Analysis
Validate with Spirent Patented Response Maps
Technical Benefit
• Extract relevant data from
responses for
measurements and
validation
Economic Benefit
• Save time on low value
pattern matching that is
better applied to expanding
test coverage
Portabilize via Abstraction
Run the Same Test Cases Against Different Testbeds
Technical Benefit
• Abstract the device specific
Abstract
interface iTest
in QuickCalls
Test Case
Economic Benefit
• Reduce maintenance costs
Vendor A
Vendor C
Vendor B
Publish – Share – Consume
Share test cases with all teams
Technical Benefit
Test Case Test Case Test Case
Creation Publishing Consumption • Publish tests for wide
consumption both in labs and
production
Economic Benefit
Abstract • Increase productivity and
iTest reduced cost (full team
Test Case empowered to consume
automation from central
portal
PROPRIETARY AND CONFIDENTIAL 9
Interpret with Rich Reporting
Quickly Find Failure Points Using a Consolidated Test Report
Technical Benefit
• Leverage detailed reports
to perform root cause
analysis
• Capture every single test
step in detailed, unified
customizable report
Economic Benefit
• Save time in determining
root cause of failures
• Increase quality and
customer satisfaction
Integrate with Python frameworks
Convert iTest to Python for Automated Regression Testing
Sprint Testing iTest to Python Regression Testing
Conversion Technical Benefit
• Turbocharge Python code
with iTest’s powerful
QuickCalls and Response
Maps
• Publish automation to
Python test regression
systems
Economic Benefit
• Save time and money
• Increase developer
productivity
Enable DevOps Continuous Testing
DevOps Tool Chain integrations enable Continuous Test and Immediate Feedback
Source Control SDLC / Bug
Technical Benefit
Management Management
• Integrates with DevOps Tool
Chain applications
• Identify defects immediately
after software builds
Economic Benefit Code Results and
Changes Developers, Testers, Bug Reporting
• Increase quality and
Product Owners
customer satisfaction
Test Jobs
Jenkins Build Test Execution
PROPRIETARY AND CONFIDENTIAL 12
Validate at Scale with Distributed Agents
Deploy test agents near devices under test (DUT)
Technical Benefit
• Integrates Execute at scale
intelligent, high
performance, distributed
automation agents
Developers and • Parallelize test execution
Testers locations
Economic Benefit
• Increase Accelerate time to
market
• Increase product/service
quality
PROPRIETARY AND CONFIDENTIAL 13
Differentiated Lab and Test as a Service Workflows
Over 15 Automated Workflows!
Testbed Data
Cable
connectivity Collection
Workorder
(L1, L2, and
Dispatch
virtual) Management
Test
Resource
Execution
Selection
Queueing
Utilization and
Resource
Test Trending
configuration
Reports
Testbed
Standups Reservation Triggered
Request and Automation
(physical,
Approvals
Inventory virtual, hybrid)
Test Reservation
management
Execution Resource
and
Scheduling Test Resolution
population
Execution Regression
Agent Reservation
Organization
Selection Sharing
and Dispatch
Lab as a Service
Features and Value Propositions
Built-in Automated Workflows
Turbo-charge automation proficiency
Streamlined natural and automated workflows
Leverage Python automation for faster
for effective use of commonly deployed lab and test
releases and increased test coverage
processes/procedures
Topology and Resource Abstraction
Physical, Virtual and Hybrid Testbeds
System intelligently creates testbeds based on abstract
Supports all resource classes to
criteria of resources ensuring high utilization of
facilitate transition from physical to virtual to cloud
devices/applications
Global Instantaneous Access Automated Testbed Creation
to Lab Resources & Orchestration
Enabling secure remote access Enable zero touch self-service testing to
and control of testbeds from anywhere at anytime increase productivity and user satisfaction
Fully Automatable and Extensible Deploy Test as a Service
Integrate and execute any automated Seamlessly deploy and consume
scripts with multi-language support, tests to maximize reuse and test coverage
customize with script-based automated workflows
16
Automated Testbed Creation &
Orchestration for Physical, Virtual and
Hybrid Resources
Create Automated Reusable Testbeds for Increased
Self-Service Efficiency
Supports OpenStack and VMware clouds
• Technical Benefits
– Reduce testbed setup times
– Guaranteed, repeatable testbed setup
– Eliminate human errors
• Economic Benefits
– Catalog of predefined/ preconfigured testbeds
– Powerful inventory search
– Optimized resource selection and routing across
L1/L2/SDN
– Manual cabling workflow management
– Triggered Automation and configurable drivers
17
Resource Abstraction
Lab consolidation is a non-starter without Abstraction
User Asks For:
Server 32-bit, 16GB
• Technical Benefits
– Convert static into dynamic environments
– Eliminate stranded resources Resource Inventory: Servers
– Isolate users from lab complexity
Match Match
• Economic Benefits
32-bit, 16GB
32-bit, 16B
– Rapidly find/reserve any resource that in use
meets the requirements
– Optimally employ resources
32-bit, no 32-bit, 32-bit,
– Increase testing coverage – combinations connection 8G 8G
and permutations 32-bit, no
connection
64-bit
PROPRIETARY AND CONFIDENTIAL 18
Topology Abstraction
Portable, reusable, scalable & maintainable topology
Resource Inventory
Firewall
Router
Router Router Switch
Switch
Server Firewall Server
PROPRIETARY AND CONFIDENTIAL 19
Easily Integrates with Your
DevOps Tool Chain
Embrace and Extend Existing Test Automation
• Technical Benefits
– REST API
– Supports Python, Bash, iTest and
other languages
• Economic Benefits
– Knowledge reuse reduces time to
market
– Extend best-of-breed Lab Ecosystems
and/or DevOps Tool Chains
20
Architecture
Stateful Lab E2E Lab Topology Topology
Inventory Connectivity Resolution Scheduler
Interactive Configuration Policy Automated
Control & Provisioning Enforcement Workflows
Distributed Automation Automation
Automation Scheduler Repository
Velocity LaaS Platform
Physical/Virtual/Hybrid Lab
Location 1 Location 2 Location N
Lab Lab Lab
1 3 N
Velocity Agent Velocity Agent Velocity Agent
Velocity Agent Velocity Agent Velocity Agent
Lab Velocity Agent Lab Velocity Agent Lab Velocity Agent
2 4 N+1
21
VNF Testing
Testbed
Physical lab System under Test
Test VNFO
Manager VNF Controller
FG
Test VNF VNF VNF Test VNFM
PNF / VNF 1 2 3 PNF / VNF
Router Switch
Virtualization
VIM
Compute Storage Network
Firewall Server
Orchestrating a virtual testbed requires instantiation of virtual devices on top of the virtualization infrastructure and existing environment.
Adding further complication are combined physical and virtual environments, known as Hybrid. 22
Dashboards and Visualizations
Reveal CapEx, ROI, test coverage and effectiveness
• Technical Benefits
– Centralized scalable data collection
– Resource metrics – Availability, Utilization,
Conflicts
– Usage metrics – reservations & executions
• Economic Benefits
– Analytics dashboards for trending analysis
– Real-time insight into test processes and
results
– Determine CapEx ROI
PROPRIETARY AND CONFIDENTIAL 23
5G CI/CD/CT Demo Sequence
Engineer Develops Automated 5GC Test Case
Open iTest and Open Landslide Parameterize Convert to test case
import testbed session and connect test case and execute
2 3 4 5
Landslide AMF
1 6
SMF
Automation
Engineer
Automation automatically
Create and reserve testbed appears in Velocity Push automation
content to Git
PROPRIETARY AND CONFIDENTIAL 25
5G Automation Dev/Execution Sequence
Engineer Develops Automation, Publishes, and Executes in Velocity
2 1 Developer creates automation in iTest, Python, or Robot
2 iTest content is easily exported to Python or Robot
1 3 Developer pushes automation to Git, appearing instantly in Velocity
4 Velocity spins up dynamic topology or reads from existing environment
5 Topology is passed to iTest, Python, or Robot upon execution
3 6 All results and artifacts are published to common destination
4 Reports and
Artifacts
PROPRIETARY AND CONFIDENTIAL
5 6 26
5G CI/CD/CT Demo Sequence
Automated Regression Triggered by Appearance of New 5GC Build
Poll / Trigger
Upload Download Regression
Image 5G Regression Setup Testbed Dynamic 5G
Registry Build Reserve Resources Testbed
VNF
Build 1 2 3 Instantiate VNFs 4
Supplier Connect and Configure
Execute Test Sequence
NFVi Readiness Testing Setup
5G Functional Testing Test
Jenkins 5G Performance Testing Teardown
E2E Integration Testing
1 VNF supplier uploads new 5GC build Teardown Testbed
Testbed Teardown
2 Jenkins detects new build available for testing Release Resources
Gather Results Publish
Report Generation Reports and
3 Jenkins invokes regression test on Velocity Save Artifacts Artifacts
Velocity spins up dynamic topology, performs regression test,
4 generates results and artifacts
PROPRIETARY AND CONFIDENTIAL 27
5G CI/CD/CT Reference Architecture
Continuously Test 5G Builds, Configurations, and VNF Templates
Push Hook Store
Testcase Test Suites
Repo
Automation
Engineer Tests
Setup Testbed
Push Hook Reserve Resources
Service Trigger Instantiate VNFs
Config Repo Regression Connect and Configure
Configs
5G SME Execute Test Sequence
Engineer NFVi Readiness Testing Setup
5G Regression 5G Functional Testing Test
Push Hook Teardown
Build 5G Performance Testing Dynamic 5G
Templates VNFD Repo E2E Integration Testing Testbed
Cloud Teardown Testbed
Engineer Testbed Teardown
Release Resources
Gather Results Publish
Poll / Report Generation Reports and
Download Jenkins Save Artifacts Artifacts
Upload
Image
VNF Builds Registry Any relevant change to the 5G service (builds,
Supplier configurations, virtual environment) triggers full end-to-
PROPRIETARY AND CONFIDENTIAL end regression testing. 28
