1 © Copyright 2017 Dell EMC

MX-Series
Networking

                 Restricted - Confidential

2 © Copyright 2017 Dell EMC
 Presenter

Eric Kuzmack
Senior Product Manager, Dell EMC Networking

3 © Copyright 2017 Dell EMC

 Agenda
This course will discuss how networking operates in the MX-Series

• Management Modes
• Hardware Overview
• Scalable Fabric Architecture
• Embedded Top of Rack functionality

Pre-requisites

A basic understanding of the M1000e networking options

4 © Copyright 2017 Dell EMC

 MX-Series Networking:
Management Modes

5
 Define terms
SmartFabric MCM Domain
In MX-Series, a A group of servers,
SmartFabric is a set storage, networking
of two or more and chassis that are
interconnected managed as part of a
switches that are single Multi-Chassis
managed as a single Management group
entity

6 © Copyright 2017 Dell EMC

 Open Networking Mode
Unprecedented choice and capability
Service
WAN, MPLS/VPLS Provider Core
Enterprise Infrastructure
IP/MPLS WAN functionality

SD-WAN based on flexible NFV

approach and deep security
SD-WAN & hybrid WAN solutions

Cloud-delivered SD-WAN

Data Center fabric &

OS10 in-rack switching
SDN fabric & network
tapping solutions
Linux on the switch, unified
server, network management
Virtualization-centric fabric optimized
Enterprise Data Center for Nutanix, Big Data, VDI

7 © Copyright 2017 Dell EMC

 Full Switch Mode – OS10 Enterprise Edition
OS10 – Redefining network operating system software
OS10
Lifecycle Management DevOps NetOps
• Integrated with OME-M Native Linux & 3rd Party OS10
• Configure Hostname, SNMP, Ports, etc… Open Source Apps Apps Enterprise Edition
• Monitor Health, logs, alerts, and events Management Automation
Common
Management
• Update/Manage firmware Tools Tools
Services (CMS)
• View physical topology
• Power Management IP Services Fabric Services L2/L3 Protocols

Linux Networking Security Services Policy Control

Dataplane Configuration
• CLI, API, Ansible, etc… Development Environment via Control Plane Services (CPS)
• Supports all protocols and features of Dell EMC Networking OS10 Open Edition (Native Linux)
OS10 Enterprise Edition
Platform Abstraction via OCP Switch Abstraction Interface (SAI)

8 © Copyright 2017 Dell EMC

 Network Simplification: The Past
 I/O aggregation
 Offers Active-Active
multi-path for servers VLT Mode
 Semi-customization

 I/O aggregation
 Stack up to 6 units
Stacking
 Fully Automated Standalone  Semi-customization Mode
 No customization
 I/O aggregation Mode
(default)  I/O aggregation
 Offers custom PMUX
configuration using CLI
 Moderate Mode
customization

 L2/L3 switching
 Everything needs to be Full-switch
configured using CLI
 Full customization Mode

Was not well integrated into the overall modular user experience
9 © Copyright 2017 Dell EMC
 The Future: SmartFabric Services for Modular

4
3
2
1 Failure and
Physical
Advanced
Datacenter
topology
life
integration
remediation
cycle
validation
management
and
automation
operations
Provides I/O Aggregation capabilities to avoid complex upstream switch
Fabric
Detectsautomatically
fabric
configuration
Fabric-wide ensures
configuration compliance
misconfigurations and link
consistency with acceptable
level
enables failure topology
global conditions
settings andas
such
heals the fabric
NTP, SNMP, etc…upon failure
to be condition
consistent removal
across all switches in the system
Notifies user multi-pathing
Active-Active of error and corrective
for hosts action to take
Hardware
Fabric levelfailure?
firmware deployment safely upgrades all switches in a
Failure
VLAN
•fabric,
Swap and
and
onethe Remediation
Priority
non-functional
switch Automation
assignments
at a time switch detects
attached
with to fabric
server
a new misconfigurations
profiles
one drives
and link level
automated
• Firmware QoS isfailure conditions
operations
automatically and heals
updated to bethe fabric upon
consistent failure
in the fabric
condition
• Physical removal
topology validated before switch is brought online
One CLI and API to manage multiple switches and multiple fabrics
10 © Copyright 2017 Dell EMC
 SmartFabric Services Topology Validation

Highlights topology errors and

indicates corrective action

11 © Copyright 2017 Dell EMC

 MX-Series Networking:
Hardware Overview

12
 MX-Series internal connections
Orthogonal connection

Fabric A Orthogonal connection

• Ethernet switch
MX5108n
12Gb
• SAS:
Fabric Switching
MX5016s
Engine SAS
• MX9116n
Ethernet switch
switch
MX5108n
• Fabric Expander
• Module
Fabric Switching
OR
Engine
MX7116
MX9116n
• Fabric Expander
Fibre Channel:
Orthogonal connection Module
Fabric B Orthogonal connection
MXG610s
MX7116Fibre
Channel switch

Fabric C Midplane Connection

13 © Copyright 2017 Dell EMC

 MX-Series switching modules
8 x 32Gb FC SFP+ ports Two 4 x 32Gb FC QSFP ports

16 server-facing 32Gb FC internal ports

High performance Gen 6,

32Gb data rate switch ideal
for connectivity to all-flash
SAN storage solutions

Designed for maximum

MXG610s Fibre Channel Switch flexibility and value with “pay-
as-you-grow” scalability with
Ports on Demand (PoD)
licensing

14 © Copyright 2017 Dell EMC

 MX-Series switching modules
8 server-facing 25GbE internal ports 40Gb QSFP+ 2 x 100Gb QSFP28 4 x 10GBASE-T

Targeted at customers with

one or two chassis

Ideal for enterprises requiring

MX5108n Ethernet Switch basic Ethernet switching with
FCoE transit capability

Performance
• 550ns latency
• 960Gbps switching fabric
• 720 Mpps forwarding
capacity
15 © Copyright 2017 Dell EMC
 Introducing: QSFP28-DD

QSFP28-DD: Double Density

• Slightly deeper than QSFP28 with a

second row of contacts

• Backwards compatible with QSFP28

• Provides eight lanes in one connection

• 2 x 40GbE/100GbE
• 8 x 10GbE/25GbE

16 © Copyright 2017 Dell EMC

 MX-Series switching modules
16 server-facing 25GbE internal ports 2 x 100GbE/32Gb FC QSFP28 Uplink Ports

MX9116n Fabric Switching Engine

12 x QSFP28-DD Fabric Expander/Uplink Ports 2 x 100Gb QSFP28 Uplink Ports

High bandwidth converged

solution with investment
protection for a large fabric

Performance and Scaling

• 112 servers @ 25GbE
with no oversubscription
• 350ns latency
• 6.4 Tbps switching fabric
• 3.248 Bpps forwarding
capacity
17 © Copyright 2017 Dell EMC
 MX-Series storage networking
SAN
FCoE
MX9116n Direct Attached FC FCoE

FCoE Fabric Switching Engine FC Gateway SAN

FCoE
MX5108n FCoE SAN
FCoE Ethernet Switch
Native FC
MXG610s Direct Attached FC FC SAN

18 © Copyright 2017 Dell EMC

 MX-Series Networking:
Scalable Fabric Architecture

19
 What is MX Scalable Fabric Architecture?

? How can multiple server chassis behave like a single

chassis from a networking perspective?
Traditional/Legacy modular solutions MX Scalable Fabric Architecture provides Best-
Ethernet switches in each chassis in-Class Multi-Chassis Ethernet
Multiple hops for east-west communication Minimum of 2x25GbE to each server
Cabling to ToR and Core Maximum of 8x25GbE to each server
Multiple switches to manage No oversubscription
Expensive! “north-south” latency penalty for multi-chassis
Cisco introduced UCS with FEX communication <80ns
• FEX reduced management complexity, but… ~550ns “any-any” latency
• Significant chassis-chassis latency (~1.4us) Scales up to 10 chassis, 80 compute sleds
• Potential for significant oversubscription Single cable supports 200Gb of bandwidth
• Firmware upgrades in UCS are complex Low TCO

20 © Copyright 2017 Dell EMC

 MX-Series switching modules
16 server-facing 25GbE internal ports

QSFP28-DD Connection to Fabric Switching Engine

Scales fabric bandwidth

across multiple chassis

MX7116n Fabric Expander No OS runs here, No

switching happens here

Module Connects to Fabric Switching

Engine through QSFP28-DD
ports

21 © Copyright 2017 Dell EMC

 MX Scalable Fabric Architecture Topology
LAN SAN

All switching happens here

Fabric Switching Engine

NO switching happens here

NO operating system here
Very low latency
Fabric Expander Module Fabric Expander Module

…
Chassis 1 …
22 © Copyright 2017 Dell EMC
Chassis 2 Chassis 10
 Recommended FSE placement is
MX Scalable Fabric Architecture Topology across two chassis for redundancy
Chassis 1: Chassis 2:
Slot A1: FSE Slot A1: FEM
Slot A2: FEM Slot A2: FSE

Chassis 3:
Slot A1: FEM
Slot A2: FEM

….
Chassis 10:
Slot A1: FEM
Slot A2: FEM

23 © Copyright 2017 Dell EMC

 Multiple fabrics in a single Multi-Chassis Group
MCM Group: 20 Chassis
Fabric 1: 10 Chassis Fabric 2: 5 Chassis Fabric 3: 3 Chassis

Fabric 1 Fabric 2
Chassis Slot A1 Slot A2 Chassis Slot A1 Slot A2
1 FSE FEM 11 FSE FEM
2 FEM FSE 12 FEM FSE
3 FEM FEM 13 FEM FEM
4 FEM FEM 14 FEM FEM
5 FEM FEM 15 FEM Fabric
FEM 3
6 FEM FEM Chassis Slot A1 Slot A2
7 FEM FEM 16 FSE FEM
8 FEMFabric 4: 1 Chassis
FEM Fabric
17 5:FEM
1 Chassis
FSE
9 FEM FEM 18 FEM FEM
10 FEM FEM

Chassis Slot A1 Slot A2 Chassis Slot A1 Slot A2

19 25G SW 25G SW 20 25G SW 25G SW

24 © Copyright 2017 Dell EMC

 Fabric Switching Engine
Embedded ToR

25
 Traditional mixed blade/rack networking
• Rack servers connect to
their own ToR switches
• Communication between MX9116n MX9116n

rack and blade servers must

traverse the spine
• Storage array consumes
expensive spine switch ports
• Together, this design adds
significant cost and latency

26 © Copyright 2017 Dell EMC

 MX-Series networking with embedded ToR
• All servers and storage connect to the MX9116n MX9116n

Fabric Switching Engine

• Communication between all devices is kept
within the FSE
• Single point of management and network
security
• Reduces cost and improves performance
and latency
• Manage PowerEdge Blade and Rack
servers from a single interface (post-RTS)
27 © Copyright 2017 Dell EMC
 Thank you
                 Restricted - Confidential
28 © Copyright 2017 Dell EMC