Kick starting Network Automation
3 likes1,191 views
The document discusses the evolution and strategies in network automation from a systems engineer's perspective, highlighting the necessity for networking to adapt to modern demands such as containerization and distributed systems. It emphasizes the importance of using open-source tools and frameworks to enhance automation, scalability, and efficiency within networking environments. Key topics include configuration management, the role of event-driven automation, and the integration of emerging technologies like AI and machine learning in network operations.
Kick starting Network Automation
- 1. Kickstarting Network Automation systems engineer perspective on NetOps Walid Shaari @walidshaari https://www.linkedin.com/in/walidshaari cfgmgmtcamp 5th February 2018 Gent background image credit: https://commons.wikimedia.org/wiki/File:Social_Network_Analysis_Visualization.png
- 2. > show user Walid Shaari @walidshaari https://www.linkedin.com/in/walidshaari ● System engineer supporting HPC Linux clusters ● Configuration management evaluation and deployment project in 2014 ● Advocating open source, automation, containers and Kubernetes ● Husband and father of 3 lovely kids ● Last 3 months in short work assignment with Network management team
- 3. Agenda ● Motivations ● Observations and hints how not to fail ● Automation history from the system persepective ● The proposed software solution ● How about non-software automation? ● Example use case
- 4. Incentives
- 5. • Open source and standards • Pure Layer 3 network implementation • Lightweight IP to IP encapsulation • Policy based secure networking • Scalable and simple • Scale out SDN controller • Openstack , Docker, Kubernetes, Mesos and CNI
- 6. Networking setups in 2018 “In 2018, we will see more demands placed on the continuous delivery of changes to networking setups due to pressure from containerisation, distributed systems, and security needs. Thus, networking must become as flexible and automation-friendly as the software that runs over it, and become less of a bottleneck.” https://www.itproportal.com/features/what-do-organisations-need-to-prepare-for-in-2018/ Nigel Kersten, Chief Technical Strategist at Puppet
- 7. How to build an event driven, dynamically reconfigurable microservices platform by Sven Beauprez: https://youtu.be/1D8hyLWMtfM
- 9. Context Team: handful network engineers supported by handful infrastructure & cabling Network: Campus Data Centers Medium scale less than 200 switches legacy heterogeneous platforms and models only programmable devices are Juniper with netconf, or Aruba with Rest Existing tooling in place CMDB IPAM NMS : SNMP based Topolgy: Traditional Three tier architecture “Core-Distribution-Access” with extended layer 2 Frequent operational activities: Changes Troubleshooting New applications constraints
- 10. CLI Cut & Paste NPA Notepad Automation Frameworks & Controllers ● Excel ● Python jinja2 ● Templating engines ● Ansible, ● Puppet ● Chef Event Driven Automation ● Sensor triggered events ● napalm_logs ● Salt ● IFTTT ○ StackStorm Ansible AWX Intent Based Networking Declarative Network Intent Composition Aspen Boulder Check salt room Micea Ulinic talk event driven network config using Salt
- 12. • • • • • • •
- 14. ● What is NetOps, NetDevOps? ● What problems challenges we would like to solve? ● How much visibility into operations we have? ● What are your responsibilities? ● Tasks/processes you are responsible for ? ● How much time you can set aside to look into automation? ● What current automations already in place? (processes, tools, checklists) ● Would you like to have automations processes/tools? Why? ● What automations you are aware of, or would like to have? processes/tools ● sharing medium, how should we enhance activity communications? ( knowledge, process, updates, documentation, in-job training) ● How should we start?
- 15. Possible Business Drives Pressing Needs: - Scale: increasing environment size, scale and diversity - Optimization: more to do with less engineers - Failure management: solve new problems, guarantee level of performance and consistency. Opportunities: - 4IR: fourth Industrial revolution - IoT - Big Data - Digital transformations and cloud initiatives - Artificial intelligence and machine learning - Services “e.g. microservices” demand growing infrastructure and hence network scale
- 20. Application and system automation Ansible CFEngine promise theory converge eventually Chef Foreman DevOps Day Kubernetes Terraform sysdig consul 2005 1993 2011 2012 Puppet Git CapistranoScripting imperative In-house 2009 2010 Vagrant Rundeck Salt Rudder Fabric 2013 2014 2015 2016 2017 Docker etcd cfgmgmtcamp Stackstorm Packer ELK stack mgmt Habitat Envoy Openfaas Puppet Bolt Istio Brigade Nomad Helm
- 21. History of Programmable Network The Road to SDN: https://queue.acm.org/detail.cfm?id=2560327
- 22. The network world perspective White Box Pica8 batfish ONIE Trigger RANCID NetFlow ssh OpenWRT Junos Stdlib Junos Ansible Junos Chef Facebook Wedge gobgp 2001 1996 2008 2012 sflow SNMP 2004 2006 NetConf Ethernet Fabric (CLOS)** SDN 2013 2014 2015 2016 2017 Junos Puppet Cumulus Networks OpenDayLight Apstra AOS Ansible 2.0 SaltStack Snaproute SONIC Intention Based Networks FaceBook Open/RNetworktoCode ntc N.A.P.A.L.M. OpenConfig * Dates may be inaccurate. they were collected from initial release of standard, commit, or project info and other talks ** Research has roots back from 1953 1993 Yang 2010 LEGEND monitoring validation management automation container validation device hardware topology RMON 2000
- 23. Challenge 1: The Sandbox vrnetlab
- 25. • • • •
- 26. • •
- 27. Process & Technology KISS Workflow The Modern CLI ● syntax highlighting ● Validation, linting, indentation ● the Automation UX Automation Platform ● Configuration management ● Orchestration ● Role Based Access ● Scheduling ● Remote Execution ● Event based triggers Version Control System ● History tracking ● Peer review ● Collaboration Engine ● Live Documentation ● Integration with issue tracker 03 01 02
- 28. Start small and simple
- 30. • • • • • • •
- 32. Ansible vendor modules Arista EOS - name: set ntp server in the device eos_commands: - “ntp server {{ ntp_servername }}” CISCO IOS - name: set ntp server in the device ios_commands: - “ntp server {{ ntp_servername }}” Juniper Junos - name: set ntp server in the device junos_commands: - “set system ntp server {{ ntp_servername }}”
- 34. - - - -
- 35. Configuration management 101 Desired Policies: - Device state(s) - Topology state - External input Contellor Data Model: - Configuration - Operational Network Device(s) Data Model: - Configuration - Operational Topology (protocols) -..-.-..-.-.-..-. Configuration difference (e.g. NetConf) =-...-.-=.. State: - Telemetry - SNMP - NetConf - RestConf - Syslog Connection:: - telnet*/ssh - https
- 36. IETF Programmability Strategy CLI best practices SNMP experience Operations requirements Netconf, RestConf and YANG Charles Eckel, Cisco DevNet: https://fosdem.org/2018/schedule/event/opendaylight/
- 37. YANG: Yet Another Next Generation Data model language used to model Configuration and operational state data easy to extend on existing models IETF standard defined in RFC 6020 Data model language for both Configuration and operational state data A solution to the multi-vendor device data discrepancy Not All vendors yet serious about it Ivan Pepelnjak on Monday, January 29,2018 blogged:: http://blog.ipspace.net/2018/01/use-yang-data-models-to-configure.html
- 38. NetConf IETF Network management protocol ● Defined in RFC 4741 (2006), updated by RFC 6241 (2011) ● Provides mechanisms to install, manipulate, and delete the configuration of network devices ● Model driven APIs ● Distinguishes between configuration and operational/state dat ● Multiple configuration datastores (candidate, running, startup) ● Configuration change validation and transactions ● Selective data retrieval via filtering ● Streaming and playback of event notifications
- 39. RestConf ● IETF RFC 8040 ● Configuration data and state data exposed as resources ● How to access the data using REST verbs (GET / PUT / POST/ …) ● How to construct URIs to access the data ● HTTP instead of SSH for transport ● JSON in addition to XML for data encoding
- 40. Takes time and effort to standardise
- 41. Junos Managed Device IOS-XR Managed Device EOS Managed Device netconf over ssh port 830 RestConf over https port 443 YANG based XML YANG based XMl/JSON Start Run Candidate data store
- 42. Other types of network modules Ansible supported modules: ● netconf: ○ netconf_config ● vendor_config ○ ios_config ● vendor_cmmand ○ ios_command Minimum Viable Platform Agnostic modules: e.g. net_interface Vendor/Community supported modules: ● netconf: ○ junos_netconf ○ ce_netconf ● Network to Code: ○ ntc_install_os ○ ntc_get_facts ● N.A.P.A.L.M: ○ napalm_diff_yang ○ napalm_get_facts Custom built module: https://www.ansible.com/ansible-module-development-101
- 43. • • • •
- 44. Resources Networktocode slack channel http://networktocode.herokuapp.com/ *** Absolutely Gorgeous ***** SDN & NFV: https://fosdem.org/2018/schedule/event/opendaylight/ Blogs: Csilla Bessenyei Networker and coder https://networkerandcoder.wordpress.com/ Kirk Byers “Python for network engineers” https://pynet.twb-tech.com/ Mircea Ulinic https://mirceaulinic.net Jason Edelman http://jedelman.com/ David Lore http://ipengineer.net/ netmiko https://github.com/ktbyers/netmiko N.A.P.A.L.M https://napalm-automation.net/ Training: gns3 Academy http://academy.gns3.com/ Ansible network automation examples: https://github.com/network-automation Salt: https://docs.saltstack.com/en/develop/topics/network_automation/index.html Net survey: https://docs.google.com/forms/d/e/1FAIpQLSdiBNMK0ZUmgBSNEaOWa-YHGQ4AlZo7EhB52_dXzvMqic3eHA/viewanalytics https://interestingtraffic.nl/2017/03/27/insights-from-the-netdevops-fall-2016-survey/ ipspace blog and podcast: https://www.ipspace.net packetpushers podcast: http://packetpushers.net/
- 52. - - - - - -