Lightning talks
Download as PPT, PDF0 likes348 views
This document contains confidential information about a cloud computing event on October 20, 2010 discussing challenges of short-term load spikes and strategies for handling increased traffic, including making content static, optimizing payloads, load testing, and using services like Amazon S3 and EC2. Another discussion covered using the ZFS file system for cloud storage and its advantages over other options for snapshots and replication between servers.
![Thanks for listening Level 2 – medium confidentiality, not for further distribution. twitter: @tabamatu * email: [email_address] (* I only have 60 odd followers – it’s safe to click links I tweet )](https://image.slidesharecdn.com/lightningtalks-101021063049-phpapp02/85/Lightning-talks-9-320.jpg)
Lightning talks
- 1. Copyright ioko 2010. This document contains proprietary information of ioko and may not be reproduced in any form or disclosed to any third party without the expressed written permission of a duly authorised representative from ioko. CloudCamp London - 20 th October 2010 Adventures in short-term load spikes (or, how I lost my hair ) Andy Parker – Operations
- 2. Fast broadband (well, ok, getting faster) WiFi everywhere Smartphones, tablet devices, netbooks and laptops consuming content constantly Ability to reach out to millions of connected users simultaneously Driving audience/consumer interaction creates increased demand for content, which increases revenue, which increases money to spend on flashy websites, which increases demand, which requires capacity, which requires building & deploying, which requires time and effort, which reduces time available to support issues, which… RAAAGGHHH – I NEED A HOLIDAY!!!! The internet user in 2010 Level 2 – medium confidentiality, not for further distribution.
- 3. 4.5M viewers (live show) given a link to interact with the characters… Asking a TV audience to come visit Level 2 – medium confidentiality, not for further distribution.
- 4. @lilyroseallen learning the hard way… Level 2 – medium confidentiality, not for further distribution.
- 5. Level 2 – medium confidentiality, not for further distribution.
- 6. Level 2 – medium confidentiality, not for further distribution. Angry Mob Database Application Load Balancer Web Server Web Server
- 7. Level 2 – medium confidentiality, not for further distribution. VERY Angry Mob Database Application WWW (LB) Web Server Web Server S3 hosted .js 302 EC2 Instance EC2 Instance EC2 Instance EC2 Instance EC2 Instance nginx/varnish reverse proxy cache nodes CNAME www to S3, serve a .js to 302 to proxy cache node only traffic is to seed proxy caches (or writes) Using EC2 as a “rain coat”
- 8. Make everything as static as possible at the front end – avoid users hitting app code directly (sacrifice UX if it means keeping the site up) Try to keep caches warm to avoid cache ‘flooding’ Cold CDN nodes can bite the origin hard, especially with video content - used tiered distribution or parent caching so CDN nodes feed each other without hitting you Optimise payload as much as editors/designers will allow - then insist it’s not good enough and optimise some more If driving users to specific content within the site, ensure it’s clearly visible (taking into account nebook/tablet sized screen resolutions) Load test – KNOW YOUR LIMITS (browsermob is a good option) Stuff to consider if large spike expected Level 2 – medium confidentiality, not for further distribution.
- 9. Thanks for listening Level 2 – medium confidentiality, not for further distribution. twitter: @tabamatu * email: [email_address] (* I only have 60 odd followers – it’s safe to click links I tweet )
- 11. Using ZFS for Cloud Storage
- 12. Snapshots What is a snapshot? Instantaneous point-in-time copy of a filesystem What are they good for? Backup Rollback Failure recovery Manageability
- 13. The cloud storage model Infrastructure (e.g. EC2) Platform (e.g. Web Cluster) Service filesystem snapshots Today's norm A better way filesystem and snapshots
- 14. Benefits of the new model Move data between servers efficiently No vendor lock-in Move data between clouds Move data in and out of the cloud (real or virtualised infrastructure) Crucial to implement a hybrid cloud where you don't require virtualisation in your data centre
- 15. Relying on infrastructure for snapshots has problems Why? Inconsistent snapshots on standard filesystems, or shut down your instance!? Even with a pausable filesystem, your app can crash: tinyurl.com/snapcrash
- 16. Comparison: Filesystems with snapshots ZFS Mature, stable, fast Send/recv replication Linux support was/is holding it back XFS on EBS Vendor lock-in Slow snapshots make MySQL crash No send/recv Veritas Mature, stable Expensive! No send/recv Btrfs Immature, not production ready No send/recv
- 17. Worked example zfs create pool/bucket1 zfs snapshot pool/bucket1@time1 zfs send pool/bucket1@time1 |ssh node2 zfs recv pool/bucket1@time1 write some data... zfs snapshot pool/bucket1@time N zfs send -i time N-1 pool/bucket1@timeN |ssh node2 zfs recv pool/bucket1@time N Only sends the blocks of the filesystem which have changed from once instance to another! We do this all automatically, snapshotting up to every ten seconds only if writes occur
- 18. Clustering with ZFS replication Redundancy, failover and automatic recovery Horizontal scalability for read-heavy datasets Partition-tolerance Easily bring offline machines up-to-date ZFS runs on FreeBSD, Linux kernel modules coming in December 2010!
- 19. Thank you! Lots more information on our website: www.hybrid-cluster.com Follow us on Twitter: @hybridcluster Native ZFS on Linux, GA in December 2010 zfs.kqinfotech.com
- 21. Why is Cloud Integration critical to SaaS? - CloudCamp 20 October 2010 Mark Allman, Development, Cast Iron Integration Appliances IBM United Kingdom Ltd, Hursley Park
- 22. Why Integrate? Sales CRM Marketing Shipping Finance What are my hottest leads? ? Where are my orders? ? Has this customer paid? ? I need to do a credit check ? Where are my invoices? ?
- 23. Why Are Companies Leaving SaaS?
- 24. Economic Value of Integration for ISVs Integration = Faster Adoption and Lower Churn Churn Rates
- 25. Integration = Maximizing Cloud Application Value Hottest Leads Sales Invoices Orders CRM Marketing Shipping Finance
- 26. Solution Considerations Deployment Flexibility Re-usability Connectivity Project Requirements UI Mash-ups Process Integration Data Migration Templates Development Kit Community Virtual Appliances Physical Appliances Cloud Offering
- 27. Key Points Integration cost is a significant factor in the failure of many SaaS adoption projects The key to successful SaaS deployment is integrating existing resources, enabling a unified view Choice of the right integration vendor is critical!
Editor's Notes
- #24: The cost integration is prohibitive to the ongoing deployment of th eSaaS solution
- #25: What do you think is the average percentage of customer churn per month for a SaaS provider who does not provide integration capabilities? It is 12% per month What do you think is the average percentage of customer churn per month for SaaS providers who do provide integration capabilities? It is one 1% per month Customers who test SaaS, but find it doesn't satisfy their needs. Vendors who offer integration as part of their solution will reduce the level of customer churn (Sticky customers).
- #26: The maximum value is when all data sources can be integrated into one “view” The sale team on the road has access to all data without the need to switch screens
- #27: Solution needs to complement reasons for original SaaS adoption Connectivity. Not just industry leading SaaS and enterprise applications but also on-premise (database, custom applications, web services etc) Suitability project requirements. Data migration, process integration or even for UI mash-ups for taking relevant data from a back-office application and displaying it within a commonly used front-office application. Re-usability. Time to value. Can experience be leveraged? Is coding required? Are there common services (session management, connection pooling, discovery) and capabilities (logic, real-time and batch) Form-Factor. Consistent across form factors, allows for seamless migration. - Cloud. Configure, run and manage in the cloud. Nothing else required for off-premise to off-premise. Requires ability for secure on-premise connectivity - Physical. Solution in a box - Virtual. Utilise existing hardware