Innovation Without Compromise: The Challenges of Securing Big Data
1 like1,554 views
The document discusses the significance of big data security, emphasizing the need for governance and protection of personal and confidential information within the context of the growing digital universe, particularly in the financial services sector. It highlights the opportunities and challenges organizations face by leveraging Hadoop and the Big Data Analytics (BDA) ecosystem, with an emphasis on the importance of hiring skilled personnel in security and governance roles. Additionally, it outlines Cloudera's comprehensive approach to Hadoop security, focusing on standards-based authentication, compliance readiness, and minimal performance impact.
Innovation Without Compromise: The Challenges of Securing Big Data
- 1. 1 Innovation without Compromise The Challenges of Securing Big Data
- 2. Presenters Carl Olofson Research Vice President, IDC Michael Versace Global Research Director, IDC Eddie Garcia Chief Security Architect, Cloudera Anil Earla Chief Data and Analytics Officer, Global Information Security, Visa 2 ©2014 Cloudera, Inc. All rights reserved.
- 3. The Third Platform Embracing the third platform is necessary to embrace the future of business. All this data, from a mix of sources, requires governance, including data quality and security. © IDC Visit us at IDC.com and follow us on Twitter: @IDC 3 Source: IDC
- 4. Hadoop and the Real Time Enterprise Real Time BI Composite Data Framework © IDC Visit us at IDC.com and follow us on Twitter: @IDC 4 4 CEP Engine Streaming Data Business Action Enterprise Analytics IMDB High Speed Transactions and Automated Decisioning Integrated Business Process Apps Data Warehouse Hadoop
- 5. Data Security Challenges What is it? • From unmanaged sources • Uncontrolled data Need to identify and protect • Personal identifying information • Other confidential data © IDC Visit us at IDC.com and follow us on Twitter: @IDC 5
- 6. The Financial Services, The Digital Universe Is Growing Faster Than Most Commercial Markets 176 exabytes 2,334 exabytes 46% annual growth 2013 2020 © IDC Visit us at IDC.com and follow us on Twitter: @IDC 6 In 2013, tablets representing the financial services portion of the DU – the financial services “Digital Galaxy” would be 40% of the volume of the Empire State Building By 2020, it would 5.1 times the volume of the Empire State Building At 46% CAGR, the financial services Digital Galaxy is growing faster than the overall DU * iPad Air – 9.4” high, 6.6” wide, 0.29” thick, 128 GB; volume of Empire State Building – 37 million cubic feet Source: IDC, 2014
- 7. At a Growing Rate, Firms Turn to Hadoop and the BDA Ecosystem, And Getting the Payoff 81% of organizations surveyed indicated “met expectations” or “exceeded expectations” as the most frequent benefit achieved from BDA initiatives in aggregate. © IDC Visit us at IDC.com and follow us on Twitter: @IDC 7 N = 84 Source: IDC's 2013 BDA Survey, July 2013
- 8. Where’s the Opportunity? In MI, Customer Acquisition, Pricing Services, and Risk Management © IDC Visit us at IDC.com and follow us on Twitter: @IDC 8 Cross Industry Comparison Commercial, Retail, Investment Banks Customer Acquisition and/or Retention 40.7 54.8 Customer Service and Support 46.8 47.6 Pricing Strategies and Programs 46.0 47.6 Product, Service, or Program Improvement and Innovation 48.6 54.8 Market and Competitive Intelligence 42.8 48.8 Process and Operations Optimization and Control 42.0 39.3 Plant, Facilities, and Equipment Design, Maintenance 19.4 11.9 Human Capital Management 27.6 29.8 Operational, Fraud, and Risk Management 24.5 38.1 Regulatory Compliance, Financial Controls 34.8 44.0 Supply Chain Management and Logistics 33.4 13.1 Information Technology Optimization and/or Modernization 40.0 38.1 Other 1.3 1.2 N=84 Source: IDC's 2013 BDA Survey
- 9. Security and Data Governance Lack Attention “Ethical Use”, Privacy, Right to be Forgotten, and Resources Constraints Over 50% of organizations surveyed indicated they have been less than successful in hiring, retraining, or rewarding big data and analytical staff involved in security and governance processes. © IDC Visit us at IDC.com and follow us on Twitter: @IDC 9 N = 84 Source: IDC's 2013 BDA Survey, July 2013
- 10. Security and Governance at Scale Scale-up and Scale-out to Workload Requirements Workload Class Business Drivers Metrics Gap Enablers Class 1 Development File & Print Small Data Marts Small-scale applications & DB servers Increased utilization Consolidation Lower unit costs Ease on demand Simplification – do more w/less Server Utilization VP Ratio (i.e. #VMs) Time-to-provision End-user Satisfaction Cost (TCO) Developer productivity Overhead & IO Performance Storage Management Backup window Pay as you go services Multi-tenant services Bandwidth CPU Performance Class 2 Medium-scale applications & DB servers Data Marts Small Data Warehouses Messaging (Exchange, etc) Greater flexibility to move workloads Shed non-core workloads Extend Class 1 benefits Increase asset utilization System availability Service management Storage admin productivity Backup efficiency RPO/RTO Overhead & IO Performance Problem Determination Initial component integration; De-facto support for Exchange and Oracle SaaS Chargebacks Class 3 Mission Critical Applications Larger scale OLTP Larger scale DB Servers Large r scale messaging Simpler application management Quicker time to market Simpler and better application integration Less expensive recovery Cost Change and new product deployment times (TTM) I/O Performance Enhanced Security & Compliance SLA Management Integrated Recovery Encapsulated workloads SLA management of high-performance systems Ability to design low RPO/RTO systems cost effectively Compliance data standard Security Ops Management Class 4 Large-scale applications requiring highest levels of availability, security & recoverability Quicker time to market Standard Infrastructure stack Deeper business integration Resiliency Cost Compliance Speed to deploy Trusted Integration High-speed recovery Proven Resilience “Certified” platforms “Cannot fail” fault tolerance Defensible Security
- 11. 11 Big Data Security Roundtable
- 12. Protecting Big Data Share what you can about your Hadoop environment, and tell us how the big data trend has changed the way Visa is thinking about protecting data and security in general? 12 ©2014 Cloudera, Inc. All rights reserved.
- 13. Using Big Data as a Security Enabler On the flip side, how are all the innovations around the big data trend helping Visa with cyber security perspective? 13 ©2014 Cloudera, Inc. All rights reserved.
- 14. When designing with Hadoop as a target technology, what were the biggest security and risk considerations? 14 ©2014 Cloudera, Inc. All rights reserved.
- 15. How did these and other design and implementation issues influence Cloudera's strategy for securing Hadoop as a big data platform? How are you seeing the adoption of Hadoop change the way practitioners think about security? 15 ©2014 Cloudera, Inc. All rights reserved.
- 16. What do you see as the greatest challenges to Hadoop security? You mentioned multiple access paths, Hive and Impala for example, as a challenge. Can you expand on that? 16 ©2014 Cloudera, Inc. All rights reserved.
- 17. What are some of the biggest challenges? For example, how does Cloudera approach the “Access Path” challenge that Carl referenced? 17 ©2014 Cloudera, Inc. All rights reserved.
- 18. Has Cloudera Security for Hadoop become part of Visa’s overall security operation – if so, how? 18 ©2014 Cloudera, Inc. All rights reserved.
- 19. Can you give us top down view of Cloudera's approach to Hadoop and security? 19 ©2014 Cloudera, Inc. All rights reserved.
- 20. What's in the future for big data security? What are some of the gaps in Hadoop security that Cloudera and the open source community are addressing? 20 ©2014 Cloudera, Inc. All rights reserved.
- 21. What advice do you have for your peers and others securing Hadoop? 21 ©2014 Cloudera, Inc. All rights reserved.
- 22. 22 The Future of Big Data Security Cloudera + Security
- 23. Our Approach to Hadoop Security Comprehensive • Standards-based Authentication • Centralized, Granular Authorization • Native Data Protection • End-to-End Data Audit and Lineage Compliance-Ready • Meet compliance requirements • HIPAA, PCI-DSS, … • Encryption and key management Transparent • Security at the core • Minimal performance impact • Compatible with new components • Insight with compliance 23 ©2014 Cloudera, Inc. All rights reserved.
- 24. CLOUDERA ENTERPRISE Comprehensive, Transparent, Compliance-Ready Security Batch Processing Analytic MPP SQL Search Engine Machine Learning Workload & Resource Management ©2014 Cloudera, Inc. All rights reserved. 24 Stream Processing End-to-End, Zero-Downtime System Administration 3rd Party Apps Distributed Filesystem Online NoSQL Database Perimeter Authentication Access Control Authorization Data Protection Encryption, Key Management Data Visibility Audit, Lineage Data Lifecycle BDR, Snapshots ANALYTIC & PROCESSING ENGINES SYSTEMS MANAGEMENT UNIFIED DATA STORAGE & INTEGRATION SECURITY & GOVERNANCE
- 25. Introducing the Cloudera Center for Security Excellence • Based in Austin, Texas • Comprehensive data and cluster security technologies • Hadoop security test and certification lab • Compliance cookbooks • Security ecosystem partner enablement • Intel chipset, cloud and virtualization security alignment 25 ©2014 Cloudera, Inc. All rights reserved.
- 26. 26 ©2014 Cloudera, Inc. All rights reserved. Questions?
Editor's Notes
- #7: EMC: what is CAGR? Remove ‘digital galaxy’ from everything, need branding of DU. Also why are we using the empire state building, can we instead include the other ones listed on slide 3 and in our 2014 report, see below: represented by memory in a stack of tablets, it would stretch two-thirds the way to the Moon* By 2020, it would be 6.6 stacks from the Earth to the moon*
- #11: % of budget
- #13: Anil to answer
- #14: Anil to answer
- #15: Anil to answer
- #16: Eddie to answer
- #17: Carl to answer IDC, from your perspective, as someone with a long term connection with database/data integration innovation, what do you see as the greatest challenges to Hadoop security – you mentioned earlier multiple access paths, Hive and Impala for example, as a challenge. Can you expand on that?
- #18: Eddie to answer
- #19: Anil to answer
- #20: Eddie answer
- #21: Eddie and Carl to answer What's in the future for big data security? What are some of the gaps in Hadoop security that Cloudera and the open source community are addressing? (Education, certification, SIEM integration, fine grained access control, hardened systems, etc.)
- #22: Anil to answer and Eddie and Carl to comment CTOs CIOs and Business IT leaders Hadoop ecosystem providers Security Professionals Standards developers