Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop Tutorial | Simplilearn
8 likes2,665 views
This presentation about Hadoop for beginners will help you understand what is Hadoop, why Hadoop, what is Hadoop HDFS, Hadoop MapReduce, Hadoop YARN, a use case of Hadoop and finally a demo on HDFS (Hadoop Distributed File System), MapReduce and YARN. Big Data is a massive amount of data which cannot be stored, processed, and analyzed using traditional systems. To overcome this problem, we use Hadoop. Hadoop is a framework which stores and handles Big Data in a distributed and parallel fashion. Hadoop overcomes the challenges of Big Data. Hadoop has three components HDFS, MapReduce, and YARN. HDFS is the storage unit of Hadoop, MapReduce is its processing unit, and YARN is the resource management unit of Hadoop. In this video, we will look into these units individually and also see a demo on each of these units. Below topics are explained in this Hadoop presentation: 1. What is Hadoop 2. Why Hadoop 3. Big Data generation 4. Hadoop HDFS 5. Hadoop MapReduce 6. Hadoop YARN 7. Use of Hadoop 8. Demo on HDFS, MapReduce and YARN What is this Big Data Hadoop training course about? The Big Data Hadoop and Spark developer course have been designed to impart an in-depth knowledge of Big Data processing using Hadoop and Spark. The course is packed with real-life projects and case studies to be executed in the CloudLab. What are the course objectives? This course will enable you to: 1. Understand the different components of the Hadoop ecosystem such as Hadoop 2.7, Yarn, MapReduce, Pig, Hive, Impala, HBase, Sqoop, Flume, and Apache Spark 2. Understand Hadoop Distributed File System (HDFS) and YARN as well as their architecture, and learn how to work with them for storage and resource management 3. Understand MapReduce and its characteristics, and assimilate some advanced MapReduce concepts 4. Get an overview of Sqoop and Flume and describe how to ingest data using them 5. Create database and tables in Hive and Impala, understand HBase, and use Hive and Impala for partitioning 6. Understand different types of file formats, Avro Schema, using Arvo with Hive, and Sqoop and Schema evolution 7. Understand Flume, Flume architecture, sources, flume sinks, channels, and flume configurations 8. Understand HBase, its architecture, data storage, and working with HBase. You will also understand the difference between HBase and RDBMS 9. Gain a working knowledge of Pig and its components 10. Do functional programming in Spark 11. Understand resilient distribution datasets (RDD) in detail 12. Implement and build Spark applications 13. Gain an in-depth understanding of parallel processing in Spark and Spark RDD optimization techniques 14. Understand the common use-cases of Spark and the various interactive algorithms 15. Learn Spark SQL, creating, transforming, and querying Data frames Learn more at https://www.simplilearn.com/big-data-and-analytics/big-data-and-hadoop-training
More Related Content
What's hot (20)
Similar to Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop Tutorial | Simplilearn (20)
More from Simplilearn (20)
Recently uploaded (20)
![GEOGRAPHY-Study Material [ Class 10th] .pdf](https://cdn.slidesharecdn.com/ss_thumbnails/geography-studymaterial-250613053858-319e1b53-thumbnail.jpg?width=560&fit=bounds)
Hadoop Tutorial For Beginners | Apache Hadoop Tutorial For Beginners | Hadoop Tutorial | Simplilearn
- 2. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism What’s in it for you?
- 3. What’s in it for you? Why Hadoop?
- 4. What’s in it for you? Why Hadoop? What is Hadoop?
- 5. What’s in it for you? Why Hadoop? What is Hadoop? Hadoop HDFS
- 6. What’s in it for you? Why Hadoop? What is Hadoop? Hadoop HDFSHadoop MapReduce
- 7. What’s in it for you? Why Hadoop? What is Hadoop? Hadoop HDFSHadoop MapReduce Hadoop YARN
- 8. What’s in it for you? Why Hadoop? What is Hadoop? Hadoop HDFSHadoop MapReduce Hadoop YARN Use case of Hadoop
- 9. What’s in it for you? Why Hadoop? What is Hadoop? Hadoop HDFSHadoop MapReduce Hadoop YARN Use case of Hadoop Demo on HDFS, MapReduce and YARN
- 10. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism Why Hadoop?
- 11. In a town far away..
- 12. Tim sells food grains in his shop
- 13. The customers were happy as Tim was very quick with the orders
- 14. Tim sensed a good demand for other products, so he thought of expanding his business
- 15. He started selling fruits, vegetables, meat, and dairy products in addition to food grains
- 16. But it wasn’t as easy as he expected it to be. The number of customers increased, and he was not able to cater to their needs on time
- 17. He had to look into assisting his customers with each of their orders and billing. It was too difficult for him to manage alone
- 18. To start delivering orders on time and to manage the customers’ demands, Tim hired 3 more people to work with him
- 19. Matt took care of the fruits and vegetable section. Luke handled the dairy and meat section. Ann was appointed as the cashier Matt Luke Ann Tim
- 20. However, this was still not a solution to Tim’s problem as there was not enough space in the shop for all the items Storage area
- 21. The storage was a bottleneck since storing and accessing became more and more difficult with increased supply and demand Storage area
- 22. Tim came up with an idea to overcome this issue. He decided to expand the storage area and distribute each category of product on different floors
- 23. Now, customers were happy, and after picking up their products from the respective sections, it was then billed
- 24. Now, customers were happy, and after picking up their products from the respective sections, it was then billed Now, let us compare this story to big data
- 25. Earlier, data was generated at a moderate rate, and all the data was structured in nature. One processor was enough to process all of it
- 26. With the increase in data generation, different types of data were generated at high speed. It became difficult for a single processor to process different types of data
- 27. Massive amount of different types of data which cannot be processed and stored using traditional databases is known as big data
- 28. To overcome this issue, multiple processors were used to process each type of data
- 29. But now the problem was that one storage system was accessed by all the processors and the storage became the bottleneck
- 30. Just like how Tim adopted the distributed approach, the storage system was also distributed and by doing so, the data was stored in individual databases
- 31. Just like how Tim adopted the distributed approach, the storage system was also distributed and by doing so, the data was stored in individual databases Through this story, we see the two approaches that are used by Hadoop that is HDFS and MapReduce
- 32. HDFS refers to the distributed storage space just like how Tim distributed the storage space amongst the various sections
- 33. Each person took care of a separate section and at the end the customers went to the cashier for the final billing, this sorted the process and made it easier. This is how Hadoop MapReduce works
- 34. This was a rough story of big data generation and why Hadoop is required. I will now explain in detail as to what Hadoop is
- 35. This sounds interesting. I would like to know more about Hadoop
- 36. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism What is Hadoop?
- 37. What is Hadoop? Hadoop is a framework which stores and processes big data in a distributed and parallel fashion
- 38. What is Hadoop? Hadoop is a framework which stores and processes big data in a distributed and parallel fashion BIG DATA
- 39. That sounds interesting, so how does Hadoop store and process all of this big data?
- 40. Hadoop has individual components, which are used for storing and processing big data
- 41. One day in an office.. HDFS MapReduce YARN Components of Hadoop The storage unit of Hadoop
- 42. One day in an office.. HDFS MapReduce YARN Components of Hadoop The storage unit of Hadoop The processing unit of Hadoop
- 43. One day in an office.. HDFS MapReduce YARN Components of Hadoop The storage unit of Hadoop The processing unit of Hadoop The resource management unit of Hadoop
- 44. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism Hadoop HDFS
- 45. What is HDFS? Each block of data is stored on multiple systems and by default has 128 MB of data Data Datanode Datanode Datanode Hadoop Distributed File System (HDFS) is known for its distributed storage method. It distributes the data amongst many computers. In addition to this, replication of data is also done to avoid loss of data
- 46. What is HDFS? Let us now see how 500 MB of data is stored in the traditional method
- 47. Let us now see how 500 MB of data is stored in the traditional method 500 MB data What is HDFS? Let us now see how 500 MB of data is stored in the traditional method
- 48. Let us now see how 500 MB of data is stored in the traditional method Here, the entire set of data is stored in one database. This overloads the database, and if it crashes, we lose all our data 500 MB data What is HDFS? Let us now see how 500 MB of data is stored in the traditional method
- 49. What is HDFS? Using Hadoop HDFS, this problem is taken care of as data is distributed amongst many systems
- 50. Using Hadoop HDFS, this problem is taken care of as data is distributed amongst many databases By doing so, a single database is not overloaded 500 MB data What is HDFS? . . . Using Hadoop HDFS, this problem is taken care of as data is distributed amongst many systems
- 51. Hadoop Distributed File System (HDFS) is specially designed for storing massive datasets in commodity hardware What is HDFS?
- 52. What is HDFS? HDFS has two main components that help with its storage NameNode DataNode Hadoop Distributed File System (HDFS) is specially designed for storing massive datasets in commodity hardware
- 53. What is HDFS? DataNode DataNode DataNode DataNode NameNode • NameNode is the master of the system • It stores all the metadata
- 54. NameNode What is HDFS? DataNode DataNode DataNode DataNode • NameNode is the master of the system • It stores all the metadata• DataNode is known as the slave node. There are multiple DataNodes • It performs the read/write operations and stores the actual data
- 55. What is HDFS? NameNode DataNode DataNode DataNode DataNode • NameNode manages all the DataNodes • The DataNodes send signals known as heartbeats to the NameNode. This signal gives the status of the DataNode
- 56. As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB What is HDFS?
- 57. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 58. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS File.txt 530 MB As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 59. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS File.txt 530 MB Block B Block DBlock C 128 MB 128 MB128 MB 128 MB Block A As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 60. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS File.txt 530 MB Block B Block D Block E 18 MB Block C 128 MB 128 MB128 MB 128 MB Block A As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 61. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS File.txt 530 MB Block B Block D Block E 18 MB Block C 128 MB 128 MB128 MB 128 MB Block A The final block uses only the remaining space for storage As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 62. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS File.txt 530 MB Block B Block D Block E 18 MB Block C 128 MB 128 MB128 MB 128 MB Block A DataNode 1 DataNode 2 DataNode 3 DataNode 4 DataNode 5 As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 63. What is HDFS? Now, let’s consider storing a file of size 530 MB in HDFS File.txt 530 MB Block B Block D Block E 18 MB Block C 128 MB 128 MB128 MB 128 MB Block A All these data blocks are stored in DataNodes – computers DataNode 1 DataNode 2 DataNode 3 DataNode 4 DataNode 5 As mentioned earlier, the actual data is stored in DataNodes. Data is stored in the form of blocks here. The default size of each block is 128 MB
- 64. What happens if the computer that contains block A crashes? Do we lose the data in block A?
- 65. No, we don’t. That’s the beauty of Hadoop HDFS. It uses replication to prevent the loss of data
- 66. c Rack 1 Replication in HDFS HDFS overcomes the issue of DataNode failure by creating copies of the data; this is known as the replication method Block ADN 1
- 67. c Rack 1 Rack 2 Replication in HDFS HDFS overcomes the issue of DataNode failure by creating copies of the data; this is known as the replication method Block ADN 1 DN 1 Block ADN 5 Block A is replicated. The replication factor is 3. The replicas are stored in different DataNodes Block ADN 6 2 replicas cannot be stored on the same datanode
- 68. c Rack 1 Rack 2 Replication in HDFS HDFS overcomes the issue of DataNode failure by creating copies of the data; this is known as the replication method Rack 3 Rack 4 Rack 5 Similarly, every other block is replicated Block ADN 1 Block DDN 2 DN 1 Block ADN 5 Block DDN 10Block BDN 4 Block CDN 7 Block CDN 11 Block EDN 13 Block DDN 14 DN 12Block ADN 6 Block BDN 8 Block BDN 9 Block CDN 15Block EDN 3 Block EDN 12
- 69. Architecture of HDFS Stores Metadata (Name, replicas, ….) NameNode
- 70. Stores DataNodes Metadata (Name, replicas, ….) DataNodes NameNode …..…. Architecture of HDFS
- 71. …..…. Stores DataNodes Metadata (Name, replicas, ….) DataNodes NameNode Rack is a collection of DataNodes Replication Rack 1 Rack 2 Architecture of HDFS
- 72. Metadata ops Stores Client DataNodes Metadata (Name, replicas, ….) DataNodes NameNode Read request? Replication …..…. Architecture of HDFS
- 73. Stores Client DataNodes Metadata (Name, replicas, ….) DataNodes NameNode Replication …..…. Architecture of HDFS Read request? Okay, read data from DataNodes Read permission
- 74. Stores DataNodes Metadata (Name, replicas, ….) DataNodes NameNode Here is the data that is read ReplicationRead data …..…. Architecture of HDFS Metadata ops Client Read request?
- 75. Metadata ops Stores Client DataNodes Metadata (Name, replicas, ….) DataNodes NameNode Write Write Client ReplicationRead data …..…. Architecture of HDFS
- 76. Features of HDFS HDFS is fault tolerant as multiple copies of data are made Fault tolerant Data security Scalability Flexibility
- 77. Features of HDFS Provides end-to-end encryption that protects data Fault tolerant Data security Scalability Flexibility
- 78. Features of HDFS Multiple nodes can be added to the cluster depending on the requirement Fault tolerant Data security Scalability Flexibility
- 79. Features of HDFS Hadoop is flexible in storing any type of data, like structured, semi structured or unstructured data Fault tolerant Data security Scalability Flexibility
- 80. Now that we have stored data in HDFS, how can we process it?
- 81. For processing data, Hadoop has a unit known as MapReduce
- 82. In the traditional approach, big data was processed at the master node Why MapReduce? big data
- 83. In the traditional approach, big data was processed at the master node Why MapReduce? Master Slave Slave Slave Slave big data
- 84. This was a disadvantage as it consumed more time to process various types of data Master Slave Slave Slave Slave Why MapReduce? big data
- 85. To overcome this issue, data was processed at each slave node. This approach is known as MapReduce Master Slave Slave Slave Slave Why MapReduce? big data
- 86. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism Hadoop MapReduce
- 87. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce
- 88. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce MapReduce tasks Map tasks Reduce tasks
- 89. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Map and Reduce steps Input Data Output Data map() map() map() Shuffle and Sort reduce() reduce() Input Data is divided to form the input splits
- 90. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Map and Reduce steps Input Data Output Data map() map() map() Shuffle and Sort reduce() reduce() Map phase is the first phase, here data in each split is passed to produce output values
- 91. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Map and Reduce steps Input Data Output Data map() map() map() Shuffle and Sort reduce() reduce() In the shuffle and sort phase, output of mapping phase is taken and similar data is grouped
- 92. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Map and Reduce steps Input Data Output Data map() map() map() Shuffle and Sort reduce() reduce() Here, the output values from the shuffling phase are aggregated. It then returns a single output value
- 93. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example
- 94. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example Input data Welcome to Hadoop Hadoop is interesting Hadoop is easy
- 95. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example Input data Welcome to Hadoop Hadoop is interesting Hadoop is easy Welcome to Hadoop Hadoop is interesting Hadoop is easy Input Splits
- 96. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example Input data Welcome to Hadoop Hadoop is interesting Hadoop is easy Welcome to Hadoop Hadoop is interesting Hadoop is easy Input Splits Hadoop, 1 is, 1 interesting, 1 Welcome, 1 to, 1 Hadoop, 1 Hadoop, 1 is, 1 easy, 1 Map phase
- 97. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example Map phase Shuffle and Sort phase Hadoop, 1 is, 1 interesting, 1 Welcome, 1 to, 1 Hadoop, 1 Hadoop, 1 is, 1 easy, 1 to, 1 Hadoop, 1 Hadoop, 1 Hadoop, 1 is, 1 is, 1 interesting, 1 Welcome, 1 easy, 1
- 98. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example Map phase Shuffle and Sort phase Hadoop, 1 is, 1 interesting, 1 Welcome, 1 to, 1 Hadoop, 1 Hadoop, 1 is, 1 easy, 1 to, 1 Hadoop, 1 Hadoop, 1 Hadoop, 1 is, 1 is, 1 interesting, 1 Welcome, 1 Reducer phase easy, 1easy, 1 Hadoop, 3 interesting, 1 is, 2 to, 1 Welcome, 1
- 99. What is MapReduce? Programming technique where huge data is processed in a parallel and distributed fashion is known as Hadoop MapReduce Let us now see how MapReduce works with an example Map phase Shuffle and Sort phase Final Output Hadoop, 1 is, 1 interesting, 1 Welcome, 1 to, 1 Hadoop, 1 Hadoop, 1 is, 1 easy, 1 to, 1 Hadoop, 1 Hadoop, 1 Hadoop, 1 is, 1 is, 1 interesting, 1 Welcome, 1 Reducer phase easy, 1 easy 1 Hadoop 3 interesting 1 is 2 to 1 Welcome 1 easy, 1 Hadoop, 3 interesting, 1 is, 2 to, 1 Welcome, 1
- 100. Features of MapReduce Good load balancing Re-execution of tasks Simple programming model Map task + Reduce task Splitting the stages into Map and Reduce tasks improves the load balancing
- 101. Features of MapReduce Good load balancing Re-execution of tasks Simple programming model There is an automatic re-execution if a certain task fails Map task + Reduce task
- 102. Features of MapReduce Good load balancing Re-execution of tasks Simple programming model MapReduce has one of the simplest programming model which is based on Java. Java is a very common programming language Map task + Reduce task
- 103. HDFS and MapReduce were the two units of Hadoop 1.0
- 104. Hadoop 1.0 was also known as MapReduce Version 1
- 105. The disadvantage with this version was that the Job tracker did both the processing of data and resource allocation
- 106. As a result, Job tracker was overburdened due to handling job scheduling, and resource management
- 107. To overcome this issue, Hadoop 2 introduced YARN as the processing layer that supported many frameworks
- 108. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism Hadoop YARN
- 109. What is YARN? Yet Another Resource Negotiator (YARN) acts as the resource management unit of Hadoop
- 110. What is YARN? Yet Another Resource Negotiator (YARN) acts as the resource management unit of Hadoop Apache YARN consists of Resource Manager It is the master daemon. Manages the assignment of resources such as CPU, memory
- 111. What is YARN? Yet Another Resource Negotiator (YARN) acts as the resource management unit of Hadoop Apache YARN consists of Resource Manager Node Manager It is the slave daemon. It reports the resource usage to the Resource Manager
- 112. What is YARN? Yet Another Resource Negotiator (YARN) acts as the resource management unit of Hadoop Apache YARN consists of Resource Manager Application Master Node Manager Works with the negotiation of resources from resource manager and works with node manager
- 116. What is YARN? Client Client Resource Manager Node Manager container Node Manager Node Manager container container container Container is a collection of physical resources such as CPU, RAM
- 117. What is YARN? Client Client Resource Manager Node Manager App Master App Master Node Manager Node Manager container container container container App Master requests container to Resource Manager. It uses container allocated by Node Manager
- 118. Node Manager App Master App Master Node Manager Node Manager container container container container What is YARN? Client Client Application Resource Manager Client program sends application request to the resource manager
- 119. What is YARN? Node Manager container App Master App Master container Node Manager Node Manager container container Client Client Resource Manager Node status Job request Node manager updates the status of the nodes to the resource manager
- 120. What is YARN? Node Manager container App Master App Master container Node Manager Node Manager container container Client Client Resource Manager Job request Resource Manager contacts the Node Manager requesting for resources(containers). The Node Manager grants the request
- 121. What is YARN? Node Manager container App Master App Master container Node Manager Node Manager container container Client Client Resource Manager Job request App Master contacts the Node Manager to use the container and runs in one of the container allocated on one of the nodes
- 122. Features of YARN Job scheduling Multitenancy YARN is responsible to process job requests and allocate resources Scalability
- 123. Features of YARN Job scheduling Multitenancy Different versions of MapReduce can run on YARN. This makes upgrading of MapReduce manageable Scalability
- 124. Features of YARN Job scheduling Multitenancy Scalability Depending on the requirement, the number of nodes can be increased
- 125. Many companies use Hadoop for storing and processing data. Now, let me tell you about one such company
- 126. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism Use case - Pinterest
- 127. You would have probably heard of the popular image sharing website Pinterest
- 129. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Pinterest is a social media platform which allows you to pin any interesting information you find on its site
- 130. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Pinterest has more than 250 million users and nearly 30 billion pins. All these account to big data concerning Pinterest
- 131. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Problem Pinterest faced a challenge in processing tremendous amount of data Pinterest has more than 250 million users and nearly 30 billion pins. All these account to big data concerning Pinterest
- 132. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Problem Pinterest faced a challenge in processing tremendous amount of data There was a difficulty in analyzing which data needs to be displayed in a user’s personalized discovery engine Pinterest has more than 250 million users and nearly 30 billion pins. All these account to big data concerning Pinterest
- 133. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Solution Pinterest has more than 250 million users and nearly 30 billion pins. All these account to big data concerning Pinterest
- 134. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Solution Pinterest uses Hadoop to process and analyze big data in a way that it helps the company to show the most relevant content to its users Pinterest has more than 250 million users and nearly 30 billion pins. All these account to big data concerning Pinterest
- 135. ` Pinterest is a social media platform which allows you to pin any interesting information you find on its site Solution Pinterest uses Hadoop to process and analyze big data in a way that it helps the company to show the most relevant content to its users Through continuous analysis of the data, Pinterest can provide its users with features such as related pins, guided search and so on Pinterest has more than 250 million users and nearly 30 billion pins. All these account to big data concerning Pinterest
- 136. This is how Pinterest benefited from Hadoop. Let’s also start using Hadoop to put an end to the big data challenges we are facing
- 137. What’s in it for you? Big Data Challenges What is HDFS? HDFS Cluster Architecture HDFS Data Blocks Data Node Failure Rack Awareness General Architecture of HDFS Read/Write Mechanism Demo on HDFS, MapReduce and YARN
- 138. Key Takeaways
Editor's Notes
- #3: Style - 01
- #4: Style - 01
- #5: Style - 01
- #6: Style - 01
- #7: Style - 01
- #8: Style - 01
- #9: Style - 01
- #10: Style - 01
- #11: Style - 01
- #37: Style - 01
- #38: Style - 01
- #39: Style - 01
- #45: Style - 01
- #46: Style - 01
- #47: Style - 01
- #48: Style - 01
- #49: Style - 01
- #50: Style - 01
- #51: Style - 01
- #52: Style - 01
- #53: Style - 01
- #54: Style - 01
- #55: Style - 01
- #56: Style - 01
- #57: Style - 01
- #58: Style - 01
- #59: Style - 01
- #60: Style - 01
- #61: Style - 01
- #62: Style - 01
- #63: Style - 01
- #64: Style - 01
- #67: Style - 01
- #68: Style - 01
- #69: Style - 01
- #70: Style - 01
- #71: Style - 01
- #72: Style - 01
- #73: Style - 01
- #74: Style - 01
- #75: Style - 01
- #76: Style - 01
- #77: Style - 01
- #78: Style - 01
- #79: Style - 01
- #80: Style - 01
- #83: Style - 01
- #84: Style - 01
- #85: Style - 01
- #86: Style - 01
- #87: Style - 01
- #88: Style - 01
- #89: Style - 01
- #90: Style - 01
- #91: Style - 01
- #92: Style - 01
- #93: Style - 01
- #94: Style - 01
- #95: Style - 01
- #96: Style - 01
- #97: Style - 01
- #98: Style - 01
- #99: Style - 01
- #100: Style - 01
- #101: Style - 01
- #102: Style - 01
- #103: Style - 01
- #109: Style - 01
- #110: Style - 01
- #111: Style - 01
- #112: Style - 01
- #113: Style - 01
- #114: Style - 01
- #115: Style - 01
- #116: Style - 01
- #117: Style - 01
- #118: Style - 01
- #119: Style - 01
- #120: Style - 01
- #121: Style - 01
- #122: Style - 01
- #123: Style - 01
- #124: Style - 01
- #125: Style - 01
- #127: Style - 01
- #129: Style - 01
- #130: Style - 01
- #131: Style - 01
- #132: Style - 01
- #133: Style - 01
- #134: Style - 01
- #135: Style - 01
- #136: Style - 01
- #138: Style - 01
- #139: Style - 01