{"id":5012,"date":"2020-03-05T06:52:06","date_gmt":"2020-03-05T06:52:06","guid":{"rendered":"https:\/\/blog.verbat.com\/?p=2345"},"modified":"2024-05-23T07:24:19","modified_gmt":"2024-05-23T07:24:19","slug":"hadoop-ecosystem-beginners-overview","status":"publish","type":"post","link":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/","title":{"rendered":"Hadoop Ecosystem: A Beginner\u2019s Overview"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">\u201cThe number all of us\nhave to really pay attention to is, there will be 50 billion connected devices\nby 2025, that means we will have 50 billion ends points that we get to really\nharness. Then comes the data part, these devices are expected to generate 175\nzettabyte of data, a quadrupled growth, from the current 45 zettabyte,\u201d- Satya\nNadella<\/p>\n\n\n\n<!--more-->\n\n\n\n<p class=\"wp-block-paragraph\">Earlier with limited\ndata, all it required was a processor and one storage unit. But with millions\nof device generating massive amounts, data cannot be stored processed and\nanalysed using traditional ways. Moreover, these data poses another set of\nchallenges like the velocity in which it is created, the variety of data\ncreated (excel, jpeg, streaming data etc.) value of data created and so on.<\/p>\n\n\n\n<h4 class=\"wp-block-heading\">Hadoop as a Solution<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Hadoop is an<a href=\"https:\/\/www.verbat.com\/software-development\"> <strong>open source software<\/strong><\/a> framework that manages data storage in a distributed way and processes it parallelly on commodity hardware.<\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg\" alt=\"\" class=\"wp-image-2346\"\/><\/figure>\n<\/div>\n\n\n<ol class=\"wp-block-list\"><li><strong>Hadoop HDFS<\/strong><\/li>\n\n\n\n<p class=\"wp-block-paragraph\">Hadoop\nDistributed File System (HDFS) is a distributed file system that allows to\nstore huge data sets across cluster nodes or multiple machines. It follows a\ndistributed File system architecture or master-slave topology.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It\nhas 2 core components<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One\nthe Namenode which manages and maintains the Datanode. It also records the metadata.\nThe other is the Datanode which stores the actual data, does reading, writing\nand processing. It performs replication as well. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Each\nfile is stored in HDFS as blocks. Each block has a default size of 128MB in\nApache Hadoop.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">&nbsp;<strong>Features<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Provides distributed storage<\/li>\n\n\n\n<li>Implemented on Commodity<br>hardware<\/li>\n\n\n\n<li>Provides Data Security<\/li>\n\n\n\n<li>Highly Fault Tolerant<\/li>\n\n\n\n<li>HDFS also provides streaming<br>access to file system data <\/li>\n\n\n\n<li>Provides file permission and<br>authentication<\/li>\n\n\n\n<li>HDFS uses a command line<br>interface to interact with Hadoop<\/li>\n<\/ul>\n\n\n\n<li><strong>Hadoop YARN&nbsp; (Yet Another Resource Negotiator)<\/strong><\/li>\n\n\n\n<p class=\"wp-block-paragraph\">YARN is a cluster resource management layer of Hadoop\nwhich responsible for managing cluster resources like RAM, memory and other\nresources. It acts like an operating system scheduling jobs and allocating\nresources.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A number of frameworks can be run on top of YARN like\nHadoop MapReduce, Tez, Apache HBase, Storm, Spark etc.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It has two components one the Resource Manager (Master)\nand two Node Manager (Slave).<\/p>\n\n\n\n<li><strong>Hadoop MapReduce<\/strong><\/li>\n\n\n\n<p class=\"wp-block-paragraph\">Hadoop Map Reduce\/ Processing unit of Hadoop.<br>Programming technique where huge amount of data is processed and stored in a parallel and distributed fashion.<br>In map reduce approach, processing is done at the slave nodes and final result is sent to master node.<\/p><\/ol>\n\n\n\n<h4 class=\"wp-block-heading\">Hadoop Ecosystem<\/h4>\n\n\n\n<p class=\"wp-block-paragraph\">Hadoop alone cannot provide all the facilities and all the processing of big data on its own. Hadoop Ecosystem, is a collection of additional software packages that can be installed on top of or alongside Hadoop for various tasks.  <\/p>\n\n\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img decoding=\"async\" src=\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-Ecosystem-Components-1024x576.jpg\" alt=\"\" class=\"wp-image-2347\"\/><\/figure>\n<\/div>\n\n\n<p class=\"wp-block-paragraph\">There are four steps for BigData processing<br> Step 1: Data Collection and Ingestion<br> Step 2: Data Processing<br> Step 3: Data Analysis<br> Step 4: Access <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Let\u2019s examine how each component help in the\nabove steps.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step\n1: Data Collection and Storage<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Data of various types and sources like\nrelational databases, systems or local files are ingested into Hadoop.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Sqoop<\/strong>\nis a tool designed to transfer data (import and export ) between Hadoop and external\ndata storage systems like relational database servers (MSSQL Server and MySQL).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Its features include parallel import\/export,\nimport results of SQL Query, full and incremental load and Kerbos security\nintegration.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Sqoop uses YARN framework to import\/export\ndata. This provides fault tolerance on top of parallelism. It also has connectors\nfor all major RDBMS databases.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Flume\n<\/strong>is a distributed servicethat collects event data such as that collects event data and\ntransfers it to HDFS. It is ideally suited for event data from multiple systems<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Apache\nHBase <\/strong>is a column oriented database management\nsystem derived from Google\u2019s NoSQL database BigTable that runs on top of HDFS.\nIt stores data in HDFS. It is mainly used when you need random, real time,\nread\/write access to your BigData. It is horizontally scalable and hence can provide\nsupport to high volume of data and high throughput.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">HBase is written in NoSQL\ndatabase written in Java which performs faster querying. It is also well suited\nfor sparse data sets.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step\n2: Data Processing<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The framework\u2019s like Spark and MapReduce\nperform the data processing. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Spark\n<\/strong>is an open source cluster computing framework\nto store and process data in real time across various clusters. It provides 100\ntimes performance for a few applications with in memory primitives as compared\nto the two stage disk based MapReduce.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Spark can run in the Hadoop cluster and\nprocesses data in HDFS. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It also supports wide variety of workload\nwhich includes machine learning business intelligence streaming and batch\nprocessing across various clusters.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 3: Data Analysis<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">After data is processed it is analysed .This can\nbe done by an open source high level data flow system like Pig and Impala.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Pig<\/strong>\nis mainly used for analytics. It is a scripting platform designed to process\nand analyze large data sets and it runs on Hadoop clusters. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Pig converts pig scripts into map and reduce\ncodes thus saving the programmer from writing complex map reduce programs. Ad-hoc\nqueries like Join and filter which are difficult to perform can be done easily\nusing pig.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Impala\n<\/strong>is an open source SQL engine that run on Hadoop\ncluster. It can be used to run a query, evaluate the results immediately, and\nfine tune the query.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Impala is ideal for interactive analysis. It\ncan analyze Hadoop data via SQL and other business intelligence tools. It has\nvery low latency and can be measured in milliseconds.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Hive<\/strong>\nis another framework used for data analysis in Hadoop. It is a data warehouse\nsystem which is used for querying and analyzing large data sets in HDFS. It executes\nqueries using MapReduce. However user need not write any code in low level map\nreduce<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Hive acts as an abstraction layer on top of\nHadoop. It is usually preferred for data processing and extract transform load\noperations (ETL)<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Step 4: Data\nExploration \/ Search <\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Search of data can be done using frameworks\nlike Cloudera search and Hue\/Hadoop User experience. <\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Cloudera\nSearch<\/strong> requires no technical or programming skill\nbecause it provides a full text interface for searching. It uses flexible\nscalable and&nbsp;&nbsp; robust storage system\n.This eliminates the need to move large data sets across infrastructures to\naddress business tasks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Hue <\/strong>is\nan acronym for Hadoop User experience<strong>. <\/strong>It\nis an open source web interface for analyzing data with Hadoop<strong>. <\/strong>You can use hue for the following\noperations<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Query<br>a table in Hive and Impala<\/li>\n\n\n\n<li>Run<br>spark and pig jobs and workflows<\/li>\n\n\n\n<li>Search<br>data<\/li>\n<\/ol>\n\n\n\n<h4 class=\"wp-block-heading\"><strong>Other Components<\/strong><\/h4>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Oozie<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Hadoop jobs such as map reduce pig hive and\nscoop uses work flows. <strong>Oozie <\/strong>is a\nworkflow or coordination system that you can use to manage Hadoop jobs. There\nare also other components like Apache Ambari, Apache Mahout Etc. Read more\nabout Oozie on <\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Another workflow coordination system on\nBigData system like Oozie is the<strong> Zookeeper<\/strong>.\nOriginally developed by Yahoo it is an open source coordination service for\ndistributed applications. It is used for keeping a log on configuration\ninformation, naming, ensuring distributed synchronization and group services. The\nprincipal features of Zookeeper include reliability, scalability and fast\nprocessing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The above article only deals with the basic of\nHadoop ecosystem. It is very robust system and has grown over the years with\nmany components added over the years. Based on the use cases, we can\nchoose a set of services from Hadoop Ecosystem and create a tailored solution\nfor an organization.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Resources:<\/strong><br><br><strong>Zookeeper:<\/strong><span> <a href=\"https:\/\/zookeeper.apache.org\/\">https:\/\/zookeeper.apache.org\/<\/a><\/span><br><strong>Oozie:<\/strong><span> <a href=\"https:\/\/oozie.apache.org\/\">https:\/\/oozie.apache.org\/<\/a><\/span><br><strong>Hue:<\/strong><span> <a href=\"https:\/\/gethue.com\/\">https:\/\/gethue.com\/<\/a><\/span><br><strong>Hadoop Map Reduce:<\/strong><span> <a href=\"https:\/\/www.google.com\/search?q=hadoop+mapreduce&amp;rlz=1C1GCEU_enIN870IN870&amp;oq=Hadoop+MapReduce&amp;aqs=chrome.0.0l7j69i61.4274j0j9&amp;sourceid=chrome&amp;ie=UTF-8\">https:\/\/www.google.com\/search?q=hadoop+mapreduce&amp;rlz=1C1GCEU_enIN870IN870&amp;oq=Hadoop+MapReduce&amp;aqs=chrome.0.0l7j69i61.4274j0j9&amp;sourceid=chrome&amp;ie=UTF-8<\/a><\/span><br><strong>Apache HBase:<\/strong><span> <a href=\"https:\/\/hbase.apache.org\/\">https:\/\/hbase.apache.org\/<\/a><\/span><br><strong>Sqoop:<\/strong><span> <a href=\"https:\/\/sqoop.apache.org\/\">https:\/\/sqoop.apache.org\/<\/a><\/span><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.verbat.com\/\">it companies in dubai\u00a0<\/a>|<a href=\"https:\/\/www.verbat.com\/search-engine-optimization\/search-engine-optimization-dubai\">\u00a0Search Engine Optimization UAE<\/a>\u00a0|\u00a0<a href=\"https:\/\/www.verbat.com\/technologies\/mobile-app-development\">mobile app development companies uae\u00a0<\/a>|\u00a0<a href=\"https:\/\/www.verbat.com\/web-hosting\">Web Hosting UAE\u00a0<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u201cThe number all of us have to really pay attention to is, there will be 50 billion connected devices by 2025, that means we will have 50 billion ends points that we get to really harness. Then comes the data part, these devices are expected to generate 175 zettabyte of data, a quadrupled growth, from [&hellip;]<\/p>\n","protected":false},"author":18,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-5012","post","type-post","status-publish","format-standard","hentry","category-others"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v22.8 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Hadoop Ecosystem: A Beginner\u2019s Overview - Verbat<\/title>\n<meta name=\"description\" content=\"Get a beginner&#039;s overview of the Hadoop ecosystem. Understand its components and how they work together in big data processing.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Hadoop Ecosystem: A Beginner\u2019s Overview - Verbat\" \/>\n<meta property=\"og:description\" content=\"Get a beginner&#039;s overview of the Hadoop ecosystem. Understand its components and how they work together in big data processing.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\" \/>\n<meta property=\"og:site_name\" content=\"Software Development Company Dubai UAE - Verbat Technologies\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/verbatltd\" \/>\n<meta property=\"article:published_time\" content=\"2020-03-05T06:52:06+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-05-23T07:24:19+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@verbatltd\" \/>\n<meta name=\"twitter:site\" content=\"@verbatltd\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\"},\"author\":{\"name\":\"\",\"@id\":\"\"},\"headline\":\"Hadoop Ecosystem: A Beginner\u2019s Overview\",\"datePublished\":\"2020-03-05T06:52:06+00:00\",\"dateModified\":\"2024-05-23T07:24:19+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\"},\"wordCount\":1307,\"publisher\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg\",\"articleSection\":[\"Others\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\",\"url\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\",\"name\":\"Hadoop Ecosystem: A Beginner\u2019s Overview - Verbat\",\"isPartOf\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg\",\"datePublished\":\"2020-03-05T06:52:06+00:00\",\"dateModified\":\"2024-05-23T07:24:19+00:00\",\"description\":\"Get a beginner's overview of the Hadoop ecosystem. Understand its components and how they work together in big data processing.\",\"breadcrumb\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage\",\"url\":\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg\",\"contentUrl\":\"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.verbat.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Hadoop Ecosystem: A Beginner\u2019s Overview\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.verbat.com\/blog\/#website\",\"url\":\"https:\/\/www.verbat.com\/blog\/\",\"name\":\"Verbat Technologies\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.verbat.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.verbat.com\/blog\/#organization\",\"name\":\"Verbat Technologies\",\"url\":\"https:\/\/www.verbat.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg\",\"contentUrl\":\"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg\",\"width\":200,\"height\":200,\"caption\":\"Verbat Technologies\"},\"image\":{\"@id\":\"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/verbatltd\",\"https:\/\/x.com\/verbatltd\",\"https:\/\/www.linkedin.com\/company\/verbatltd\"]},{\"@type\":\"Person\",\"@id\":\"\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Hadoop Ecosystem: A Beginner\u2019s Overview - Verbat","description":"Get a beginner's overview of the Hadoop ecosystem. Understand its components and how they work together in big data processing.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/","og_locale":"en_US","og_type":"article","og_title":"Hadoop Ecosystem: A Beginner\u2019s Overview - Verbat","og_description":"Get a beginner's overview of the Hadoop ecosystem. Understand its components and how they work together in big data processing.","og_url":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/","og_site_name":"Software Development Company Dubai UAE - Verbat Technologies","article_publisher":"https:\/\/www.facebook.com\/verbatltd","article_published_time":"2020-03-05T06:52:06+00:00","article_modified_time":"2024-05-23T07:24:19+00:00","og_image":[{"url":"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg"}],"twitter_card":"summary_large_image","twitter_creator":"@verbatltd","twitter_site":"@verbatltd","twitter_misc":{"Written by":"","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#article","isPartOf":{"@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/"},"author":{"name":"","@id":""},"headline":"Hadoop Ecosystem: A Beginner\u2019s Overview","datePublished":"2020-03-05T06:52:06+00:00","dateModified":"2024-05-23T07:24:19+00:00","mainEntityOfPage":{"@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/"},"wordCount":1307,"publisher":{"@id":"https:\/\/www.verbat.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg","articleSection":["Others"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/","url":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/","name":"Hadoop Ecosystem: A Beginner\u2019s Overview - Verbat","isPartOf":{"@id":"https:\/\/www.verbat.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage"},"image":{"@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage"},"thumbnailUrl":"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg","datePublished":"2020-03-05T06:52:06+00:00","dateModified":"2024-05-23T07:24:19+00:00","description":"Get a beginner's overview of the Hadoop ecosystem. Understand its components and how they work together in big data processing.","breadcrumb":{"@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#primaryimage","url":"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg","contentUrl":"https:\/\/blog.verbat.com\/wp-content\/uploads\/2020\/03\/Hadoop-components-1024x534.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/www.verbat.com\/blog\/hadoop-ecosystem-beginners-overview\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.verbat.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Hadoop Ecosystem: A Beginner\u2019s Overview"}]},{"@type":"WebSite","@id":"https:\/\/www.verbat.com\/blog\/#website","url":"https:\/\/www.verbat.com\/blog\/","name":"Verbat Technologies","description":"","publisher":{"@id":"https:\/\/www.verbat.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.verbat.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.verbat.com\/blog\/#organization","name":"Verbat Technologies","url":"https:\/\/www.verbat.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg","contentUrl":"https:\/\/www.verbat.com\/blog\/wp-content\/uploads\/2024\/04\/verbatltd_logo.jpg","width":200,"height":200,"caption":"Verbat Technologies"},"image":{"@id":"https:\/\/www.verbat.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/verbatltd","https:\/\/x.com\/verbatltd","https:\/\/www.linkedin.com\/company\/verbatltd"]},{"@type":"Person","@id":""}]}},"_links":{"self":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts\/5012","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/comments?post=5012"}],"version-history":[{"count":1,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts\/5012\/revisions"}],"predecessor-version":[{"id":5611,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/posts\/5012\/revisions\/5611"}],"wp:attachment":[{"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/media?parent=5012"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/categories?post=5012"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.verbat.com\/blog\/wp-json\/wp\/v2\/tags?post=5012"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}