Every day, every minute, millions of pictures videos and other forms of data are being dumped on to the internet via websites like Facebook, you tube etc. Ever wondered where this data is being stored to be used effectively year after year? The growing number of data sources like social media are challenging the big data technologies. Being the latest sensation, media giants like Google, Facebook and Yahoo have decided to choose Hadoop for their data management predicaments.
Any enterprise wishing to leverage its data and analytics is advised to install Hadoop framework; open source software that allows processing of large data over clusters of computers.
History of Hadoop
Hadoop was created back in 2005 by computer scientists Doug Reed Cutting and Mike Cafarella. Hadoop was named by Doug after his son's stuffed toy elephant and is now being managed by Apache Software Foundation. In 2006 Dough joined Yahoo! which dedicated a team to develop Hadoop. By 2008, Hadoop was being used by other companies beside Yahoo! like Facebook, New York Times and Last.fm.
The Hadoop architecture is made up of the Hadoop Common, Hadoop distributed file system (HDFS) and a MapReduce engine. MapReduce and HDFS are designed to handle any node failures. The architecture distributes data into chunks across many servers for the programmers to easily analyze and visualize easily.
Demand for Hadoop
The market for Hadoop is projected to rise from a $1.5 billion in 2012 to an estimated $16.1 billion by 2020 as per report by Allied Market Research. The profits are predicted to be made by the Commercial Hadoop companies like Amazon Web Services, Cloudera, Hortonworks etc.
The reason for the success for this platform is its low cost implementation which helps companies to adopt this technology more conveniently. It is also adept at automatically handling node failures and data replications and does all the hard work.
It is clear that data management industry has expanded from software and web into retail, hospitals, government etc. This creates a huge demand for scalable and cost effective platforms of data storage like Hadoop. Hence it comes as a no surprise that a skill in Hadoop is most desired as of now. The future for data storage is endless, as it is highly unlikely that the companies will stop storing their data or find an alternative to do so anytime soon.
Training in Hadoop basics is sure to go long way and will pay off in the long run as companies are willing to offer competitive salaries for candidates with desired skill-sets. Banking on this demand will definitely prove beneficial.
