Summer Internship / Industrial Training 2016 for Engineering Students

Summer Internship

Linuxworld informatics pvt.ltd. invites students to spend the summer months working in their Summer Internship program. The summer internship program is dedicated to providing students with the opportunity to work beside some of today's most important computer science engineering various technologies namely ‪

BigData Hadoop, Cloud Computing, RedHat Linux, Cisco Networking, Python, OpenStack,  Docker, DevOps, Splunk, Ethical Hacking, Java, J Boss, PHP, Oracle and many more.

Students who are Pursing or completed B-tech, M.C.A. M.Sc, B.C.A, B.Sc  are welcome to apply for the summer internship program.

Summer Training/Internships are very important as far as an engineer's career is involved. Summer Internship is common for one particular reason: big summer holidays. Students learn a lot and gain industry exposure through trainee positions. Engineer's take their opportunity to learn, develop and apply known skills. Apart from these, there are a bunch of other benefits as well.

Benefits for Students
Chances to gain jobs increases
Real exposure of the field
Lots of practical experience gained
Excellent place to interact with professionals
Discovering & applying new techniques
Develops professional skills
Develops a work ethic standard
Boosts self-confidence
Employers prefer past-interns as employees

This is an exciting opportunity to directly work with the experienced leading Trainer. This is a wonderful opportunity for a student who is pursuing a degree or interested in Computer science Engineering. Make a difference this summer and lend a helping hand to the LinuxWorld Informatics pvt. ltd.

Hadoop turns 10, Big Data industry rolls along

Apache Hadoop, the open source project that arguably sparked the Big Data craze, turned 10 years old this week. The project's founder, Cloudera's Doug Cutting, waxed nostalgic as vendors in the space churned out new releases of their own.

It's hard to believe, but it's true. The Apache Hadoop project, the open source implementation of Google's File System (GFS) and MapReduce execution engine, turned 10 this week.

The technology, originally part of Apache Nutch, an even older open source project for Web crawling, was separated out into its own project in 2006, when a team at Yahoo was dispatched to accelerate its development.

Proud dad weighs inDoug Cutting, founder of both projects (as well as Apache Lucene), formerly of Yahoo, and presently Chief Architect at Cloudera, wrote a blog post commemorating the birthday of the project, named after his son's stuffed elephant toy.

In his post, Cutting correctly points out that "Traditional enterprise RDBMS software now has competition: open source, big data software." The database industry had been in real stasis for well over a decade. Hadoop and NoSQL changed that, and got the incumbent vendors off their duffs and back in the business of refreshing their products with major new features

Sleeping giants awaken
Microsoft SQL Server now supports columnstore indexes in order to handle analytic queries on large volumes of data and its upcoming 2016 version adds PolyBase functionality for integrated query of data in Hadoop. Meanwhile, Oracle and IBM have added their own Hadoop bridges, along with better handling of semi-structured data.

Teradata has pivoted rather sharply towards Hadoop and Big Data, starting with its acquisition of Aster Data and continuing through its multifaceted partnerships with Cloudera and Hortonworks. Meanwhile, in the Hadoop Era, perhaps in deference to Teradata, virtually every megavendor acquired one of the data warehousing pure plays.

New generationCutting points out, also accurately, that the original core components of Hadoop have been challenged and/or replaced: "New execution engines like Apache Spark and new storage systems like Apache Kudu (incubating) demonstrate that this software ecosystem evolves rapidly, with no central point of control." Granted, both of these projects are heavily championed by Cloudera, so take the commentary with a grain of salt.

Salt or no salt though, Cutting's comment that the Hadoop ecosystem has "no central point of control" is one worth considering carefully; because, while it is correct, it's not necessarily good. The term "creative destruction" sometimes truly is an oxymoron. The Big Data scene's rapid technology replacement cycles leave the space stability-challenged.

Give peace a chancePerhaps, but the moving technology target may also mean they get no software at all, because the current environment is sufficiently risk-prone as to hinder the growth of enterprise projects. We need some equilibrium if we want growth to be proportionate to the level of technological innovation.

Cutting concludes his post by declaring: "I look forward to following Hadoop's continued impact as the data century unfolds." While I'm not sure data and analytics will define the whole century, they probably have a good decade or two. Hopefully the industry can get a little better at developing standards that are cooperative and compatible, rather than overlapping and competitive. We don't want to go back to stasis, but more navigable terrain would suit the industry and its customers

Meanwhile, back in the competitive marketSpeaking of the industry, there were a slew of announcements this week, beside (and even despite) Hadoop's birthday.:

• Pentaho introduced Python language integration into its Data Integration Suite
• Paxata launched its new Winter '15 release (albeit in 2016), which includes new auto number and fill down transformations, new algorithms to aid its data prep recommendations, and integration with LDAP and SAML, for enterprise security, single sign-on and identity management
• SkyTree, a predictive analytics vendor, discussed that it will soon launch a free single-user version of its product, which it will soon announce more formally (and RapidMiner, also in the predictive space, released its new version 7 last week, with a revamped UI)
• NoSQL vendor Aerospike launched a new release of its eponymous database, which now features geospatial data support, added resiliency in cloud-hosted environments and server-side support for list and map data structures

Weekend pondering
That's a pretty busy week. And I dare say, without Hadoop as a catalyst, it would have been much less so. As climate change, financial markets, geopolitics and the price of oil reach frightening new levels of volatility, the data sector of the technology industry is thriving. We might hope that the technology around Big Data could be deployed to help solve, or at least better understand, some of our world's truly big problems.
This won't be the century of data unless that in fact happens

Article Source - http://www.zdnet.com/article/hadoop-turns-10-big-data-industry-rolls-along/

Hive Basic Understanding

Hive is Petabyte scale dataware house system on Hadoop.
Hadoop based system for querying & managing structured data.
Its used to Query Big Data in SQL fashion.

For Execution Hive uses - Map/Reduce
For Storage Hive uses    - HDFS
For Metadata                 - RDBMS



Origin of Hive -
Hive was designed by Facebook for querying from petabytes of data. There was sudden data explosion at Facebook which was impossible to store in traditional DBMS & query.

Hive made users job extremly esay to query data stored on HDFS.
Hive now became parallel DBMS which uses Hadoop for its storage & execution architecture.


Why Hive -
Hive is another dataware house system designed because existing Dataware house systems do not meet all the requirement in scalable , agile & cost effeciant way.

Programming model used in Hadoop is - MapReduce. Its very difficult to write Map-Reduce program for every small or big reports. Also it's requires highly skilled resources to write such a complex code.
Using Hive one can simply issue the query as simple & similar we do in SQL. But here Hive generates Map Reduce code for user based on Query issued.


Advantages of Hive -
Hive can work with very large data (100's to Terabytes).
Hive can work on large hadoop cluster (100's of Nodes).
Data stored on Hive has defined Schema.
Hive is used for Batch jobs also (Load & Query).


Where not to use Hive -
If you need responses in seconds.
If you don't want to impose a schema.
If traditional DBMS already can do the job.
If your data is measured in GB's or even less.
If you don't have enough time & highly skilled resources.



Hive Entities -
Database, Table, Partitions, Bucketing Columns.
MORE....



Hive Data Types -

Primitive Data Types
TINYINT          1 Byte Signed Integer
SMALLINT         2 Byte Signed Integer
INT              4 Byte Signed Integer
BIGINT           8 Byte Signed Integer
BOOLEAN          True or False (Boolean)
FLOAT            Single precision floating bytes
DOUBLE           Double precision floating point
STRING           Sequence of charaters (within Sigle or double quotes)
TIMESTAMP        java.sql.Timestamp format
etc...


Collection Data Types
STRUCT           Similar to Structure in C.
MAP              (Key,Value) pair
ARRAY            Ordered sequence of similar data types.



Hive operations -

DDL operations
[CREATE/ALTER/DROP] [TABLE/VIEW/PARTITION]
CREATE TABLE AS SELECT

DML operations
INSERT OVERWRITE

Queries...
Sub-Queries within "FROM" clause.
Joins  [Inner join & Outer (Left, Right & Full outer join)]
Multi-Table insert
Sampling

Interfaces
JDBC/ODBC/THRIFT

6 Reasons Why Java Developers Should Learn Hadoop

Imagine there are two girls standing in front of you - The first girl is cute, beautiful, interesting and has the smile that any guy would die for. And the other girl is average-looking, quiet, not-so-impressive... no different from the ones that you usually see in the restaurant cash counter. Which girl will you call out for a date? If you're like me, you will choose the attractive girl. You see, life is full of options and making the right choice is what matters the most.

If you're a Java developer, then you probably have more choices to make - like the switch from Java to Hadoop.

Big data and Hadoop are the two most popular buzzwords in the industry. Chances are that you have come across these two terms on the Java payscale forums or seen your senior colleagues making the switch to get bigger paychecks. I'll tell you what, the upgrade from Java to Hadoop is not just about staying updated with the latest technology or getting appraisals - it's about being competent and putting your career on the fifth gear.

The good news for all the aspiring Hadoop developers is that, the Big Data industry has already crossed the $50 billion dollar mark and over 64% of the top 720 companies worldwide are interesting to invest in this forward-thinking technology as revealed by Gartner in 2013.

If that's not convincing, then take a look at these stats:

1. According to an IDC report, the Big Data industry is growing at the rate of 31.7% per year.
2. Java developers are seen as the best replacement option for Hadoop developers, says Forrester.
3. Hadoop developers enjoy a mighty 250% pay hike than Java developers, as stated in an Analytics Industry Report.

What's special about Hadoop?

Unlike the traditional databases which weren't capable of dealing with large volumes of data, Hadoop offers the quickest, cheapest, and smartest way to store and process giant volumes of data - and that's the reason why it is so popular among big corporations, government organizations, hospitals, universities, financial services, marketing agencies, etc. The best way to familiarize with the language is to check out a beginner's big data hadoop course.

Okay, now let's some reasons why Java developers should switch to Hadoop.

1. Easy To Learn For Java Developers

A tennis player like Rafael Nadal loves clay courts because the surface suits him well and that's where he has been most successful. Similarly, any Java developer would love Hadoop because it's completely written in Java - a language that you are already so familiar with. Switching from Java to Hadoop is a cake-walk for professionals like you because the MapReduce script used in the Hadoop is actually written in Java itself. Awesome, isn't it?

Your Java skills will come in handy when debugging Hadoop apps and employing Pig (programming tool) Latin commands.

2. Helps You To Stay Ahead Of Your Competition

If you are a Java professional, you are just seen as a person in the crowd. But, if you are a Hadoop developer, you are seen as potential leader in the crowd. Big Data and Hadoop jobs are a hot deal in the market and Java professionals with the required skill set are easily picked by big companies for high salary packages. All you have to do is attend a big data hadoop  training program and learn the concepts from an expert.

3. Scope To Move Into Bigger Domains

Fortunately for you, the road doesn't end with Hadoop and MapReduce. There is always the golden opportunity to use your Hadoop skills and expertise to move into higher levels such as Artificial Intelligence, Data Science, Sensor Web data, and Machine Learning. These are emerging markets, and you'll see them dominate the industry in the next 4-5 years. Good knowledge in Big Data and Hadoop could boost your chances of getting into some of the bigger Big Data-dependent companies such as Amazon, Yahoo, Facebook, Twitter, IBM, and eBay.

4. Lucrative Packages For Hadoop Professionals

By switching from Java to Hadoop, you can expect a higher salary and better career prospects - the kind of salary and designation that your wife would like to rave about. According to Indeed, the average salary for a Big Data Hadoop developer with 1-2 years of experience is around $140,000 per annum in the United States. However, as you gain experience and become a senior Hadoop developer, you will be able to make a good $400,000+ salary.

5. An Improved Quality Of Work

Learning Big Data Hadoop can be highly beneficial because it will help you to deal with bigger, complex projects much easier and deliver better output than your colleagues. In order to be considered for appraisals, you need to be someone who can make a difference in the team, and that's what Hadoop lets you to be.

6. Grow With The Industry

With IDC predicting that the Big Data and Hadoop user base (big companies and government organizations) is likely to increase at 27% per year, you have a great opportunity to upgrade your knowledge and skills and grow with the industry.

Big Data and Hadoop are widely used in applications such as IT log analytics, Fraud detection, Social media analysis, and Call centre analytics - and learning a big data hadoop tutorial could be the way to kick-start your Hadoop career right away. Once you do that, you will find that staying updated with the latest technology will be a lot easier and getting into top organizations will never be 'just a dream' - it will be a reality.

That's about it, folks! These are some rock-solid reasons why learning Hadoop is important and how it can help take your career to the next level

Article Source: http://EzineArticles.com/9189164

Data Lake Showdown: Object Store or HDFS?

The explosion of data is causing people to rethink their long-term storage strategies. Most agree that distributed systems, one way or another, will be involved. But when it comes down to picking the distributed system–be it a file-based system like HDFS or an object-based file store such as Amazon S3–the agreement ends and the debate begins.

The Hadoop Distributed File System (HDFS) has emerged as a top contender for building a data lake. The scalability, reliability, and cost-effectiveness of Hadoop make it a good place to land data before you know exactly what value it holds. Combine that with the ecosystem growing around Hadoop and the rich tapestry of analytic tools that are available, and it’s not hard to see why many organizations are looking at Hadoop as a long-term answer for their big data storage and processing needs.

At the other end of the spectrum are today’s modern object storage systems, which can also scale out on commodity hardware and deliver storage costs measured in the cents-per-gigabyte range. Many large Web-scale companies, including Amazon, Google, and Facebook, use object stores to give them certain advantages when it comes to efficiently storing petabytes of unstructured data measuring in the trillions of objects.

But where do you use HDFS and where do you use object stores? In what situations will one approach be better than the other? We’ll try to break this down for you a little and show the benefits touted by both.

Bigdata Hadoop Training in Jaipur

Why You Should Use Object-Based Storage

According to the folks at Storiant, a provider of object-based storage software, object stores are gaining ground among large companies in highly regulated industries that need greater assurances that no data will be lost.

“They’re looking at Hadoop to analyze the data, but they’re not looking at it as a way to store it long term,” says John Hogan, Storiant’s vice president of engineering and product management. “Hadoop is designed to pour through a large data set that you’ve spread out across a lot of compute. But it doesn’t have the reliability, compliance, and power attributes that make it appropriate to store it in the data lake for the long term.”

Object-based storage systems such as Storiant’s offer superior long-term data storage reliability compared to Hadoop for several reasons, Hogan says. For starters, they use a type of algorithm called erasure encoding that spreads the data out across any number of commodity disks. Object stores like Storiant’s also build spare drives into their architectures to handle unexpected drive failures, and rely on the erasure encoding to automatically rebuild the data volumes upon failure.

If you use Hadoop’s default setting, everything is stored three times, which delivers five 9s of reliability, which used to be the gold standard for enterprise computing. Hortonworks architect Arun Murthy, who helped develop Hadoop while at Yahoo, pointed out at the recent Hadoop Summit that if you only storing everything twice in HDFS, that it takes one 9 off the reliability, giving you four 9s. That certainly sounds good. Source

Step by Step learning guide for Hadoop

Just as I was frustrated and disappointed with the training I attended with Bigdata training academny in chennai, I decided to publish best sites and reference materials for Hadoop that I come across.

Atleast this way I can be of some help for the "to be"  Hadoop aspirants and professionals so that won't waste their money in cheap institutes like the Bigdata training 

I aim my blog to be one stop shop for learning Bigdata Apache Hadoop, PIG and Hbase ,,,

Also as and when time permits , I will also create tuorials for hadoop, pig and hbase and publish them.

First at the beginner level and then to advaned level

The best site is cloudera.com for all the beginners of Apache Bigdata Hadoop and its ecosystem . Go and visit this URL http://university.cloudera.com/onlineresources.html