Big data has a lot of potential to benefit organizations in any
industry, everywhere across the globe. Big data is much more than just a
lot of data and especially combining different data sets will provide
organizations with real insights that can be used in the decision-making
and to improve the financial position of an organization. Before we can
understand how big data can help your organization, let's see what big
data actually is:
It is generally accepted that big data can be
explained according to three V's: Velocity, Variety and Volume. However,
I would like to add a few more V's to better explain the impact and
implications of a well thought through big data strategy.
Velocity
The
Velocity is the speed at which data is created, stored, analyzed and
visualized. In the past, when batch processing was common practice, it
was normal to receive an update to the database every night or even
every week. Computers and servers required substantial time to process
the data and update the databases. In the big data era, data is created
in real-time or near real-time. With the availability of Internet
connected devices, wireless or wired, machines and devices can pass-on
their data the moment it is created.
The speed at which data is
created currently is almost unimaginable: Every minute we upload 100
hours of video on YouTube. In addition, over 200 million emails are sent
every minute, around 20 million photos are viewed and 30.000 uploaded
on Flickr, almost 300.000 tweets are sent and almost 2,5 million queries
on Google are performed.
The challenge organizations have is to cope with the enormous speed the data is created and use it in real-time.
Variety
In
the past, all data that was created was structured data, it neatly
fitted in columns and rows but those days are over. Nowadays, 90% of the
data that is generated by organization is unstructured data. Data today
comes in many different formats: structured data, semi-structured data,
unstructured data and even complex structured data. The wide variety of
data requires a different approach as well as different techniques to
store all raw data.
There are many different types of data and
each of those types of data require different types of analyses or
different tools to use. Social media like Facebook posts or Tweets can
give different insights, such as sentiment analysis on your brand, while
sensory data will give you information about how a product is used and
what the mistakes are.
Volume
90% of all
data ever created, was created in the past 2 years. From now on, the
amount of data in the world will double every two years. By 2020, we
will have 50 times the amount of data as that we had in 2011. The sheer
volume of the data is enormous and a very large contributor to the ever
expanding digital universe is the Internet of Things with sensors all
over the world in all devices creating data every second.
If we
look at airplanes they generate approximately 2,5 billion Terabyte of
data each year from the sensors installed in the engines. Also the
agricultural industry generates massive amounts of data with sensors
installed in tractors. John Deere for example uses sensor
Big data hadoop Training in Jaipur to
monitor machine optimization, control the growing fleet of farming
machines and help farmers make better decisions. Shell uses
super-sensitive sensors to find additional oil in wells and if they
install these sensors at all 10.000 wells they will collect
approximately 10 Exabyte of data annually. That again is absolutely
nothing if we compare it to the Square Kilometer Array Telescope that
will generate 1 Exabyte of data per day.
In the past, the creation
of so much data would have caused serious problems. Nowadays, with
decreasing storage costs, better storage options like Hadoop and the
algorithms to create meaning from all that data this is not a problem at
all.
Veracity
Having a lot of data in
different volumes coming in at high speed is worthless if that data is
incorrect. Incorrect data can cause a lot of problems for organizations
as well as for consumers. Therefore, organizations need to ensure that
the data is correct as well as the analyses performed on the data are
correct. Especially in automated decision-making, where no human is
involved anymore, you need to be sure that both the data and the
analyses are correct.
If you want your organization to become
information-centric, you should be able to trust that data as well as
the analyses. Shockingly, 1 in 3 business leaders do not trust the
information they use in the decision-making. Therefore, if you want to
develop a big data strategy you should strongly focus on the correctness
of the data as well as the correctness of the analyses.
Variability
Big data is extremely variable. Brian Hopkins, a Forrester principal analyst, defines
variability as the "variance in meaning, in lexicon". He refers to the
supercomputer Watson who won Jeopardy. The supercomputer had to "dissect
an answer into its meaning and [... ] to figure out what the right
question was". That is extremely difficult because words have different
meanings an all depends on the context. For the right answer, Watson had
to understand the context.
Variability is often confused with
variety. Say you have bakery that sells 10 different breads. That is
variety. Now imagine you go to that bakery three days in a row and every
day you buy the same type of bread but each day it tastes and smells
different. That is variability.
Variability is thus very relevant
in performing sentiment analyses. Variability means that the meaning is
changing (rapidly). In (almost) the same tweets a word can have a
totally different meaning. In order to perform a proper sentiment
analyses, algorithms need to be able to understand the context and be
able to decipher the exact meaning of a word in that context. This is
still very difficult.
Visualization
This is
the hard part of big data. Making all that vast amount of data
comprehensible in a manner that is easy to understand and read. With the
right visualizations, raw data can be put to use. Visualizations of
course do not mean ordinary graphs or pie-charts. They mean complex
graphs that can include many variables of data while still remaining
understandable and readable.
Visualizing might not be the most
technological difficult part; it sure is the most challenging part.
Telling a complex story in a graph is very difficult but also extremely
crucial. Luckily there are more and more big data startups appearing
that focus on this aspect and in the end, visualizations will make the
difference.
Value
All that available data
will create a lot of value for organizations, societies and consumers.
Big data means big business and every industry will reap the benefits
from big data. McKinsey states that potential annual value of big data
to the US Health Care is $ 300 billion, more than double the total
annual health care spending of Spain. They also mention that big data
has a potential annual value of € 250 billion to the Europe's public
sector administration. Even more, in their well-regarded report from
2011, they state that the potential annual consumer surplus from using
personal location data globally can be up to $ 600 billion in 2020. That
is a lot of value.
Of course, data in itself is not valuable at
all. The value is in the analyses done on that data and how the data is
turned into information and eventually turning it into knowledge. The
value is in how organizations will use that data and turn their
organization into an information-centric company that bases their
decision-making on insights derived from data analyses.
Use cases
Know that the definition of big data is clear, let's have a look at the different possible use cases.
Of course, for each industry and each individual type of organization,
the possible use cases differ. There are however, also a few generic big
data use cases that show the possibilities of big data for your
organization.
1. Truly get to know your customers, all of them in real-time.
In
the past we used focus groups and questionnaires to find out who our
customers where. This was always outdated the moment the results came in
and it was far too high over. With big data this is not necessary
anymore. Big Data allows companies to completely map the DNA of its
customers. Knowing the customer well is the key to being able to sell to
them effectively. The benefits of really knowing your customers are
that you can give recommendations or show advertising that is tailored
to the individual needs.
2. Co-create, improve and innovate your products real-time.
Big
data analytics can help organizations gain a better understanding of
what customers think of their products or services. Through listening on
social media and blogs what people say about a product, it can give
more information about it than with a traditional questionnaire.
Especially if it is measured in real-time, companies can act upon
possible issues immediately. Not only can the sentiment about products
be measured, but also how that differs among different demographic
groups or in different geographical locations at different timings.
If anyone want to know more about
Hadoop Training in Jaipur
3. Determine how much risk your organization faces.
Determining
the risk a company faces is an important aspect of today's business. In
order to define the risk of a potential customer or supplier, a
detailed profile of the customer can be made and place it in a certain
category, each with its own risk levels. Currently, this process is
often too broad and vague and quite often a customer or supplier is
placed in a wrong category and thereby receiving a wrong risk profile. A
too high-risk profile is not that harmful, apart from lost income, but a
too low risk profile could seriously damage an organization. With big
data it is possible to determine a risk category for each individual
customer or supplier based on all of their data from the past and
present in real-time.
4. Personalize your website and pricing in real-time toward individual customers.
Companies
have used split-tests and A/B tests for some years now to define the
best layout for their customers in real-time. With big data this process
will change forever. Many different web metrics can be analyzed
constantly and in real-time as well as combined. This will allow
companies to have a fluid system where the look, feel and layout change
to reflect multiple influencing factors. It will be possible to give
each individual visitor a website specially tailored to his or her
wishes and needs at that exact moment. A returning customer might see
another webpage a week or month later depending on his or her personal
needs for that moment.
5. Improve your service support for your customers.
With
big data it is possible to monitor machines from (great) distance and
check how they are performing. Using telematics, each different part of a
machine can be monitored in real-time. Data will be sent to the
manufacturer and stored for real-time analysis. Each vibration, noise or
error gets detected automatically and a when the algorithm detects a
deviation from the normal operation, service support can be warned. The
machine can even schedule automatically for maintenance at a time when
the machine is not in use. When the engineer comes to fix the machine,
he knows exactly what to do due to all the information available.
6. Find new markets and new business opportunities by combining own data with public data.
Companies
can also discover unmet customer desires using big data. By doing
pattern and/or regression analysis on your own data, you might find
needs and wishes of customers you did not know they were present.
Combining various data sets can give whole new meanings to existing data
and allows organizations to find new markets, target groups or business
opportunities it was previously not yet aware of.
7. Better understand your competitors and more importantly, stay ahead of them.
What
you can do for your own organization can also be done, more or less,
for your competition. It will help organizations better understand the
competition and knowing where they stand. It can provide a valuable head
start. Using big data analytics, algorithms can find out for example if
a competitor changes its pricing and automatically adjust your prices
as well to stay competitive.
8. Organize your company more effectively and save money.
By
analyzing all the data in your organization you may find areas that can
be improved and can be organized better. Especially the logistics
industry can become more efficient using the new big data source
available in the supply chain. Electronic On Board Recorders in trucks
tell us where they are, how fast they drive, where they drive etc.
Sensors and RF tags in trailers and distribution help on-load and
off-load trucks more efficiently and combining road conditions, traffic
information and weather conditions with the locations of clients can
substantially save time and money.
Of course these are just
generic use cases are just a small portion of the massive possibility of
big data, but it shows that there are endless opportunities to take
advantage of big data. Each organization has different needs and
requires a different big data approach. Making correct usage of these
possibilities will add business value and help you stand out from your
competition.
