How we reached the age of big data and what the new information world beholds

in #technology7 years ago


In this information age, data is the biggest asset that runs the world. It is through this data that tomorrow's weather is forecasted, customer satisfaction is determined, poller preferences are understood and the future of our planet is predicted. 

How we get here? 

There has been a massive shift in the way we collect, understand and interpret things around us. And with it the guiding force of humanity has changed.

 Earlier it was knowledge- developed from information. People used to discover new information about anything encountered, like a new edible plant or a poisonous mushroom. They conveyed it to others and that's how knowledge was formed. 


               )

Now we focus on even small packets of information i.e. Data. This data can be combined in various ways to produce a new set of information every time. So in we have removed all barriers to the amount of  knowledge we can generate. 

For example, a data collection of how often a person feels sad and due to what reasons can help us investigate not one but many aspects of our behavior and interaction with things. If this is combined with how we act when we are sad, a new field of predictive behavioural science can substantially affect almost any other field. 

                                 


So one thing is certain, there is now no limit to human knowledge. It has become boundless. But there's a catch! It can only create new knowledge when we do these three things efficiently-

● Data Collection, 

● Data storage, and, 

● Data Analysis

As technologies  developed, there has been a significant shift in the first two points. And it is this shift that is driving an entirely new stream full of possibilities- Data Analytics. 

Before we get there and discuss what new avenues it has opened, let us talk a little more about the first two. 

Data Collection is perhaps the most important step without which the other two become obsolete. Also, it is the way and the amount in which we collect data that directs the other two steps. 

In the early days of computing, before the invention of internet exploded, data entries were generally limited to offices and were carried out by employees. However, as this data started become too huge to be kept at one place (CERN), it triggered the the creation of the world wide web by Tim Berners-Lee. 

Once that happened, the data entry process became universal. Earlier what was limited to transcription and soft copy generation, was now expanded for common folk to upload pictures, posts, videos, songs and almost anything. So as that happened, suddenly there was an outburst of unprecedented data which is only growing.

                             

This growth is facilitated by a third member in the data collection process, one which is more relentless than employees or layman sitting at home.

 The third member here is none other than machines. With increasing automation, more and more machines are now doing the jobs that we used to do; only more efficiently. Satellites collect our planetary data, surveillance cameras capture our every move, our smart tools record our every interest. 

                                               

As these three continue to collect huge piles of data, so are we trying to advance the ways in which store and process them. Now the data doesn't need a remote physical device to be stored. Everything goes up in the cloud and pours down whenever, wherever required. Data is no longer taken to a CPU, many tiny CPUs are allocated to every set of data. This has become the power of data. It's sheer amount, if  not influence, makes us call it- “Big Data”

Where is it taking us? 

Having understood the limitless implications of data, many companies, political parties, research laboratories and think tanks are now finding newer ways to exploit it. 

Let us have a look at some of them-

● Consumer products

As more and more people are shopping online, the companies are collecting and tracking user preferences with every click. Choices are not only interpreted at an individual level but also based on gender, age, location and economic background.

This helps the companies to determine what price and offers can result in maximum sale and also how to better improve the existing products to meet the future expectations of the consumer.

● Elections

Elections are perhaps the biggest determinants of the citizens state of being. Politics affects every sphere of life whether it be economy, security, science, education or justice. 

Understanding the immense implications and the subsequent benefits generated from staying in power, political parties are using big data to understand and manipulate voter behaviour not only on the state or precinct level but to the level of families and sometimes even individuals. 

In the 2012 presidential election, the Obama Campaign made use of voter models on a scale never before seen. They were able to identify specific voters who would make a difference in the election and targeted messages to those voters. The results in the end proved how efficient big data was in getting the desired outcome. 

● Urban Planning

Cities are no longer a collection of buildings, streets, shops houses linked by bridges and roads. 

They are becoming a hub for the majority of population of any country. Cities are where the density of citizens is the highest. To facilitate day to day life for these thousands, it’s become imperative for cities to be smart. 

Today, such smartness reflects on every aspect of city life. People get to one place to another quicker, access basic facilities easier and thus increase the overall productivity. In order to do this and a lot more, Cities have started to track every movement of people and vehicles. 

A MIT professor recently found that a small number of Boston-area drivers contribute disproportionately to traffic jams and gridlock. If such information is incorporated with data pertaining to events and activists going on in the city, it would make it much easier to divert traffic through less cumbersome roads.

● Healthcare and Genetics

Healthcare is a field that has already reaped out a lot of benefits from Big Data. 

Hospitals have already started collaborating patient records to better understand diseases that affect mankind on a grave scale. 

This might one day help doctors to diagnose you better by comparing your symptoms with thousands others who have had similar issues and belonged to the same age group. 

This will also help them prescribe better medicines by analysing your genetic profile. Every person’s body reacts differently to a medicine and comparing the gene structure and drug interaction pattern can exponentially increase the cure rate. Huge investments have already been made in genetic profiling which enables us to ‘engineer’ human life as per preferences. 

● Security 

Security infrastructure has developed a lot in the last decade and it rightly calls for more advancements. 

As state surveillance is becoming more and more commonplace, governments now  record a lot of data whether it be your daily movements, voice calls, internet activity, leisure activities or anything else. 

Up until now, facial recognition as a security tool was useful but it's efficiency was limited. Simply because there are so many people and even computers had trouble processing every face they came across. But, with time, we are getting  better and better at processing big data, security cameras coupled with more sophisticated facial recognition is helping us prevent major mishaps. 

These were just some of the applications of big data. Big data is today driving innovation in any field you name. Just like knowledge has no bounds, there is no limit to the applicability of big data.