It isn’t a surprise that the amount of data generated in the past few years is astounding—in fact, the data generated within the last two years is close to 90 percent of all data ever generated. However, 80 percent of all enterprise data is unstructured as the numbers continue to grow. Gone are the days when an organization could completely run business intelligence off relational databases. The plethora of information, known as Big Data, gathered over years needs to be mined. Each and every organization stands to benefit by taking this approach, and ultimately drive towards higher profitability. But how you mine data or whom do you hire? Well, you need a data scientist.

Whether it’s retailers who want to track their customer’s purchasing interests and predict the next purchase to run a targeted marketing campaign, or the Department of Homeland Security who would spend hundreds of hours on link analysis to uncover terrorist activity, every organization can benefit by applying data science regardless of their industry.

Who Are Qualified Data Scientists?

Data scientists are individuals who have advanced degrees in mathematics or statistics, extensive domain knowledge, and a strong programming background. According to the Burtch Works study, 46 percent of data scientists have PhDs, while 42 percent hold Masters Degrees. However realistically, these candidates are not an easy find.

McKinsey Global Institute published a report stating that by 2018, the U.S. alone may face a 50 to 60 percent gap between supply and requisite demand of deep analytic talent. The institute further projected that in U.S. alone, four million positions will seek individuals with strong quantitative and analytics background by 2018. Unfortunately, there exists a gap to find a data scientists in almost every industry, whether it is finance, aerospace, hi-tech, or any other.

As Demand for Data Scientists have Increased, So Have Salaries

According to, data-scientist salaries are 113 percent higher than the average salary for all jobs in United States. In fact, the average data scientist salary, of 123,000 USD, increased by roughly 20 percent in the past two years. The 2014 Burtch Works study of salaries and data scientists concluded that data scientists with nine or more years of experience draw on average a salary of 150,000 USD, which escalates at the managerial level. More specifically, an individual managing a team of three or less earns an average of 140,000 USD while someone who manages a team of ten or more would earn an average of 232,500 USD. When compared to the average salary for a doctor (183, 940 USD) or a lawyer (131,990 USD) data scientists are very well compensated based on data published by the U.S. Bureau of Labor Statistics. Although it takes an advanced degree, strong business acumen and intellect to land at a high-paying job, organizations will pay top dollar for data scientists who providing value to the business.

Decision Time: Hiring Within and Growing or Hiring External Candidates?

In 2014, Accenture surveyed its clients on their Big Data strategies and found that more than 90 percent of the respondents planned to hire employees with expertise in data science. Training employees to be data scientist are difficult but not entirely impossible since they already have domain knowledge and an understanding of company’s business. However, to truly fill shoes of a data scientist, an employee needs to be a great researcher with background in statistics, a great problem solver, and a great software developer. Unfortunately, many organizations believe that employees proficient in one area can be good data scientists.

In addition to teaching an employee how to use Big Data technology and tools, the employee needs to have an aptitude and an interest in data science. Without this passion, employees in the data scientist roles will not deliver the compelling insights an organization seeks to gain. Employees within a company tend to get bored doing the same task over and over, and search for different roles and responsibilities within the company or elsewhere. When this is the case, it diminishes the value and effort it takes to train an employee to be data scientists.

Hiring from the outside is more straight-forward. Organizations can conduct extensive interviews which include, asking for sample work or academic presentations and determining the candidate’s aptitude for problem solving to better understand his or her data exploring capabilities. Another beneficial characteristic is being a team player. Employees can have great skillsets, but an organization can build great things with a team.

It is imperative to setup an interview as a two-way street; the candidate should feel that the company they consider joining has an interesting problem to solve. They should be curious not only about how the company is solving critical business issues using data science but also how they can add value. The ideal individual can make solid contributions to grow a team that can take the company further and create long-lasting value in an organization’s success.

Ensuring Long-Term Success of Data-Science Driven Organizations

Data scientists who succeed become role models for what’s valued in an organization, and that defines company’s culture. The best way to ensure long-term success is to clearly articulate and provide transparency about key company and team values. Over time, an organization’s HR practices, including hiring decisions, tenure of employees, and company success will help define and reflect the key values and culture. A company’s culture is its core asset, and chaos arises when key values start to fall apart.

Muddu_Photo Muddu Sudhakar, CEO of Caspida, is an experienced entrepreneur in the world of data driven technology. He previously served as VP & GM at VMware and Pivotal from 2010 to 2014 for Big Data Analytics and Cloud Services. At VMware and Pivotal he held responsibility for range of projects, including Software-defined Storage and Mobile Security. Muddu was also Co-Founder of Cetas, which was acquired by VMware in 2012.

(image credit: COD Newsroom)

Previous post

Flink Forward 2015: The First Conference for Apache Flink

Next post

Global Climate Change Data Competition