AI has been facing a PR problem. Too often AI has introduced itself as a misogynist, racist and sinister robot. Remember the Microsoft Twitter chatbot named Tay, who was learning to mimic online conversations, but then started to blur out the most offensive tweets? Think of tech companies creating elaborate

The success of data-driven projects has quite a few challenges and barriers. Here is a look at how you could overcome them by simply asking yourself three questions. Data has become probably the most valuable asset that companies could have nowadays. It can give you insights into your customers’ behaviour

Linear regression is a basic tool. It works on the assumption that there exists a linear relationship between the dependent and independent variable, also known as the explanatory variables and output. However, not all problems have such a linear relationship. In fact, many of the problems we see today are

This article is a continuation of my first article, 25 Big Data terms everyone should know. Since it got such an overwhelmingly positive response, I decided to add an extra 50 terms to the list.  Just to give you a quick recap, I covered the following terms in my first

In recent years’ evidence has been mounting that points to a crisis in the reproducible results of scientific research. Reviews of papers in the fields of psychology and cancer biology found that only 40% and 10%, respectively, of the results, could be reproduced. Nature published the results of a survey of

This article is part of a media partnership with PyData Berlin, a group helping support open-source data science libraries and tools. To learn more about this topic, please consider attending our fourth annual PyData Berlin conference on June 30-July 2, 2017. Miroslav Batchkarov and other experts will be giving talks

The R language is often perceived as a language for statisticians and data scientists. Quite a long time ago, this was mostly true. However, over the years the flexibility R provides via packages has made R into a more general purpose language. R was open sourced in 1995, and since

The late data visionary Hans Rosling mesmerised the world with his work, contributing to a more informed society. Rosling used global health data to paint a stunning picture of how our world is a better place now than it was in the past, bringing hope through data. Now more than

The rise of the data scientists continues and social media is filled with success stories – but what about those who fail? There are no cover articles praising the failures of the many data scientists that don’t live up to the hype and don’t meet the needs of their stakeholders.

Oftentimes while analyzing big data we have a need to make checks on pieces of data like number of items in the dataset, number of unique items, and their occurrence frequency. Hash tables or Hash sets are usually employed for this purpose. But when the dataset becomes so enormous that