The field of data science has been overcome by buzzwords over the past few years. Such words often start as potentially powerful and revolutionary ideas, but through overuse and miscomprehension lose their potency. One of the terms within the field of data science that is adhering to this trajectory is deep learning. We’ve heard it used as a synonym for machine learning. We’ve heard it described as “the future”. We’ve also heard it described as a meaningless “buzzword for neural nets”. Aside from the fact some of these descriptions are wildly inaccurate, none of them bring the listener any closer to understanding what deep learning actually entails.
What is deep learning?
Deep learning uses neural networks (and multiple levels of abstraction and representation)to make sense of data, and improve functions such as image classification, speech recognition and natural language processing.
One of the best high-level and easily comprehensible definitions of machine learning and deep learning for the totally unitiated is the one provided by Gary Marcus, a cognititve Professor at NYU. He states: “A computer is confronted with a large set of data, and on its own asked to sort the elements of that data into categories, a bit like a child who is asked to sort a set of toys, with no specific instructions. The child might sort them by color, by shape, or by function, or by something else. Machine learners try to do this on a grander scale, seeing, for example, millions of handwritten digits, and making guesses about which digits looks more like one another, “clustering” them together based on similarity. Deep learning’s important innovation is to have models learn categories incrementally, attempting to nail down lower-level categories (like letters) before attempting to acquire higher-level categories (like words).”
Why the hype?
In short, because relatively-new deep learning techniques have dramatically outperformed their predecessors in certain use cases. A frequently-cited example is the Google “Brain” deep learning project, which was fed 10,000 Youtube images without being told what to look for, and from this was able to recognise and classify what a cat looked like. This system worked around 70% than its predecessors, constituing a dramatic improvement. But a 70% improvement means it was able to classify around a sixth of images.
Another reason for the hype is that many tech giants- Facebook, Google, Microsoft– are working on deep learning systems, and acquiring deep learning startups left, right and center. Twitter acquired Madbits, as well as Google acquiring DNNResearch & Jetpac, and Pinterest acquiring Visual Graph. When we asked Cloudera’s Director of Data Science Sean Owen about such acquisitions back in August, he stated:
Interestingly all of these small start-ups have managed to make deep learning do something related to image recognition. It’s not surprising that image classification is a problem the tech giants have, so it’s not surprising that they want to buy these technologies at almost any price. In a way I’m not sure this is reflective of a lot of trends in machine learning and in the industry, even if these are the most visible transactions in this space.
I haven’t really seen deep learning make a big dent in any application, except for image recognition classification, and maybe related fields. In theory neural networks should be able to improve a lot of classification problems – in practice, it really hasn’t. It’s good at classification over a large number of continuous values, and that’s exactly image recognition.
What Other Applications Exist?
Although the application of deep learning in computer vision has been dominating the narrative, it is deep learning’s application in other fields which could most radically change our futures.
Facial detection and image search are undoubtedly useful, but not as powerful as the capabilities of natural language processing and speech recognition. Yoshua Bengio of the University of Montreal identifies NLP in particular as “something that companies like Facebook and Google are very interested in because the possibility of understanding the meaning of the text that people type or say is very important for providing better user interfaces, advertisements, and posts for your [news feed]. If deep learning can make the kind of impact in this area that it has in speech and object recognition, that could be a very, very important development in terms of value.”
Andrew Ng of Stanford University and Baidu predicts speech and image recognition will have a profoundly disruptive effect in the future, particularly when it comes to search. He sees speech and images as a “much more natural way to communicate”, and predicts that within the next five years, half of our search queries will be speech and images. In a recent Google Hangout talk here in Berlin, he predicted that future phones would be almost entirely controlled by speech. Deep learning will be the driving force behind these innovations.
So is deep learning a meaningless hype term, or an integral part of our futures? Even the sharpest minds in the field can only speculate at this point. Time will tell if deep learning will be as revolutionary in the advancement of technology as many hope it will be.
