Deep learning is one of the hottest topics of this industry today. Deep Learning is evolving and it is top of the Data Science world. Deep Learning has led to amazing innovations, incredible breakthroughs, and we are only just getting started. A lot of people carry an impression that deep learning involves a lot of mathematics and statistical knowledge.
If you had similar questions about deep learning, but were not sure how, when and where to ask them you are at the right place. This article should answer most of what you would want to know. By end of this article, we will dispel a few myths about deep learning and answer some widely asked questions about this field. We have also included plenty of resources to get you started.
Table of Contents:
What is Deep Learning and why is it so popular?
Is Deep Learning just a hype or does it have real-life applications?
What is the difference between Deep Learning and Machine Learning?
What are the prerequisites for starting out in Deep Learning?
Is it necessary to have a research background in Deep Learning to start out in it?
Which Tools/Languages should I prefer to build Deep Learning models?
Why are GPUs necessary for building Deep Learning models?
When (and where) to apply Deep Learning?
Do we need a lot of data to train deep learning models?
Where can I find some basic project ideas in order to practice deep learning?
What are the free learning resources for Deep Learning?
What are Deep Learning interview questions?
What is the future of Deep Learning?
1. What is Deep Learning and why is it so popular these days?
Deep Learning is nothing but a paradigm of machine learning which has shown incredible promise in the recent years. This is because of the fact that Deep Learning shows great analogy with the functioning of the human brain. The superiority of the human brain is an evident fact, and it is considered to be the most versatile and efficient self-learning model that has ever been created.
Let us understand the functioning of a deep learning model with an example:
2. Is Deep Learning just a hype or does it have real-life applications?
Deep Learning has found many practical applications in the recent past. From Netflix's famous movie recommendation system to Google's self-driving cars, deep learning is already transforming a lot of businesses and is expected to bring about a revolution in almost all industries. Deep learning models are being used from diagnosing cancer to winning presidential elections, from creating art and writing literature to making real life money. Thus it would be wrong to say that it is just a hyped topic anymore.
Some major applications of deep learning that are being employed by technology companies are:
Google and Facebook are translating text into hundreds of languages at a time. This is being done through some deep learning models being applied to NLP tasks and is a major success story.
Conversational agents like Siri, Alexa, Cortana basically work on simplifying the speech recognition techniques through LSTMs and RNNs. Voice commands have added a whole new domain to the possibilities of a machine.
Deep learning is being used in impactful computer vision applications such as OCR (Optical Character Recognition) and real time language translation
Multimedia sharing apps like Snapchat and Instagram apply facial feature detection which is another application of deep learning.
Deep Learning is being used in Healthcare domain to locate malignant cells and other foreign bodies in order to detect complex diseases.
However, some people develop a thinking that deep learning is over hyped because of the fact that labeled data required for training deep learning models is not readily available. Even if the data is available, the computational power required to train such models does not come cheap. Hence, due to these barriers, people are not able to experience the power of deep learning and term it as just hype.
3. What is the difference between Deep Learning and Machine Learning?
This is one of the most important questions that most of us need to understand. The comparison can be done mainly on the below three verticals:
The most important difference between deep learning and traditional machine learning is its performance as the scale of data increases. When the data is small, deep learning algorithms don't perform that well. This is because deep learning algorithms need a large amount of data to understand it perfectly. On the other hand, traditional machine learning algorithms with their handcrafted rules prevail in this scenario. Below image summarizes this fact.
Feature engineering is a process of putting domain knowledge into the creation of feature extractors to reduce the complexity of the data and make patterns more visible to learning algorithms to work. This process is difficult and expensive in terms of time and expertise.
In Machine learning, most of the applied features need to be identified by an expert and then hand-coded as per the domain and data type. For example, features can be pixel values, shape, textures, position and orientation. The performance of most of the Machine Learning algorithm depends on how accurately the features are identified and extracted.
Deep learning algorithms try to learn high-level features from data. This is a very distinctive part of Deep Learning and a major step ahead of traditional Machine Learning. Therefore, deep learning reduces the task of developing new feature extractor for every problem. Like, convolutional neural network will try to learn low-level features such as edges and lines in early layers then parts of faces of people and then high-level representation of a face.
Last but not the least, we have interpretability as a factor for comparison of machine learning and deep learning. This factor is the main reason deep learning is still thought 10 times before its use in industry.
Let's take an example. Suppose we use deep learning to give automated scoring to essays. The performance it gives in scoring is quite excellent and is near human performance. But there's is an issue. It does not reveal why it has given that score. Indeed mathematically you can find out which nodes of a deep neural network were activated, but we don't know what there neurons were supposed to model and what these layers of neurons were doing collectively. So we fail to interpret the results.
On the other hand, machine learning algorithms like decision trees give us crisp rules as to why it chose what it chose, so it is particularly easy to interpret the reasoning behind it. Therefore, algorithms like decision trees and linear/logistic regression are primarily used in industry for interpretability.
4. What are the prerequisites for starting out in Deep Learning?
Starting out in deep learning is not as difficult as people might make you believe. There are a few elementary basics that you should cover before diving into deep learning. Deep learning requires knowledge of the following topics:
Mathematics: You should be comfortable with probability, derivatives, linear algebra and a few other basic topics. Khan Academy offers a decent course covering almost all the above topics here.
Statistics: The basics of statistics are required for going forward with any machine learning problem. Understanding the concepts of statistics are essential because most of the deep learning concepts are derived from assimilating the concepts of statistics. You can check the online courses available here.
Tool: A decent level of coding skills are required for implementing deep learning into real life problems. Coursera's. Introduction to Data Science in Python is a decent course to start off with Python as a tool.
Machine Learning: Machine learning is the base for deep learning. One can not start learning deep learning without understanding the concepts of machine learning. You could go through Intro to Machine Learning or Andrew Ng's course Machine Learning for a theoretical base.
5. Do I need to do a PhD to make a career in Deep Learning?
No, a PhD is not a mandatory requirement to make a career in deep learning. You can learn, experiment, and build up your work experience portfolio without going to university. The emphasis for any job or role is usually on demonstrating your competence, and not on your degree.
Having said that, a PhD in a specific field (like linguistics for NLP) will definitely accelerate your path if you choose to combine that with deep learning.
6. Which Tools/Languages should I prefer to build Deep learning models?
The most recommend language to use is Python, because of its robust ecosystem for machine learning. The python ecosystem comprises of developers and coders who are providing open source libraries and support for the community of python users. This makes the task of writing complex codes for various algorithms much easier and the techniques easier to implement and experiment with.
Also, Python being a more generalized programming language, can be used for both the development and implementation. This greatly simplifies the transition from development to operations. That is, a deep learning product that can predict the price of flight tickets, can not only be developed in python but can also be attached with your website in the same form. This is what makes Python a universal language. Besides this, I would suggest that beginner's use high level libraries like Keras. This makes experimentation easier by providing abstraction to the unnecessary information that is hidden under the algorithms. And giving access to the parameters that can be tweaked to enhance the performance of such models.
7. Why are GPUs necessary for building Deep Learning models?
When you train a deep learning model, two main operations are performed:
In forward pass, input is passed through the neural network and after processing the input, an output is generated. Whereas in backward pass, we update the weights of neural network on the basis of error we get in forward pass.
8. When and where to apply Neural Networks ?
Deep Learning have been in the spotlight for quite some time now. Its Ã¢??deeperÃ¢?? versions are making tremendous breakthroughs in many fields such as image recognition, speech and natural language processing etc.
Now that we know it is so impactful the main question that arises is when to and when not to apply neural networks? This field is like a gold mine right now, with many discoveries uncovered everyday. And to be a part of this Ã¢??gold rushÃ¢??, you have to keep a few things in mind:
Firstly, deep learning models require clear and informative data (and mostly big data) to train. Try to imagine deep learning model as a child. It first observes how its parent walks. Then it tries to walk on its own, and with its every step, the child learns how to perform a particular task. It may fall a few times, but after few unsuccessful attempts, it learns how to walk. If you don't let it, it might not ever learn how to walk. The more exposure you can provide to the child, the better it is.
It is prudent to use Deep Learning for complex problems such as image processing. Deep Learning algorithms belong to a class of algorithms called representation learning algorithms. These algorithms break down complex problems into simpler form so that they become understandable. Think of it as chewing food before you gulp. This would be harder for traditional (non-representation learning) algorithms.
When you have an appropriate type of deep learning to solve the problem. Each problem has its own twists. So the data decides the way you solve the problem. For example, if the problem is of sequence generation, recurrent neural networks are more suitable. Whereas, if it is image related problem, you would probably be better of taking convolutional neural networks for a change.
Last but not the least, hardware requirements are essential for running a deep neural network model. Neural nets were Ã¢??discoveredÃ¢?? long ago, but they are shining in the recent years for the main reason that computational resources are better and more powerful. If you want to solve a real life problem with these networks, get ready to buy some high-end hardware.
9. Do we need a lot of data to train deep learning models?
It is true that we need a large amount of data to train a typical deep learning model. But we can generally overcome this by using something called transfer learning. Let me explain thoroughly.
One of the barrier for using deep learning models for industry applications is where the data is not in huge amount. A few examples of data needed to train some of the popular deep learning models are: However, a deep learning model trained on a specific task can be reused for different problem in the same domain even if the amount of data is not that huge. This technique is known as Transfer Learning.
For instance, we have a set of 1000 images of cats and dogs labeled as 1 and 0 (1 for cat and 0 for dog) and we have another set of 500 test images that we need to classify. So, instead of training a deep learning model on the data of 1000 images, we can use a pre-trained VGGNet model and retrain it on our data and use it to classify the unlabeled set of images. A pre-trained model may not be 100% accurate in your application, but it saves huge efforts required to reinvent the wheel.
You may have a look at this article to get a better intuition of using a pre-trained model. Where can I find basic project ideas in order to practice deep learning? To practice deep learning, ideas alone will not help. We also need labeled data to test our ideas using deep learning.
For beginners, getting started with the MNIST data is highly recommended. The data-set contains handwritten digits with their actual labels, i.e., numbers from 0 to 9. You can even compete in the Identify the Digits competition to evaluate your models. For intermediate users, this Age Detection challenge is a nice project to work on. The data-set consists of facial images of Indian movie actors. The task is to predict the age of a person from his or her facial attributes. For simplicity, the problem has been converted to a multi-class problem with classes as Young, Middle and Old.
10. What are some of the free learning resources for Deep Learning?
Being a comparatively newer technology, there is not enough content and tutorials available for the beginners. However, free-quality content and resources related to deep learning are steadily increasing. The learning resources can be classified on the different applications of deep learning.
11. What are some Deep Learning interview questions?
Some common questions that may be asked on deep learning are:
How do deep learning models learn?
What are some limitations of a deep learning model?
What are the differences between feed forward neural networks and recurrent neural networks?
What are activation functions and why are they required?
What is a CNN and what are its applications?
What is pooling? How does it work?
What is a dropout layer and why is it used?
What is the vanishing gradient problem and how do we overcome that?
What are optimization functions? Name a few of the common optimization functions.
Do note that this is not an exhaustive list that will make you completely ready for an interview. You can also learn from the tutorials as well.
12. What is the future of Deep learning?
Deep learning has come a long way in recent years, but still has a lot of untapped potential. We are still in the nascent stages of this field, with new breakthroughs happening seemingly every day. One of the use-cases that we can definitely see in the suture is of automobile industry, where Deep Learning can revolutionize it by making self-driving cars a reality. While we don't have a crystal ball to predict the future, we can see deep learning models requiring less and less involvement from human data scientists and researchers.
In the immediate future, we can definitely see a trend where the knowledge of deep learning will be a skill required by every Data Science practitioner. In fact, you must have caught sight of a job position spurn out recently, called a Ã¢??Deep Learning EngineerÃ¢??. This person is responsible to deploy and maintain Deep Learning models used by various departments of that company. Needless to say, there will be a huge demand of such people in the industry.
Currently, one of the limitations of deep learning is that it does what a human asks of it. It requires tons of data to learn it's target objective, and replicates that. This has induced bias in certain applications. We can see this improving over time such that the bias is eliminated in the training process.
We might even stop differentiating deep learning from the other types of learning, with time. It is primed to become a popular and commonly used field and will not require special branding efforts to market or sell.
There are a lot of cases still where researchers, after training a deep learning model, are unable to explain the why' behind it. It's producing great results but why did you tune a hyper parameter a certain way. Hopefully with the rapid advancement in Deep Learning, we will see this black box concept becoming history, and we can explain the intuition behind the decision it takes.