Data science requires the effective application of skills in a variety of machine learning areas and techniques. A recent survey by Kaggle, however, revealed that a limited number of data professionals possess competency in advanced machine learning skills. About half of data professionals said they were competent in supervised machine learning (49%) and logistic regression (53%). Deep learning techniques were among the ML skills with the lowest competency rates: Neural Networks - GAN (7%); NN - RNNs (15%) and NN - CNNs (26%).
Kaggle conducted a survey in August 2017 of over 16,000 data professionals (2017 State of Data Science and Machine Learning). The survey asked respondents about their competence across a variety of AI-related approaches and techniques. Looking at different AI skills will give us a more detailed look into the specific AI skills that are driving this talent gap.
Competency in Machine Learning Areas
All respondents (employed or not) were were given a list of 13 machine learning areas and asked to indicate in which areas they consider themselves competent. The top 10 machine learning areas in which data professionals are competent were (see Figure 1):
Supervised Machine Learning (49%)
Unsupervised Learning (26%)
Time Series (25%)
Natural Language Processing (19%)
Outlier detection (16%)
Computer Vision (15%)
Recommendation Engines (14%)
Survival Analysis (8%)
Reinforcement Learning (6%)
Adversarial Learning (4%)
On average, the respondents said they were competent in two (2 - median) machine learning areas. Competency rates varied by the job title. Competency rates were higher for data professionals who self-identified as "data scientists" and "machine learning engineers," each saying they were competent in three (3) ML areas. Competency rates were lower for data professionals who self-identified as "business analyst," "engineers," and "programmers," each saying they were competent in one (1) ML area.
Competency in Machine Learning Techniques
The survey included a question for all data professionals, employed or not, regarding their competency in 13 machine learning techniques (In which areas of machine learning do you consider yourself competent? (Select all that apply).) The top 10 machine learning techniques in which data pros are competent were (see Figure 2):
Logistic Regression (54%)
Decision Trees - Random Forests (43%)
Support Vector Machines (32%)
Decision Trees - Gradient Boosted Machines (31%)
Bayesian Techniques (27%)
Neural Networks - CNNs (26%)
Ensemble Methods (22%)
Gradient Boosting (17%)
Neural Networks - RNNs (15%)
Hidden Markov Models HMMs (9%)
Data professionals said they were competent in two (2 - median) machine learning areas. Competency rates varied by job title. Self-identified data scientists said they were competent in five (5) ML techniques, while data professionals who identified as machine learning engineers or predictive modelers said they were competent in four (4) ML techniques. Competency rates were lower for database engineers and programmers, each possessing competency in one (1) ML technique.
Summary
Even though a few ML areas and techniques are known by about half of the data professionals (i.e., supervised machine learning, logistic regression, decision trees), there are many other ML areas and techniques in which there is a paucity of talent. A majority of data professionals lack competency in many advanced machine learning areas and techniques like neural networks, evolutionary techniques, reinforcement learning and adversarial learning.
If you're an aspiring or practicing data professional, you need a good foundation of the basic skills, including supervised machine learning, logistic regression and decision trees. These types of skills apply to a wide variety of data science problems that focus on identifying predictors of important organizational outcomes.
If you want to set yourself apart from your data peers, you might consider becoming competent in rare AI skills like NLP, reinforcement learning, adversarial learning, recommendation engines and neural networks. In fact, 4 out of 10 data professionals said they are most excited to learn about deep learning this year. The high demand of AI talent coupled with the limited talent supply could result in extremely high salaries for those willing to learn these AI skills.
