From detecting screen cancer to sorting cucumbers to detecting escalators in need of repair, machine learning has granted computer systems entirely new abilities. But how does it really work under the hood? Let's walk through a basic example and use it as an excuse to talk about the process of getting answers from your data using machine learning. Here we learn the art, science and tools of the machine learning. Let's pretend that you we have been asked to create the system that answers the question of whether a drink is wine or beer. This question answering system that we build is called a model and this model is created via a process of training. Machine Learning, the goal of training is to create an accurate model that answers our questions correctly most of the time.
But in order to train a model, we need to collect data to train on. This is where we will begin. Our data will be collected from glasses of wine and beer. There are many aspects of drinks that we could collect data on everything from the amount of foam to the shape of the glass. But for our purpose, we'll just pick two simple ones the color as a wavelength of light and the alcohol content as a percentage. The hope is that we can split our two types of drinks along these two factors alone. So, we now call all these our features from on color and alcohol. The first step to our process will be to run out to the local grocery store buy up a bunch of different drinks and get some equipment to do our measurements a spectrometer for measuring the color and a hydrometer to measure the alcohol content. It appears that our grocery store has an electronics hardware section as well. Once our equipment got it all set, following are the seven steps in machine learning:
Gathering the Data: This step is very important because the quality and the quantity of the data you gather will directly determine that how good your predictive model is can be. In this case, we collect data will be the color and alcohol content of each drink. This will yield us a table of color, alcohol content and whether it's beer or wine. This will be our training data. So few hours of measurement's later we've gathered our training data and had a few drinks, perhaps.
Data Preparation: Here we load our data into a suitable place and prepare it for use in our machine learning training. We first put all the data together and then randomize the ordering. We wouldn't want the order of our data to affect how we learn since that's not part of determining whether a drink is beer or wine. In other words, we want to make a determination of what a drink is independent of what drink came before or after it in the sequence. This is also a good time to do any pertinent visualizations of your data, helping you to see if there of your data, helping you see if there is any relevant relationships between different variables as well as show you if there are any data imbalances. For any instance, if we collected way or more data points about beer than wine, the model we train will be heavily biased towards guessing that virtually everything that it sees is beer since it would be right most of the time. However the real world, the model may see beer and wine in equal amount, which would mean that it would be guessing beer wrong half the time.
We also need to split the data into two parts, i.e. training and evaluation. The first part is used in our training model will be the majority of our dataset. The second part will be used to for evaluating our train models performance. We don't want to use the same the data that the model was trained on for evaluation since then it would just be able to memorize the questions just as you wouldn't want to use the questions from your math homework on the math exam. Sometimes the data we collected needs other forms of adjusting and manipulation things like duplication, normalization, error correction and others. These would all happen at the data preparation step. In our case we don't have any further data preparation needs.
Choosing a Model: There are many more models which were created by researchers and data scientists over the years. Some are very well suited for image data, others for sequences, such as text or music, or some for numerical data and others for text-based data. In our case we have just two features color and alcohol percentage. Here we can use a small linear model, which is a fairly simple one that will get the job done.
Training: Now we move on to what is often considered the bulk of machine learning training. In this step we use the data to incrementally improve our models ability to predict whether a given drink is wine or beer. In some ways this is similar to someone first learning to drive. At first, they don't know how any of the pedals, knobs and switches work or when they should be pressed or used. However after lots of practice and correcting for their mistakes, a licensed driver emerges. Moreover after a year of driving they've become a quite adept at driving. The act of driving and reacting to real-world data has adapted their driving abilities, honing their skills. Once the training is completed then it's the time to see that the model is any good.
Evaluation: Using evaluation, this is where that dataset that we set aside earlier comes into play. Evaluation allows us to test our model against data that has never been used for training. This metric allows us to see how the model might perform against data that it has not yet seen. This is meant to be representative of how the model might perform in the real world. A good rule of thumb we use for training-evaluation split is somewhere on the order of 80%-20%. Much of this depends on the size of the original source dataset. If you have a lot of data, perhaps you don't need as big of a fraction for the evaluation of the dataset.
Parameter Tuning: Once we done with the evaluation it's possible that you want to see if you can further improve the training is any ways. We can do this by tuning some of our parameters, there are few that we implicitly assumed when we did our training and now it's a good time to go back and test those assumptions, try other values.
Prediction: After all these steps now it's the finally time to use the model for something useful. Machine Learning is using data to answer questions so prediction or interference is that step where we finally get to answer some questions. This is the point of all this work where the value of machine learning is realized. We can finally use our model to predict whether a given drink is wine or beer, given its color and alcohol percentage. The power of machine learning is that we were able to determine how to differentiate between wine and beer using our model rather than using human judgment and manual rules.
Hence, these all are the seven easy steps to get master in machine learning. Machine Learning accesses the data and uses it for their learning purpose and gives us a trained data for the complex problems.