Top 10 machine learning algorithms most commonly used by data analysts
In the field of machine learning, there is a saying that “there is no free lunch in the world”. In short, it means that no algorithm can get the best results on all issues. This theory is reflected in supervised learning. Particularly important.
For example, you can't say that neural networks are always better than decision trees, and vice versa. Model running is dominated by many factors, such as the size and structure of the data set.
Therefore, you should experiment with many different algorithms based on your problem, while using a data test set to evaluate performance and pick the best terms.
Of course, the algorithm you try must be in line with your problem, and the doorway is the main task of machine learning. For example, if you want to clean the house, you might use a vacuum cleaner, a broom or a mop, but you certainly won't start digging with a shovel.
For machine learning newcomers who are eager to learn the basics of machine learning, here are the top ten machine learning algorithms most commonly used by data scientists and data analysts to introduce you to the characteristics of these top 10 algorithms so that you can better understand them. And the application, come to check it out.
1.Linear regression
Linear regression is probably one of the best known and most understandable algorithms in statistics and machine learning.
Because predictive modeling focuses on minimizing the error of the model, o making the most accurate prediction at the expense of interpretability. We borrow, reuse, and steal algorithms from many different areas, including some statistical knowledge.
Linear regression is represented by an equation that describes the linear relationship between the input variable (x) and the output variable (y) by finding the specific weight (B) of the input variable.
Example: y = B0 + B1 * x
Given the input x, we will predict y, and the goal of the linear regression learning algorithm is to find the values of the coefficients B0 and B1.
Linear regression models can be learned from the data using different techniques, such as linear algebraic solutions for ordinary least squares and gradient descent optimization.
Linear regression has existed for more than 200 years and has been extensively studied. If possible, some rule of thumb when using this technique is to remove very similar (related) variables and remove noise from the data. This is a quick and simple technique and a good first algorithm.
- Logistic regression
Logistic regression is another technique that machine learning borrows from the field of statistics. This is a special method for the two-category problem (two class value problems).
Logistic regression is similar to linear regression because the goal of both is to find the weight value of each input variable. Unlike linear regression, the prediction of the output is worth transforming using a nonlinear function called a logic function.
The logic function looks like a large S and can convert any value into a range of 0 to 1. This is useful because we can apply the corresponding rules to the output of the logic function, classify the values as 0 and 1 (for example, if IF is less than 0.5, then output 1) and predict the category value.
The predictions made by logistic regression can also be used to calculate the probability of belonging to class 0 or class 1 due to the unique learning style of the model. This is useful for questions that require many basic principles.
As with linear regression, logical regression is better when you remove attributes that are not related to the output variable and attributes that are very similar (related) to each other. This is a model for quickly learning and effectively dealing with binary classification problems.
- Linear discriminant analysis
Traditional logistic regression is limited to the two-category problem. If you have more than two classes, Linear Discriminant Analysis (LDA) is the preferred linear classification technique.
The representation of LDA is very simple. It consists of the statistical properties of your data, calculated according to each category. For a single input variable, this includes:
- The average for each category.
- The variance calculated across all categories.
The LDA is performed by calculating the discriminant value of each class and predicting the class having the largest value. This technique assumes that the data has a Gaussian distribution (bell curve), so it is best to manually remove outliers from the data. This is a simple and powerful method for classifying predictive modeling problems.
- Classification and regression trees
Decision trees are an important algorithm for machine learning.
The decision tree model can be represented by a binary tree. Yes, it is a binary tree from algorithms and data structures, nothing special. Each node represents a single input variable (x) and left and right children on that variable (assuming a variable is a number).
The leaf node of the tree contains the output variable (y) used to make the prediction. The prediction is performed by traversing the tree, stopping when a leaf node is reached, and outputting the class value of the leaf node.
Decision trees are fast and fast. It is often accurate for many problems, and you don't need to make any special preparations for the data.
- Naive Bayes
Naive Bayes is a simple but extremely powerful predictive modeling algorithm.
The model consists of two types of probabilities that can be calculated directly from your training data: 1) the probability of each category; 2) the conditional probability of the given category of each x value. Once calculated, the probability model can be used to predict new data using Bayes' theorem. When your data is numerical, you usually assume a Gaussian distribution (bell curve) so that you can easily estimate these probabilities.
The reason why Naive Bayes is called simplicity is that it assumes that each input variable is independent. This is a tough assumption that is impractical for real data, but it is still very effective for complex problems on a large scale.
- K nearest neighbor
The KNN algorithm is very simple and very efficient. The KNN model is represented by the entire training data set. Is it very simple?
Predict new data points by searching for the K most similar instances (neighbors) throughout the training set and summarizing the output variables of these K instances. For regression problems, the new point may be the average output variable, and for classification problems, the new point may be the mode category value.
The trick to success is how to determine the similarity between data instances. If your properties are all the same, the easiest way is to use the Euclidean distance, which can be calculated directly from the difference between each input variable.
KNN may require a lot of memory or space to store all the data, but it will only perform calculations (or learning) when it needs to be predicted. You can also update and manage your training set at any time to maintain the accuracy of your forecasts.
The concept of distance or tightness can crash in a high-dimensional environment (a large number of input variables), which can have a negative impact on the algorithm. This type of event is called a dimensional curse. It also implies that you should only use input variables that are most relevant to the predicted output variable.
- Learning vector quantization
The downside of K-nearby is that you need to maintain the entire training data set. The Learning Vector Quantization Algorithm (or LVQ for short) is an artificial neural network algorithm that allows you to suspend any training instances and learn them accurately.
LVQ is represented by a collection of codebook vectors. The vector is randomly selected at the beginning, and then iterated multiple times to fit the training data set. After learning, the codebook vector can be used for prediction like K-nearest neighbors. The most similar neighbor (best match) is found by calculating the distance between each codebook vector and the new data instance, and then the category value of the best matching unit or the actual value in the case of regression is returned as a prediction. If you limit the data to the same range (such as between 0 and 1), you get the best results.
If you find that KNN gives good results on your dataset, try using LVQ to reduce the memory requirements for storing the entire training dataset.
- Support vector machine
Support vector machines are perhaps one of the most popular and discussed machine learning algorithms.
A hyperplane is a line that divides the input variable space. In the SVM, a hyperplane is selected to separate the points in the input variable space by their category (class 0 or class 1). Think of it as a line in 2D space, and all input points can be completely separated by this line. The SVM learning algorithm is to find the coefficients that allow the hyperplane to best separate the categories.
The distance between the hyperplane and the nearest data point is called the boundary, and the hyperplane with the largest boundary is the best choice. At the same time, only these closely related data points are related to the definition of the hyperplane and the construction of the classifier. These points are called support vectors and they support or define hyperplanes. In practice, we will use an optimization algorithm to find the coefficient values that maximize the boundary.
SVM is probably one of the most powerful ready-to-use classifiers, and it's worth a try on your data set.
- bagging and random forest
Random forests are one of the most popular and powerful machine learning algorithms. It is an integrated machine learning algorithm called Bootstrap Aggregation or Bagging.
Bootstrap is a powerful statistical method for estimating a quantity, such as an average, from a data sample. It extracts a large amount of sample data, calculates the average, and then averages all the averages to more accurately estimate the true average.
The same method is used in bagging, but the most common one is the decision tree, not the entire statistical model. It trains the data for multiple sampling and then builds a model for each data sample. When you need to predict new data, each model will make predictions and average the predictions to better estimate the true output value.
Random forest is an adjustment to the decision tree. Compared with selecting the best segmentation point, the random forest realizes suboptimal segmentation by introducing randomness.
Therefore, the differences between the models created for each data sample will be greater, but still accurate in their own sense. Combined with the prediction results, the correct potential output value can be better estimated.
If you use high-variance algorithms (such as decision trees) to get good results, then adding this algorithm will work better.
- Boosting and AdaBoost
Boosting is an integration technique that creates a strong classifier from some weak classifiers. It first builds a model from the training data and then creates a second model to try to correct the error of the first model. Continue to add models until the training set is perfectly predicted or has been added to the upper limit.
AdaBoost is the first truly successful Boosting algorithm developed for the second category and the best starting point for understanding Boosting. The most famous algorithm currently built on AdaBoost is stochastic gradient boosting.
AdaBoost is often used with short decision trees. After the first tree is created, the performance of each training instance on the tree determines how much attention the next tree needs to put in this training instance. Unpredictable training data is given more weight, while instances that are easier to predict are given less weight. The models are created in sequence, and each model update affects the learning outcome of the next tree in the sequence. After all the trees have been built, the algorithm predicts the new data and weights the performance of each tree by the accuracy of the training data.
Because the algorithm is extremely focused on error correction, it is important to have a neat data with no outliers.