#
Introduction
The simplest solution is often the best, especially when solving a specific machine learning problem.
I have seen many people use large language models (LLMs) and generative AI systems for tasks like time series forecasting, image classification, and tabular prediction. In many cases, a simple machine learning model can solve the same problem faster, cheaper, and with much less complexity.
For data scientists, knowing the core machine learning algorithms and when to use them is still an essential skill. In this guide, we will cover seven algorithms every data scientist should know, briefly explain how they work, and show how to use them in Python.
#
1. Linear Regression
Linear regression is one of the simplest and most widely used machine learning algorithms for predicting continuous numerical values. It can be used for tasks such as predicting house prices, estimating monthly revenue, or forecasting energy consumption.
The model works by learning the relationship between the input features and the target value. It tries to find a straight-line relationship that produces predictions as close as possible to the actual values in the training data.
During training, the model learns how much each feature contributes to the final prediction. Once trained, it can use these learned relationships to make predictions on new data.
from sklearn.linear_model import LinearRegression
model = LinearRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)Here, fit() trains the linear regression model using the training data. The predict() method then uses the learned relationships to generate predictions for the test data.
Linear regression is fast, easy to implement, and simple to interpret. It is also commonly used as a baseline model to compare against more advanced regression algorithms.
#
2. Logistic Regression
Logistic regression is one of the most widely used algorithms for classification. It is commonly used for problems with two possible outcomes, such as spam or not spam, customer churn or retention, and fraudulent or legitimate transactions.
The model works by estimating the probability that an observation belongs to a particular class. It learns how each input feature affects that probability and uses the result to assign a class.
Despite its name, logistic regression is a classification algorithm. It is fast, relatively easy to interpret, and a strong baseline for many classification problems.
from sklearn.linear_model import LogisticRegression
model = LogisticRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)Scikit-learn applies regularization by default, which helps control model complexity and reduce overfitting.
#
3. LightGBM
LightGBM is a gradient boosting algorithm designed for tree-based machine learning. It is especially effective for structured or tabular datasets.
The model builds decision trees one after another. Each new tree focuses on improving the errors made by the existing trees, and their predictions are combined to produce the final result.
LightGBM uses histogram-based learning, which groups continuous feature values into bins. This can reduce memory usage and make training more efficient, particularly on larger datasets.
from lightgbm import LGBMClassifier
model = LGBMClassifier()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)This example uses the LGBMClassifier for classification. LightGBM also provides LGBMRegressor for regression tasks.
It also supports parallel, distributed, and GPU training, making it a popular choice for large-scale tabular machine learning.
#
4. XGBoost with Histogram Trees
XGBoost is another popular gradient boosting algorithm for structured data. It is widely used for classification, regression, and ranking problems.
Like LightGBM, XGBoost builds decision trees sequentially. Each new tree tries to correct errors in the current predictions, gradually improving the model.
Instead of relying on one large decision tree, XGBoost combines many smaller trees to produce a stronger final prediction.
from xgboost import XGBClassifier
model = XGBClassifier(tree_method="hist")
model.fit(X_train, y_train)
y_pred = model.predict(X_test)The tree_method="hist" setting uses histogram-based tree construction. Feature values are grouped into bins before XGBoost searches for useful splits, making tree building more efficient.
XGBoost is flexible, reliable, and remains one of the strongest algorithms for many tabular machine learning problems.
#
5. Random Forest
Random forest is an ensemble machine learning algorithm that combines multiple decision trees.
Instead of relying on a single tree, it trains many trees using different samples of the training data and subsets of the available features. Their predictions are then combined.
For classification, the trees vote on the predicted class. For regression, their predictions are averaged. Combining multiple trees usually makes the model less likely to overfit than a single decision tree.
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier(
n_estimators=100,
random_state=42
)
model.fit(X_train, y_train)
y_pred = model.predict(X_test)The n_estimators=100 setting tells random forest to build 100 decision trees.
Random forest is easy to use, works well on many tabular datasets, and can also provide feature importance scores to help understand which inputs influence its predictions.
#
6. Long Short-Term Memory Networks
Long short-term memory networks, or LSTMs, are a type of recurrent neural network designed for sequential data.
An LSTM processes a sequence step by step while maintaining information from earlier steps. It uses internal memory and gates to decide what information to keep, update, or ignore.
This allows earlier observations to influence later predictions, making LSTMs useful when the order of the data matters. Examples include sales forecasting, traffic prediction, sensor readings, and other time series problems.
from tensorflow import keras
from tensorflow.keras import layers
model = keras.Sequential([
keras.Input(shape=(X_train.shape[1], X_train.shape[2])),
layers.LSTM(64),
layers.Dense(1)
])
model.compile(
optimizer="adam",
loss="mean_squared_error"
)
model.fit(X_train, y_train, epochs=20)
y_pred = model.predict(X_test)The LSTM(64) layer contains 64 LSTM units that process the sequence. The Dense(1) layer produces a single numerical prediction.
LSTM input data is usually arranged as samples × time steps × features. These models can learn complex sequential patterns but often require more data and computation than traditional machine learning algorithms.
#
7. K-Means Clustering
K-means is an unsupervised machine learning algorithm that groups similar observations into clusters. Unlike classification, it does not require labeled training data.
The algorithm starts with a selected number of cluster centers called centroids. Each observation is assigned to its nearest centroid, and the centroids are recalculated based on the observations in each group.
This process repeats until the clusters stop changing significantly.
from sklearn.cluster import KMeans
model = KMeans(
n_clusters=3,
n_init=10,
random_state=42
)
clusters = model.fit_predict(X)The n_clusters=3 setting tells k-means to create three groups. The n_init=10 setting runs the algorithm with multiple centroid initializations and keeps the best result.
K-means is useful for discovering patterns in unlabeled data, such as customer segments or groups with similar behavior. Its main limitation is that the number of clusters must be selected before running the algorithm.
#
Final Thoughts
These algorithms became popular for a reason, and they are still used in modern AI applications today. Even in my own projects, I often return to traditional machine learning because it gives me a better solution for the problem I am trying to solve.
These models are faster, easier to implement, and usually require far less CPU, RAM, and infrastructure. Somewhere along the way, we have almost forgotten that simplicity is often the best solution.
Not every problem requires an LLM or a generative AI model. There are many specialized tasks where a simple machine learning algorithm can do the job without fine-tuning a huge model or building a complex AI system.
The important skill is not always choosing the newest model. It is choosing the right model for the problem.
Abid Ali Awan (@1abidaliawan) is a certified data scientist professional who loves building machine learning models. Currently, he is focusing on content creation and writing technical blogs on machine learning and data science technologies. Abid holds a Master’s degree in technology management and a bachelor’s degree in telecommunication engineering. His vision is to build an AI product using a graph neural network for students struggling with mental illness.
