The dataset used in this study consists of 150 entries and 14 columns. Each column represents a variable providing information about individuals. Table 1 presents an overview of the key variables, their meanings, and types.

Table 1

Variable Name	Meaning	Type
Level	Academic level of the individual	Continuous
Faculty	Faculty affiliation	Categorical
Gender	Gender of the individual	Categorical
Age	Age of the individual	Continuous
Family Size	Size of the family	Continuous
Obese	Obesity status (Target)	Categorical
Income	Individual’s income level	Continuous
Daily Eating	Daily eating habits	Categorical
Fruit Intake	Frequency of fruit intake	Continuous
BMI Aware	Awareness of BMI	Categorical
Sleeping Hours	Hours of sleep per day	Continuous
Exercise Freq	Frequency of exercise	Continuous
Need help	Indicates if help is needed	Categorical
Need app	Indicates the need for an app	Categorical

The dataset provides a mix of continuous and categorical variables, with “Obese” serving as the target variable indicating the obesity status of individuals. These variables lay the foundation for subsequent modeling and analysis.

SMOTE is a powerful method used to address class imbalance in datasets by generating synthetic samples of the minority class ^13,14. It works by synthesizing new instances of the minority class by interpolating between existing minority class instances, thus creating a more balanced dataset. SMOTE helps to mitigate the problem of biased classification models that tend to favor the majority class due to its higher representation in the dataset. By introducing synthetic samples, SMOTE enhances the diversity of the minority class, allowing machine learning algorithms to better learn the underlying patterns and improve classification performance ^15-17. This technique has been widely adopted in various fields, including healthcare ^18-20, finance ^21-23, and image recognition ^24,25, where imbalanced datasets are common, contributing to more accurate and robust predictive models. The steps on how SMOTE works are shown below:

Identify the Minority Class: Determine the minority class in the dataset that needs over-sampling.

d x , y =   ∑ i = 1 n x 1 - y 1 2 (1)

Where (𝑥 = 𝑥₁, 𝑥₂, ⋯, 𝑥_𝑛) 𝑦 = (𝑦₁, 𝑦₂, ⋯, 𝑦_𝑛) are feature vectors.

Generate Synthetic Samples: Randomly select one of the 𝑘-nearest neighbors, denoted as 𝑥 _neighbor. Create a synthetic sample 𝑥_syntetic by interpolating between 𝑥 and 𝑥_neighbor:

x s y n t e t i c = x +   δ ∙ x n e i g h b o r - x (2)

where δ is a random number between 0 and 1.

Figure 1 below show the Pseudo code for SMOTE and Figure 2 show an illustration of SMOTE. Apart from its application in prediction of heart disease, SMOTE has widely been used in other health events where prediction is warranted. For instance, a study ²⁶ used the SMOTE technique to balance the dataset for improving the prediction performance of heart disease using the Decision Tree algorithm on the Cleveland Heart Disease Dataset. The SMOTE technique addressed data imbalances between minority and majority classes. Experimental results showed that balancing the data with SMOTE improved decision tree classification accuracy from 73.3% to 91.4%, an increase of up to 18.1%. A study ²⁷ used the SMOTE technique to balance the dataset and improve the accuracy of rule induction and decision tree models for predicting kidney disease using data from Apollo Hospitals, Tamil Nadu, India. The initial imbalanced dataset hindered model accuracy but applying SMOTE minimized class variation. Experimental findings showed an average accuracy improvement to 98.73%. This method can also enhance accuracy in other imbalanced datasets and be applied in Big Data contexts using Hadoop and MapReduce. A study ²⁸ proposed a two-step approach to improve predictive accuracy in healthcare, addressing the limitations of basic SMOTE. First, modified SMOTE techniques Distance-based SMOTE was used to reduce class imbalance and showed improved accuracy over basic SMOTE. Second, a Stacking Ensemble Framework combining machine learning, deep learning, and ensemble algorithms significantly increased accuracy to 96-97% for various datasets. This framework was validated using the Framingham dataset, Wisconsin Hospital data, and Novel Coronavirus 2019 dataset.

Figure 1

Figure 2

The classification models chosen for this analysis include Logistic Regression, 𝐾-Nearest Neighbors (KNN), Random Forest and Deep Learning

Logistic Regression is a widely used linear classification algorithm that models the probability of a binary outcome. It is particularly suitable for predicting binary variables, such as whether an individual is obese or not. The logistic regression model predicts the log-odds (logit) of the probability of the positive class ²⁹ In logistic regression, the hypothesis function ℎ_θ(𝑥) is defined as:

h θ x = g ( θ T x ) (3)

where the function 𝑔 is the sigmoid function, represented as:

g z =   1 1 + e - z   (4)

The sigmoid function ensures that the output of the hypothesis lies between 0 and 1, making it suitable for binary classification tasks.

Regarding the loss function in logistic regression, it is commonly defined as the logistic loss or binary cross-entropy loss. For a single training example with true label y and predicted probability ℎ_θ(𝑥), the logistic loss is computed as:

L o s s h θ x , y = - y log ⁡ h θ x - 1 - y   l o g ⁡ ( 1 - h θ x ) (5)

The objective is to minimize this loss function over the entire training dataset. The loss function penalizes the model more heavily for incorrect predictions, particularly when the predicted probability diverges significantly from the true label. Minimizing this loss function effectively adjusts the model parameters θ to improve the accuracy of predictions in logistic regression.

The 𝑘-nearest neighbors (KNN) algorithm is a simple, yet effective method used for classification tasks in machine learning. It operates based on the principle of similarity: If an instance is like other instances in the dataset, it is likely to belong to the same class ^30,31.

Here’s how the KNN algorithm works:

Training Phase: The algorithm stores all the available training data points and their corresponding class labels.

The working framework of KNN involves calculating the Euclidean distance between the new instance (denoted as 𝑥) and each point in the training dataset (denoted as 𝑦). The Euclidean distance between two points 𝑥 and 𝑦 in an 𝑛-dimensional space can be calculated using the following formula:

E u c l i d e a n   D i s t a n c e =   ∑ i = 1 n ( x 1 - y 1 ) 2 (6)

Where:

𝑥₁ and 𝑦₁ are the 𝑖-th dimensions of points 𝑥 and 𝑦 respectively.

𝑛 is the number of dimensions (or features) in the dataset.

By calculating the Euclidean distance between the new instance and each point in the training dataset, the KNN algorithm identifies the 𝑘-nearest neighbors, which are then used for classification.

Bayesian classification, a supervised learning and statistical method, operates on an underlying probabilistic model, allowing for the quantification of uncertainty in outcomes. It effectively addresses predictive problems by determining probabilities associated with various outcomes ^31,32. In Naive Bayes classification, we aim to predict the class label 𝑦 based on the features 𝑥₁, 𝑥₂, ⋯, 𝑥_𝑝. We assume that the features are conditionally independent given the class label y, which means that:

P x 1 ,   x 2 , ⋯ , x p y = c = P x 1 , x 2 , ⋯ , x p y = c   × x 2 x 3 ⋯ x p ,   y = c × ⋯ ×   P x p y = c

We can then apply Bayes’ theorem to calculate the probability of each class given the observed features:

P y = c x 1 ,   x 2 , ⋯ , x p = P x 1 ,   x 2 , ⋯ , x p y = c × P ( y = c ) P ( x 1 , x 2 , ⋯ x p )

To classify a new instance with features 𝑥₁, 𝑥₂, ⋯ 𝑥_𝑝, we select the class label that maximizes the posterior probability:

y ^ =   P y = c x 1 ,   x 2 , ⋯ , x p = c arg ⁡ m a x   P x 1 ,   x 2 , ⋯ , x p y = c × P ( y = c ) c arg ⁡ m a x

The classical procedure for Naive Bayes is as follows:

Calculate Class Priors: Calculate the prior probabilities 𝑃 (𝑦 = 𝑐) for each class c based on the frequency of class labels in the training dataset.

Deep learning for classification is a sophisticated approach that leverages neural networks with multiple layers to categorize input data into distinct classes (Figure 3). The process begins with data preprocessing, where input features are normalized or standardized to ensure uniformity in scale and the dataset is divided into training, validation, and testing sets for evaluation. Subsequently, the architecture of the neural network is defined, specifying the number of layers, neurons per layer, and activation functions. In classification tasks, the output layer typically comprises neurons equivalent to the number of classes, with a sigmoid or softmax activation function employed to generate class probabilities ³³.

Figure 3

Figure 3 show an illustration of deep learning neural network. During the forward propagation phase, input data traverse through the neural network, undergoing computations layer by layer to produce the network’s output. Each layer computes its output through a linear transformation followed by an activation function. To quantify the disparity between predicted output and true labels, an appropriate loss function is selected. For classification tasks, common choices include categorical cross-entropy for multi-class classification and binary cross-entropy for binary classification. The backpropagation process calculates gradients of the loss function with respect to model parameters and updates these parameters using an optimization algorithm, like stochastic gradient descent (SGD), Adam, or RMSprop ³⁴.

Training ensues by iteratively feeding batches of data through the network, computing loss, and adjusting parameters through backpropagation. Regular monitoring of the model’s performance on the validation set helps prevent overfitting and fine-tune hyperparameters. Finally, the model is evaluated on the test set to gauge its performance on unseen data. Metrics such as accuracy, precision, recall, and F1-score are computed to assess its efficacy. Activation functions, including sigmoid and ReLU play crucial roles in determining the output of neurons within the network, contributing to its overall classification capability. Figure 4 show the activation functions used in deep neural networks.

Figure 4

Due to the small size of the data, we utilized cross-validation to avoid overfitting. Cross-validation is a technique used to assess the performance of machine learning models by dividing the dataset into multiple subsets, or folds. Each fold is used as a validation set, while the rest of the data is used for training. This process is repeated multiple times for our case we will use 5 folds, with each fold serving as the validation set exactly once. The main advantage of cross-validation is that it provides a more robust estimate of the model’s performance compared to a single train-test split. In this scenario, train-test split was not utilized due to the relatively small size of the dataset, which could lead to high variability in model performance estimates. By using cross-validation, we ensure that each data point is used for both training and validation, leading to a more reliable assessment of the model’s generalization ability ^35,36,37. The formula for calculating the average metric across all folds in cross-validation can be expressed as:

A v e r a g e   M e t r i c =   1 k ∑ i k M e t r i c i

Where:

𝑘 is the number of folds in cross-validation.

The Figure 5 below illustrates how cross validation works in training and validation of our classification models.

Figure 5

In this section, we define and discuss key evaluation metrics used for assessing the performance of classification models.

The distribution of the target variable, ’obese,’ reveals important insights into the composition of the dataset. In our analysis, the target variable represents whether an individual is classified as obese or not. The distribution of this variable is crucial for understanding the prevalence of obesity within the dataset. Upon examining the distribution, we observe that the dataset exhibits an imbalanced distribution in terms of the ’obese’ classes. There are two possible classes: ’Not Obese’ and ’Obese.’ The imbalanced distribution implies that one class significantly outnumbers the other. The Figure 6 shows the plot of target variable Distribution before SMOTE.

Figure 6

To address the class imbalance and enhance the model’s ability to accurately predict both classes, oversampling techniques such as Synthetic Minority Over-sampling Technique (SMOTE) were employed. SMOTE generates synthetic instances of the minority class by interpolating between existing instances. This augmentation helps balance the class distribution and improves the model’s generalization to the minority class, ultimately contributing to more robust predictions. Figure 7 shows the class distribution of the target after SMOTE.

Figure 7

In the subsequent sections, we will assess the impact of before oversampling and after oversampling on model performance through various metrics and visualizations.

Table 2 shows the performance of the classification models on various metrics before SMOTE as percentage. Table 3 shows the performance of the classification models after SMOTE. Table 4 show the metric improvement of the classification models as percentage. Figure 9 shows visualization of metrics deviation after SMOTE and Figure 8 show visualization of the metrics before and after SMOTE.

Table 2

Model	Accuracy	Balanced Accuracy	Sensitivity	Specificity	Precision	F1-Score
Logistic Regression	80.8	55.8	18.7	18.7	28.3	22.4
Naive Bayes	76.7	62.2	39.3	39.3	35.0	34.4
KNN (k=5)	80.0	48.0	0.0	0.0	0.0	0.0
Deep Learning	76.0	53.4	19.3	87.4	32.4	19.7

Table 3

Model	Accuracy	Balanced Accuracy	Sensitivity	Specificity	Precision	F1-Score
Logistic Regression	72.0	72.4	75.4	75.4	71.7	72.2
Naive Bayes	79.0	78.9	87.2	87.2	75.0	80.5
KNN (k=5)	75.5	76.3	96.0	96.0	68.9	79.6
Deep Learning	82.5	82.8	89.1	76.4	78.4	83.2

Table 4

Model	Accuracy	Balanced Accuracy	Sensitivity	Specificity	Precision	F1-Score
Logistic Regression	-8.8	+16.6	+56.7	+56.7	+43.4	+47.8
Naive Bayes	+2.3	+16.7	+47.9	+47.9	+40.0	+46.1
KNN (k=5)	-4.5	+28.3	+96.0	+96.0	+68.9	+79.6
Deep Learning	+6.5	+29.4	+69.8	-10.9	+45.9	+66.7

Figure 8

Figure 9

Tree-based machine learning approaches, including Logistic Regression, Random Forest (RF), and Extreme Gradient Boosting (XGBoost), were employed to classify obesity levels ³⁸. The study found that LR performed best across most metrics after addressing class imbalance using SMOTE-NC and feature selection via Recursive Feature Elimination. Our findings similarly highlight the robustness of LR in the context of obesity prediction, with SMOTE significantly enhancing various performance metrics. A study ⁹ focused on preprocessing an obesity dataset and used SVM, RF, and Decision Trees for classification. The RF model showed the highest prediction accuracy (96%). This aligns with our findings, which show that RF is effective in obesity prediction, though our use of SMOTE further improved the model’s performance metrics. Assessed ML methods, including Logistic Regression, Classification and Regression Trees, and Naive Bayes, to predict obesity ³⁹. Logistic regression showed the highest performance, which is consistent with our study. The application of SMOTE in our research also addressed data imbalance effectively, leading to significant improvements in sensitivity, specificity, and balanced accuracy.