Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Atika Andrian Asmiran*.
Azzka Farikatun Nassa .
Ayuni Layinah.
Warto.
Informatika.
Dakwah dan Saintek.
UIN Prof.
Saifuddin Zuhri.
Indonesia Informatika.
Dakwah dan Saintek.
UIN Prof.
Saifuddin Zuhri.
Indonesia Informatika.
Dakwah dan Saintek.
UIN Prof.
Saifuddin Zuhri.
Indonesia Informatika.
Dakwah dan Saintek.
UIN Prof.
Saifuddin Zuhri.
Indonesia Article Info Keywords: Sentiment analysis.
Shopee.
Nayve Bayes, e-commerce Article history:
Received 12 Agustus 2025 Revised 25 Agustus 2025 Accepted 27 Agustus 2025 Available online 01 November 2025 DOI :
48144/suryainformatika.
* Corresponding author.
Atika Andrian Asmiran E-mail address:
atikaandrian2@gmail.
ABSTRACT
Technological developments have made various activities easier, including shopping.
Through applications, users can make transactions without having to meet face-to-face with sellers.
Although it provides convenience, this service does not always run perfectly.
One example of a well-known e-commerce application in Indonesia is Shopee.
This study analyzes the Shopee application because it significantly supports the Indonesian economy through digitizing trade.
Data from 3,000 Shopee reviews obtained from the Google Play Store website in 2025 were analyzed using Natural Language Processing with the Nayve Bayes method, classifying reviews into positive and negative sentiment categories.
From the labeling results, text from each sentiment category was extracted to generate essential and valuable information for decision-making.
The sentiment classification using the Nayve Bayes Classifier achieved an accuracy rate of 81.
The findings of this study can serve as a foundation for enhancing customer satisfaction and comfort, as well as improving the quality of Shopee's products and services, in line with consumer expectations.
INTRODUCTION
Information and communication technology development has rapidly impacted various aspects of life, including the digital commerce or e-commerce sector in Indonesia and Southeast Asia.
Shopee has become Indonesia's most dominant e-commerce platform, even leading the Southeast Asian market with a 46.
9% market share in 2024 .
This dominance reflects the high level of interest among the public in using Shopee compared to competitors like Tokopedia and Lazada.
However, a high number of users does not always indicate customer Complaints, such as delivery delays, product quality, and customer service issues, frequently appear in user reviews on the Google Play Store.
Therefore, it is crucial to analyze user satisfaction to understand and improve service quality and identify areas for By collecting and analyzing the sentiment from the reviews that have been written, we can obtain reliable data on the factors most influential to user satisfaction and provide an overview of the social and economic impact of this platform.
This situation raises an important question: to what extent does user sentiment toward Shopee reflect their satisfaction, and how can sentiment analysis help understand customer perceptions more deeply.
The Natural Language Processing (NLP) method has been widely used in sentiment analysis to automatically process and understand text .
Natural Language Processing (NLP) technology in sentiment analysis enables the efficient and accurate processing of large amounts of data to understand user opinions, emotional responses, and attitudes toward various products and services provided by e-commerce platforms.
This study uses a dataset of 2,696 user reviews of the Shopee app, comprising 1,349 positive and 1,347 negative reviews.
This method can identify hidden patterns in user reviews, pinpoint aspects of the service that need improvement, and provide a comprehensive overview of users' perceptions of the Shopee app .
Based on the principle that the Nayve Bayes Classifier has conditional independence regarding the output it One of the main advantages of using the Nayve Bayes Classifier method is its efficiency in terms of Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
training data requirements, as this algorithm can produce 3.
RESEARCH METHODOLOGY
accurate classification parameter estimates even with a This study uses a quantitative approach with descriplimited dataset .
In a textual context, using the Python tive and associative methods.
Based on the sentiment programming language is the primary choice for impleof reviews written by users on the Google Play Store menting the entire text mining process and stages, from platform, this study aims to analyze how satisfied Shopre-processing to sentiment classification using the Napee app users are .
Sentiment analysis uses a Natural yve Bayes Classifier method.
Language Processing (NLP)-based approach with the Nayve Bayes Classifier method.
Several stages in this This study aims to explain in depth and systematistudy include data collection, labeling, pre-processing, cally how this sentiment analysis process is carried splitting, modeling, and model evaluation.
This section out, from collecting Shopee user reviews on the aims to provide clear transparency about the stages Google Play Store platform, the training stage, and taken in this study so that readers can understand the testing using the Nayve Bayes Classifier, to create results.
meaningful data.
As such, the results of this study are expected to make an essential contribution to the development of e-commerce platforms in Indonesia, particularly in efforts to enhance user experience and customer satisfaction STUDY LITERATUR According to Muktafin.
, analyzing product reviews is essential.
This study uses the KNN and TF-IDF algorithms with an NLP approach to classify reviews of "instant hijabs" into two categories .
ositive and negativ.
Classification using the NLP approach achieved an accuracy of 76.
92%, while without NLP, it only achieved an accuracy of 69.
Furthermore, according to Sihombing.
, identifying problems that arise from customer reviews is essential.
However, reading and classifying each review takes a long time.
Because of this, we analyzed customer sentiment toward the Xiaomi Redmi Note 9 product sold on the Shopee website using the nayve bayes algorithm and found an accuracy rate of 85%.
Meanwhile.
Idris.
conducted a sentiment analysis on the Shopee application using the Support Vector Machine (SVM) algorithm.
The aim was to classify data from comments made by Shopee application users into positive and negative comments.
In this study, 3,000 reviews were collected, and the SVM results showed a high accuracy rate of 98%.
Another study by Utami.
used the Recurrent Neural Network (RNN) method to analyze user sentiment on the Shopee app based on review data.
The results showed that approximately 80% of Shopee app users had positive sentiments and 20% had negative sentiments, indicating that the data was unbalanced.
In this study, a pre-processing process was conducted using a combination of the synthetic minority oversampling technique (SMOTE) and the Tomek link method to address this The performance achieved was quite good, with an accuracy of 80%, which is better than sentiment analysis without pre-processing to handle imbalanced .
Figure 1.
Research Methodology Stages Scrapping Data Collecting user reviews from the Google Play Store using web scraping techniques.
Web scraping is one of the processes for collecting data from user reviews on the Google Play Store using the Python programming language .
Web scraping is performed using the Visual Studio Code code editor and the Python programming language obtained from the Shopee app URL on the Google Play Store, with data collected from 3,000 reviews in 2025.
By installing the GooglePlay-Scraper library, the existing review data is then extracted for analysis.
The results of the scraped data are then exported in CSV format.
Table 1.
Example of Scraping Results Document Dokumen Dok.
Dok.
Dok.
Dok.
Dok.
Dok.
Ulasan Score Label Belanja di shopee itu hemat dengan harga, tapi kualitas bgs tapi ga murahan Bagus, pengiriman cepat, jaga mutu barang Setiap buka shopee pasti muncul halaman tidak tersedia Tolong perbaiiki lagi aplikasinya Mudah aplikasinya, tdk membingungkan, banyak diskon dan Selama ada shopee kurang lebih 4 tahun sangat Positif Positif Negatif Negatif Positif Positif Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
meringankan ssaya dalam ber3.
Text Pre-Processing Dok.
Dok.
Dok.
Pokoknya happy shopping dech kalo di shopee Saya penjual non star yang jarang daper orderan Terima kasih shopee.
Sudah memudahkan berbelanja yang Positif Negatif Positif Using the Nayve Bayes Classifier approach, the text pre-processing stage is used in the text processing The text pre-processing process consists of several stages, including:
EDA (Exploratory Data Analysi.
Exploratory Data Analysis (EDA) is essential in understanding the characteristics and helping to analyze a dataset.
EDA helps identify patterns, relationships, anomalies, understand the structure, characteristics, and distribution of data, and detect potential problems such as duplicate data, missing data, class imbalance, or outliers .
The primary purpose of EDA is to gain initial insights into the data before running more complex statistical or machine learning models.
EDA standard methods include raw data collection, data processing, data cleaning, and feature visualization .
Labeling data Table 2.
Case Folding Process belanja di shopee itu hemat dengan harga, tapi kualitas bgs tapi ga murahan Labeling is used to assign sentiment categories or classes to each piece of text data, such as "positive," "negative," or "neutral," which serve as variables to be predicted by the model.
We labeled each text in the "score" column in this dataset.
The labels assigned are Positive and Negative.
Out of the 3,000 data points collected, if the 'score' column is 5 or 4, it is classified as Positive .
If the score is below 3, it is classified as Negative .
If the value is 3, it is removed.
Based on the established criteria, the following results were obtained:
1349 reviews classified as Positive sentiment 1347 reviews classified as Negative sentiment It can be seen that the ratio of positive to negative tweets is not significantly different .
Case Folding Case folding is a step to convert all capital letters in the text to lowercase letters so that the text is uniform and the data is consistent, thereby avoiding differences due to varying letter writing .
At this stage, there is also a character check from the beginning to the end of the data.
Characters other than letters and numbers, such as punctuation marks and spaces, are removed because they are considered delimiters .
The results of the case folding process can be seen in Table 2.
Text Cleaning Text cleaning is the process of cleaning text data prepared for further analysis.
Review data will be cleaned, simplified, and standardized at this stage so that the text data becomes more structured.
The steps involved in text cleaning include removing characters, numbers.
URLs, excess spaces, and emojis from unnecessary or irrelevant data.
This is necessary to improve data quality and make it easier for machine learning to read, use, and analyze the data.
The results of the text cleaning process can be seen in Table 3.
Table 3.
Text Cleaning Process belanja di shopee itu hemat dengan harga tapi kualitas bgs tapi ga murahan Normalisasi Normalization involves changing non-standard words to standard words and abbreviations to their original words .
For example:
"ga", "gue", and others, as well as the frequent use of word fragments, such as: "yg", "brp", 'bgm', "bgs", and others.
Words that are not normalized will be recognized as different For example, "bgs" and "bagus," which should have the same meaning, will have different meanings because they are written differently.
For this reason, words are normalized from non-standard to standard .
Table 4.
Normalization Process Figure 2.
Number of Positive and Negative Sentiments belanja di shopee itu hemat dengan harga tapi kualitas bagus tapi tidak murahan Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
Tokenize TF-IDF Weighting Tokenization is breaking down text or sentences TF-IDF aims to convert user review data text into nuinto smaller units called tokens.
In this context, merical form, so that the processed data can be used as tokens can be words, phrases, or characters in user input for machine learning, especially in the Nayve reviews for further analysis using Natural LanBayes Classifier approach.
In the TF-IDF process, each guage Processing (NLP) approaches.
The tokeniword in the review is weighted based on two main faczation process involves breaking down and sepators: how often the word appears in the review (Term rating text into individual units based on spaces or Frequenc.
and how rarely the word appears in the enpunctuation marks.
By performing tokenization, tire review data (Inverse Document Frequenc.
machine learning algorithms can convert text into a format easier to process.
The results of the toa.
Term Frequency Calculation (TF) kenization process can be seen in Table 5.
In this calculation, term frequency measures how often a word appears in a review, normalized by Table 5.
Tokenization Process dividing the number of occurrences by the total [AobelanjaAo.
AodiAo AoshopeeAo.
AoituAo.
AohematAo.
AodenganAo.
AohargaAo, number of words in the review.
Aotapi.
AokualitasAo,AobagusAo.
AotapiAo.
AotidakAo.
AomurahanA.
yceyc TF.
, .
= ycA Stopword Removal Stopword removal at this stage removes common words with irrelevant meanings in the sentiment analysis of Shopee user reviews.
These familiar words usually consist of conjunctions, pronouns, or words that do not contribute significantly to a sentence or text.
Examples include words like "in," "that," "with," and "but.
" This process enhances the accuracy of the Nayve Bayes Classifier approach in machine learning.
The results of the stopword removal process can be seen in Table 6.
Description:
Ft = number of occurrences of word .
in document .
- N = Total Number of words in the document .
With the example of TF value calculation in document review-1 as follows: ['shopping', 'shopee', 'savings', 'price', 'quality', 'good', 'cheap'].
If a calculation is performed from this data, the result will be shown in Table 9.
Tabel 6.
Penghapusan Kata Henti Table 9.
TF calculation results [AobelanjaAo,AoshopeeAo,AohematAo,AohargaAo.
AokualitasAo,AobagusAo,AomurahanA.
Kata Stemming Stemming in this stage is converting affixed words into root words or stems.
This stage is carried out to reduce the variation of words with the same meaning and simplify them, thereby improving the efficiency of data analysis in user reviews and making them easier to process in sentiment analysis based on Natural Language Processing (NLP).
The results of the stemming process can be seen in Table 6 .
Table 7.
Proses Stemming The stages of data splitting in the classification process, using the Hold-out method, involve dividing the dataset into two parts: training and testing data.
In the processed data, 80% was obtained for training and 20% for testing.
In this dataset division process, the Python programming language was used as follows.
Table 8.
Data Splitting Process X_train.
X_test, y_train, y_test = train_test_split .
f_baru [Aostemmed_textA.
, df_baru [AoLabelA.
, test_size = 0.
20, random_state = 42, stratify=y yceyc TF= ycA = 0.
= 0.
= 0.
= 0.
= 0.
= 0.
= 0.
All words in document-1 review have the same TF result because they have the same number, which is one each in 7 words.
[AobelanjaAo,AoshopeeAo,AohematAo,AohargaAo.
AokualitasAo,AobagusAo,Aomura.
Spliting Data Ft (Jumlah Inverse Document Frequency Calculation (IDF) In this process.
IDF is used to calculate reviews to determine how rarely a word appears in the existing data .
ya IDF .
= log yccyc Description:
- t = certain words in a document - D = total number of documents in a document collection Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
can assign a higher weight to words that appear - Dt = number of documents containing the infrequently but have a high value for distinguishword .
ing sentiment analysis.
The following is the TFAn example of the IDF equation calculation appliIDF calculation shown in Table 12.
cation in document review-1 is ['shopping', 'shopee', 'economical', 'price', 'quality', 'good', 'cheap'].
Table 12.
TF-IDF results The number of words appearing in a single dataset
Kata IDF
TF-IDF
has been obtained through the coding process in Belanja Table 10.
Shopee Hemat Harga Kualitas Murah Table 10.
Frequency of Words Appearing Kata Frekuensi Table 11.
IDF Results IDF.
= log( yccyc ) log( 347 )=0.
log( 746 )=0.
log( 39 )=1.
log( 138 )=0.
log( 23 )=1.
log( 202 )=0.
Nayve Bayes Model Training The Nayve Bayes classification algorithm is a classification technique that utilizes probability and statistical As a supervised learning method.
Nayve Bayes Multinomial requires data labeling before training.
The equation can be seen as follows:
log( 80 )=1.
Table 11 shows that the calculation of the word "Shopee" has the lowest IDF with a result of 0.
and a word frequency of 746 in the data, so that word is considered a common word and does not provide vital information in this sentiment analysis.
Meanwhile, the word "quality" has the highest IDF value of 1.
76, with 23 occurrences across the entire This value indicates that the word "quality" significantly influences the sentiment analysis.
the other hand, the words "price," "cheap," and "good" have moderate IDF values ranging from 82 to 1.
These words appear frequently but still hold relevant value in the sentiment analysis.
Based on the TF-IDF calculation results in Table 12, it is known that the word "quality" has the highest value of 0.
This indicates that the word is rarely used and appears, but it has a high weight in distinguishing the sentiment of Shopee app user reviews.
Thus, it can be interpreted that the quality of products and services provided on the Shopee app is crucial to users.
Words with the highest TF-IDF weights are more critical in distinguishing between positive and negative sentiments in user reviews.
This enables the Nayve Bayes classifier to focus on words that significantly impact the interpretation of user reviews on the Shopee app.
Table 10 shows the frequency or number of words in document-1.
Thus, the IDF results for the reviews in document-1 can be seen in Table 11.
Kata TF-IDF calculation Based on the results of the Term Frequency (TF) and Inverse Document Frequency (IDF) calculations in Tables 7 and 9.
TF-IDF calculations can be performed to determine the weight of each word in the Shopee app user review data on the Google Play Store.
By calculating the TF and IDF values and then multiplying them, the TF-IDF calculation ycE.
: Calculate the probability of the appearance of a set of feature values x when the target label value y is known.
Then multiply that value by ycE.
: probability of target label value y.
Next, that value is divided by ycE.
: the probability value of a set of feature values x .
Prediction & Evaluasi This test was conducted to determine the results of the model created and assess the model's ability to classify sentiment based on input text automatically.
The prediction results of the model were then evaluated using several performance metrics, namely the confusion matrix, accuracy, precision, recall, and F1-score .
Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
an AUC value of 0.
88 for both classes, both curves are well above the diagonal line, indicating performance that is far better than random guessing RESULTS AND DISCUSSION Figure 3.
Evaluation Metrics To evaluate the model that has been created, the confusion matrix and cross-validation score methods are used.
The confusion matrix is used to show the number of correct and incorrect classification results from the model.
Meanwhile, the cross-validation score is used to measure how well the model performs in predicting data that has not been seen before .
Figure 4.
Confusion Matrix In addition to using accuracy, precision, recall, and F1score as evaluation metrics, this study also utilizes AUCAeROC (Area Under the Curve-Receiver Operating Characteristi.
as an additional indicator to assess the performance of binary classification models.
In this study, sentiment analysis was conducted on user reviews of the Shopee application using the Nayve Bayes Classifier method based on Natural Language Processing (NLP).
The purpose of this study was to automatically classify user reviews into two sentiment categories, namely positive and negative.
The selection of Nayve Bayes was based on its advantages as a simple algorithm, fast in the training process, and efficient in the use of computational resources.
Additionally, this method has proven effective in processing text data with TF-IDF representation and is often used as a baseline in sentiment analysis research.
This process is expected to provide a clear picture of consumer satisfaction with the Shopee app.
The results of this sentiment analysis are not only beneficial for users in making purchasing decisions but are also important for sellers and app developers to evaluate and improve product and service quality based on user feedback.
The data used in this study was collected through web scraping from the Google Play Store platform, with 3,000 reviews of 1,349 positive and 1,347 negative reviews.
The review data was then processed through labeling and pre-processing, such as case folding, text cleaning, normalization, tokenization, stopword removal, and stemming, before being converted into numerical form using the TF-IDF method.
This process aimed to prepare the data for use in machine learning algorithms.
The model was then built and implemented using the Python programming language, with data split using the Hold-out technique, allocating 80% for training data and 20% for testing data.
The results of the model performance evaluation show that the Nayve Bayes method can produce an accuracy of 81.
30%, precision of 84.
21%, recall of 04%, and an F1-score of 80.
This indicates that the model has a pretty good ability to classify user reviews correctly.
However, several challenges were also identified during this analysis, namely that improving accuracy was difficult despite several optimizations.
This is thought to be due to the model's limitations in understanding complex linguistic contexts and the ambiguity of the review data .
Figure 5.
ROC Curve The ROC curve above shows the classification performance of the Nayve Bayes model in distinguishing between two sentiment labels .
ositive and negativ.
With This study demonstrates that applying the Nayve Bayes Classifier method in analyzing user sentiment in Shopee app reviews can provide accurate and helpful These results are expected to be a foundation for developing e-commerce services that are more responsive to user needs and expectations.
Sentiment Analysis of Shopee App User Reviews Based on Nayve Bayes Classifier Jurnal Surya Informatika Journal homepage: https://jurnal.
id/index.
php/surya_informatika P-ISSN: 2477-3042.
E-ISSN: 3026-3034
Vol.
No.
November 2025.
Pp.
Muktafin.
Kusrini, and E.
Luthfi.
CONCLUSION AND SUGGESTION AuAnalisis Sentimen pada Ulasan Pembelian Based on the results of the analysis attached, it can be Produk di Marketplace Shopee Menggunakan concluded that the application of the Nayve Bayes ClasPendekatan Natural Language Processing,Ay J.
sifier method in analyzing user sentiment reviews on the Eksplora Inform.
, vol.
10, no.
1, pp.
32Ae42.
Shopee app is capable of classifying reviews into two 2020, doi: 10.
30864/eksplora.
sentiment categories, namely positive and negative, out .
Sihombing.
Hannie, and B.
of 3,000 Shopee user reviews, comprising 1,349 positive Dermawan.
AuSentimen Analisis Customer reviews and 1,347 negative reviews, achieving an accuReview Produk Shopee Indonesia racy rate of 81.
This analysis process was supported Menggunakan Algortima Nayve Bayes by Natural Language Processing (NLP) techniques Classifier,Ay Edumatic J.
Pendidik.
Inform.
through text pre-processing.
TF-IDF weighting, and data 5, no.
2, pp.
233Ae242, 2021, doi:
separation using the Hold-out method.
These findings 29408/edumatic.
are expected to serve as a basis for evaluation by devel.
Idris.
Mustofa, and I.
Salihi, opers and sellers in improving services and product qualAuAnalisis Sentimen Terhadap Penggunaan ity on the Shopee platform.
Aplikasi Shopee Mengunakan Algoritma Support Vector Machine (SVM),Ay Jambura J.
ACKNOWLEDGEMENTS
Electr.
Electron.
Eng.
, vol.
5, no.
1, pp.
32Ae 35, 2023, doi: 10.
37905/jje.
The author would like to express his deepest gratitude to Utami.
AuAnalisis Sentimen dari Aplikasi all those who have supported and assisted in preparing .
Shopee Indonesia Menggunakan Metode this scientific paper.
Thank you to the supervising lecRecurrent Neural Network,Ay Indones.
Appl.
turer and teammates who have worked well together.
Stat.
, vol.
5, no.
1, p.
31, 2022, doi:
the context of e-commerce, user satisfaction is one of the 13057/ijas.
most crucial aspects to ensure service quality.
Therefore.
Huda.
Habrizons.
Satriawan.
reviews on platforms such as the Google Play Store are .
Iranda, and T.
Pramuda, "Analisis Usability considered important indicators, as they directly reflect Testing Menggunakan Metode SUS (System user experiences and opinions.
By analyzing these reUsability Scal.
Terhadap Kepuasan views through sentiment analysis, this work is expected Pengguna Aplikasi Shopee," Simkom, vol.
to provide benefits and contributions to the development 2, pp.
208Ae220, 2023, doi:
of science, particularly in the areas of sentiment analysis 51717/simkom.
and e-commerce technology.
Nurzaman.
Suarna, and W.
Prihartono.
AuAnalisis Sentimen Ulasan Aplikasi Threads REFERENCES