YAYASAN AKRAB PEKANBARU
Akrab Juara : Jurnal Ilmu-ilmu Sosial Volume 10 Nomor 4 Edisi November 2025 .
APPLICATION OF THE BERT MODEL IN MEASURING USER
PERCEPTION OF THE MAGIC INVESTMENT APPLICATION ON THE
GOOGLE PLAY STORE
-------------------------------------------------------------------------------------------------Tabina Fasya Benedicta.
Ade Setiawan.
Luthfi Indriyani.
Astrid Noviriandini.
Sandra Dewi Saraswati Universitas Bina Sarana Informatika (Naskah diterima: 1 October 2025, disetujui: 28 October 2.
Abstract Investment is one of the most effective ways to achieve long-term financial gains.
Nowadays, numerous digital platforms offer investment services, including the Ajaib application.
The growing public interest in investing has been driven by influencers and online advertisements, yet it has also led to the rise of fraudulent schemes and fake investment Therefore, evaluating user satisfaction through sentiment analysis of application reviews becomes essential.
This study aims to analyze user sentiments toward the Ajaib investment application based on reviews collected from the Google Play Store.
The dataset consists of Indonesian-language reviews from the period 2019Ae2024, processed using Google Colab and the BERT (Bidirectional Encoder Representations from Transformer.
The classification results yielded 1,393 reviews, comprising 696 positive and 697 negative sentiments, indicating that negative opinions were slightly more dominant.
The model achieved an accuracy of 85%.
F1-score of 85%, recall of 85%, and precision of 87%, demonstrating that the BERT algorithm performs effectively in sentiment analysis for investment-related applications.
Keywords: Web-Based Point of Sale System.
Software Architecture.
CV Digital Printing Abstrak Investasi merupakan salah satu cara paling efektif untuk meraih keuntungan finansial jangka Saat ini, banyak platform digital yang menawarkan layanan investasi, termasuk aplikasi Ajaib.
Meningkatnya minat masyarakat untuk berinvestasi didorong oleh influencer dan iklan daring, namun juga menyebabkan maraknya skema penipuan dan platform investasi Oleh karena itu, mengevaluasi kepuasan pengguna melalui analisis sentimen ulasan aplikasi menjadi penting.
Penelitian ini bertujuan untuk menganalisis sentimen pengguna terhadap aplikasi investasi Ajaib berdasarkan ulasan yang dikumpulkan dari Google Play Store.
Dataset ini terdiri dari ulasan berbahasa Indonesia dari periode 2019Ae2024, yang diolah menggunakan Google Colab dan algoritma BERT (Bidirectional Encoder Representations from Transformer.
Hasil klasifikasi menghasilkan 1.
393 ulasan, yang terdiri dari 696 sentimen positif dan 697 sentimen negatif, yang menunjukkan bahwa opini negatif sedikit lebih dominan.
Model ini mencapai akurasi 85%, skor F1 85%, recall 85%, dan presisi 87%, yang menunjukkan bahwa algoritma BERT bekerja efektif dalam analisis sentimen untuk aplikasi terkait investasi.
Kata Kunci: Sistem Point of Sale Berbasis Web.
Arsitektur Perangkat Lunak.
Percetakan Digital CV Copyright A 2025 by Author.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.
0 International License Tabina Fasya Benedicta, et.
INTRODUCTION
Nowadays, people have changed their financial management behavior, one of which is Many people want to invest because of the potential for long-term profits.
Besides long-term gains, investing also requires minimal activity, resulting in passive income, which is why many people are interested in investing.
In this study, the author was interested in analyzing sentiment regarding the Ajaib app to determine satisfaction based on reviews on the Google Play Store.
The growing public interest in investing has led to numerous influencers and advertisements promoting investment opportunities, leading to an increase in fraud and fraudulent investment cases.
"Ajaib is an investment app that makes it easy for users to invest in mutual funds and stocks online.
Furthermore.
Ajaib also provides an investment learning platform for both beginners and experienced users" (Ajaib, 2.
The problem arises from measuring and analyzing satisfaction levels in reviews using the BERT algorithm, which can separate data from public trust in investing in the app, which contains negative and positive reviews, aimed at assessing the emotional state of app users.
In research conducted by Alifa and al Faraby.
BERT is a deep learning model that has achieved outstanding results in various natural language translation tasks.
Because it has six layers of transformers layered on top of each encoder and decoder, the training process is very complex, the configuration is very difficult, the training time is very long, and the cost is very high.
However.
Google's BERT pre-trained model is available as open source and can be used without first building the model.
BERT processing begins with the word and embedding representations from the embedding layer.
To create new intermediate representations, each layer uses telescoping attention calculations on the word representations from the previous layer (Kurniawan et al.
, 2.
Sentiment analysis in the Ajaib application can help companies identify strategic investors and increase investor confidence.
By analyzing customer opinions and reviews, companies can determine the level of customer satisfaction.
Sentiment analysis using the BERT algorithm can play a role.
It is crucial to improve user service in the investment sector.
Currently, many investors are unsure where to invest and lack confidence due to the abundance of news reporting investment fraud.
This situation has given rise to a crucial role for social media, particularly Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
the Google Play Store, as a source of information regarding opinions on the Ajaib app.
Ajaib app review data on the Google Play Store is unstructured and uses much informal language, necessitating a process for analyzing the review data.
In a study conducted by Tarisa Putri Cahyani and Danang Teguh Qoyyimi.
Sc.
Ph.
, using a similar case, they conducted sentiment analysis on the Halodoc app on the Google Play Store using the Bidirectional Encoder Representations from Transformers (BERT) algorithm and the Long Short Term Memory Network (LS-TM) algorithm.
Three data labeling methods were used: automatic data labeling using a Valence Aware Dictionary and Sentiment Reasoner (VADER) lexicon, manual data labeling using user ratings, and manual labeling by researchers.
Research results showed that the BERT algorithm, labeled through user ratings, was the most effective for implementation (Cahyani & Qoyyimi, 2.
Another study, using the same case, was conducted by Ekka Pujo Ariesanto Akhmad in Sentiment Analysis of DLU Ferry App Reviews on the Google Play Store.
This study used 1,575 reviews, consisting of positive, neutral, and negative sentiments.
This study measured the performance of the Bidirectional Encoder Representations from Transformers (BERT) method in sentiment classification using a pretrained IndoBERT-base model with fine-tuning The test results obtained an accuracy of 86% (Akhmad, 2.
Another study also discussed the BERT model, entitled "Sentiment Analysis of Customer Reviews of the Ruang Guru App Using the BERT Method," conducted by Raden Mas Rizqi Wahyu Panca Kusuma Atmaja and Wiyli Yustanti.
This model obtained an F1 score of 98.
9% with a 70:30 training and test data ratio.
The model was then evaluated, yielding an accuracy of 99%, a precision of 64.
13%, and a recall of 60.
The sentiment percentage criterion was weighted at 99% and was declared valid based on averaging over 10 epochs with consistent improvement (Atmaja & Yustanti, 2.
This demonstrates the BERT algorithm's high accuracy in processing text, possessing a well-structured understanding of language meaning, thus addressing some language processing challenges.
The results of this study are expected to instill confidence in investors regarding investment platforms and assist companies in improving their services.
Therefore, the researchers intend to conduct a study entitled: "Sentiment Analysis of the Ajaib Application to Determine Satisfaction Based on Opinions Using the BERT Algorithm.
Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
II.
RESEARCH METHODS
In the research stage regarding sentiment analysis of the magic application to determine satisfaction based on opinions, the diagram can be described as follows:
Figure 1.
Research Stages Problem Identification and Needs Analysis:
This study examines the problems encountered during the implementation of the BERT algorithm for sentiment analysis on the Ajaib app and its data grouping process.
Data Collection The data collection process used a scraping technique by collecting Ajaib app user reviews from the Google Play Store.
The review data collected consisted of 1,393 ungrouped Data Labeling Data labeling is the process of dividing reviews into positive and negative categories.
Labels are assigned to the positive and negative categories in the data.
Data Pre-Processing Pre-processing is the pre-processing of the collected data to maximize the results of the algorithm application, thus facilitating the subsequent steps.
There are several stages in data processing, as follows:
Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
Data Cleaning Data cleaning is the process of removing incomplete and inconsistent data from the The primary goal of data cleansing is to ensure the quality of the data used for Tokenization Tokenization is the process of breaking down words, phrases, or specific symbols.
Stopwords Stopwords are the process of removing frequently occurring text that doesn't add meaning to the text.
Steaming Steamming is the process of converting different words but having the same meaning into text.
Application of the BERT Algorithm In this study, the author used the BERT algorithm to solve the research problem.
The BERT algorithm was created as a pre-trained model, so it didn't require a large amount of training data because the BERT algorithm was trained on a large amount of data.
Results This process ensures that the BERT algorithm can produce accurate data using the method used.
Several tests have shown how well the BERT method predicts sentiment Using the BERT algorithm ensures its accuracy, resulting in good predictions for the Ajaib app's sentiment analysis results to determine satisfaction based on opinions.
RESEARCH RESULTS
Data Collection Data collected using the following scraping technique is the data collection method:
Installing Figure 1.
Installation Script Import Google Scraper.
Pandas, and Numpy.
Figure 2 Import Script Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
Request the review data you wish to retrieve.
Figure 3 Scraping Script Creating a data frame from the data.
Figure 4 Data Frame Script Displays the amount of data obtained and then saves it to a CSV file.
Figure 5 Script Displays the Number Data Table 1 Scarping Result Data So far so good because I'm just a beginner Oh my gosh, this is the first time I've played an investment app.
I swear, processing just one order takes so long, even during working hours.
Very easy to understand thank you for helping me/us to invest This app is really top notch Please, make the app magical, don't change it too much.
It's just frustrating.
I've created a waiting list and it's gone.
What can I do? A version from 2023 or earlier would be better.
The chat forum is also getting more and more hidden.
What needs to be improved is the server.
Data verification takes a really long time, it's been more than 6 months.
Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
good, complete and easy to understand code leveral andrxm t9 I've been investing in Ajaib stocks for over a year, but I still don't know the customer service.
The portfolio was closed just to request investor data.
It's inaccessible due to difficulties uploading an ID photo.
It's strange that we've already topped up and bought/sold shares, but we're being asked to upload an ID photo again.
Meanwhile, uploading the ID photo keeps failing.
What should I do? I topped up my balance, why didn't it come in full? Only 50% came in, but in the full history, there was no response in the chat.
It's a loss.
I haven't invested yet, just deposited and it's already like this.
Data Labeling Data labeling to separate positive and negative reviews, here is how to label data:
Importing into pandas, numpy and google colab.
Figure 6.
Labeling Import Script Reading CSV data.
Figure 7 Script to read data Apply labeling Figure 8 Labeling Script Applying the labeling function.
Figure 9.
Labeling Function Script Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
Saving to CSV.
Figure 10: CSV Saving Script Table 2: Labeling Results Sentiment Positive So far so good because I'm just a beginner Negative Oh my gosh, this is my first time using an investment app, and I swear, processing just one order takes so long, even during working hours.
Positive Very easy to understand Positive thank you for helping me/us to invest Positive This app is really top notch Negative Please, stop changing the app so much.
It's frustrating.
I've created a waiting list and it's gone.
What can I do? A version from 2023 or earlier would be better.
The chat forum is also getting more and more hidden.
What needs to be improved is the server upgrade.
Negative Data verification takes a really long time, it's been more than 6 months.
Positive good, complete and easy to understand code leveral andrxmt9 Negative I've been investing in Ajaib stocks for over a year, but I still don't know the customer The portfolio was closed just to request investor data.
It's inaccessible due to difficulties uploading an ID photo.
It's strange that we've already topped up and bought/sold shares, but we're being asked to upload an ID photo again.
The ID photo upload keeps failing.
What should I do? Negative I topped up my balance, why didn't it come in full? Only 50% came in, but in the full history, there was no response in the chat.
It's a loss.
I haven't invested yet, just deposited and it's already like this.
Data processing Import the required libraries.
Figure 11: Import Data Processing Script Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
Entering Data Figure 12 Data Entry Script Removing duplicate text.
Figure 13 Duplicate Text Script Exploratory data analysis.
Figure 14 EDA Script Cleaning up text in a data frame.
Figure 15 Script to Delete Data Frame Text Clearing text of irrelevant words Figure 16 Script for Deleting Text Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
Deleting words in a text field.
Figure 17 Script to Delete Text Columns Tokenization Figure 18 Tokenization Script Table 3 Tokenization Results Sebelum sejauh,ini,bagus,soalnya,baru, pemula sejauh ini bagus soalnya baru pemula astaga,baru,kali,ini,main,apk, investasi, astaga baru kali ini main apk investasi sumpah,nge,proses,satu,orderan,aja,lama,beut,lama Sumpah nge proses satu orderan aja lama ,dahal,jam,kerja,loh beut lama dahal jam kerja loh sangat,mudah,difahami sangat mudah difahami terima,kasih,ajaib,sudah,membantu, terima kasih ajaib sudah membantu saya saya,kami,untuk, berinvestasi kami untuk berinvestasi emang,markotop,sih,inii,app emang markotop sih inii app Stopwords.
Figure 19 Stopword Script Table 4 Stopword Results Sesudah bagus,pemula astaga,kali,main,apk,investasi,sumpah, nge,proses, orderan,aja,beut, dahal,jam, kerja,loh mudah,difahami terima,kasih,ajaib,membantu,berinvestasi emang,markotop, sih,inii,app Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Sebelum Sejauh ini bagus soalnya baru pemula astaga baru kali ini main apk investasi sumpah nge proses satu orderan aja lama beut lama dahal jam kerja loh sangatmudah difahami terima kasih ajaib sudah membantu sayakami untuk berinvestasi emang markotop sih inii app Tabina Fasya Benedicta, et.
Stemming.
Figure 20: Stemming Script Table 5: Stemming Results Sesudah Sebelum .
agus,pemul.
sejauh ini bagus soalnya baru pemula .
staga,kali,main,apk,investasi,sumpah,nge astaga baru kali ini main apk investasi sumpah , proses,orderan,aja,beut,dahal,jam,kerja, nge proses satu orderan aja lama beut lama dahal jam kerja loh .
udah,difaham.
sangat mudah difahami .
erima,kasih,ajaib,membantu,berinv terima kasih ajaib sudah membantu saya kami untuk berinvestasi .
mang,markotop,sih,inii,ap.
emang markotop sih inii app Lemmatization of Indonesian language.
Wordcloud To visualize frequently occurring words in a text, here's how to display a wordcloud:
Figure 22 Wordcloud Script Figure 23 Wordcloud Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
BERT Implementation Import libraries and modules into BERT.
Figure 24 BERT Implementation Import Script Implementing the BERT algorithm.
Figure 25 BERT Implementation Script Create a CSV file Figure 26.
BERT CSV Creation Script Confusion Matrix The data evaluation stage involves comparing the model's predictions with the actual labels of the data being tested.
The following is a confusion matrix:
BERT prediction evaluation.
Figure 27 BERT Prediction Script Akrab Juara : Jurnal Ilmu-ilmu Sosial Vol.
No.
4 Tahun 2025 Tabina Fasya Benedicta, et.
Results.
Figure 28 BERT Result Script IV.
CONCLUSION
In the study, entitled "Sentiment Analysis of the Ajaib App to Determine Satisfaction Based on Opinions Using the BERT Algorithm," 1,393 data sets were used, with 696 positive reviews and 697 negative reviews.
This study concluded that Ajaib app users had more negative reviews.
The results showed an accuracy of 85%, an F1 Score of 85%, a recall of 85%, and a precision of 87%.
This study demonstrated the relationship between information and user satisfaction, enabling the company to understand user needs for future evaluation.
This study found a significant number of negative reviews, which could impact user trust in the Ajaib app.
Therefore, the Ajaib app must improve its system in the future to garner more positive reviews and increase user trust.
REFERENCES