Feature-Level Usability and User Retention Prediction in TikTok Shop using Naïve Bayes

Jessica Gabriel Lantang, Evi Maria

Abstract


User retention is a critical factor in the sustainability of social commerce platforms; however, the relationship between usability and retention remains inconsistent across various studies. This study aims to analyze the effect of feature-level usability on user retention in TikTok Shop among Generation Z users, as well as to identify retention patterns using a predictive approach. The study adopts a quantitative approach with an explanatory–predictive research design. Data were collected from 115 respondents through a questionnaire based on the System Usability Scale (SUS), focusing on the usability of live shopping and product review features. The analysis was conducted using binary logistic regression and the Naïve Bayes algorithm. The results show that both features fall into the marginal-low usability category, with scores below the acceptable threshold. However, the logistic regression results indicate that usability does not have a significant effect on user retention. On the other hand, the Naïve Bayes model achieved an accuracy of 88.64%, with a macro-average F1-score of 58.71%, but showed limitations in detecting non-retained users, as indicated by a minority-class recall of 14.29% due to class imbalance. These findings suggest that feature-level usability has limited explanatory power in predicting user retention within the social commerce context. This study contributes by reinforcing the argument that user retention is not solely determined by usability and demonstrates that predictive approaches can provide additional insights that are not captured by explanatory models.

Keywords


feature usability; logistic regression; naïve bayes; social commerce; user retention

Full Text:

PDF

References


W. F. Mustafa, S. Hidayat, and D. H. Fudholi, “Prediksi Retensi Pengguna Baru Shopee menggunakan Machine Learning,” J. Media Inform. Budidarma, Vol. 8, No. 1, pp. 612–623, 2024, DOI: 10.30865/mib.v8i1.7074.

A. Bhattacherjee, “Understanding Information Systems Continuance: An Expectation-Confirmation Model,” MIS Q., Vol. 25, No. 3, pp. 351–370, Sep. 2001, DOI: 10.2307/3250921.

F. M. Tambajong, R. Sengkey, and S. D. E. Paturusi, “Analyzing TikTok Shop User Experience with System Usability Scale Method,” J. Tek. Inform., Vol. 19, No. 3, pp. 251–258, 2024, DOI: 10.35793/jti.v19i3.53811.

F. Fieryando and B. Kristianto, “Analisis Sentimen terhadap TikTok Shop dengan K-Nearest Neighbor, Decision Tree, dan Naive Bayes,” J. Buana Inform., Vol. 15, No. 1, pp. 21–29, 2024, DOI: 10.24002/jbi.v15i1.8205.

DataReportal, “Essential TikTok Stats,” DataReportal.

F. Fathiyah, “TikTok dan kebebasan berekspresi di Ruang Digital bagi Generasi Z,” MEDIALOG J. Ilmu Komun., Vol. 6, No. 1, pp. 166–177, 2023, DOI: 10.35326/medialog.v6i1.4139.

I. Purwandani, N. O. Syamsiah, and S. Nurwahyuni, “Perceived Usability Evaluation of TikTok Shop Platform using the System Usability Scale,” Sink. J. dan Penelit. Tek. Inform., Vol. 7, No. 3, pp. 1389–1399, 2023, DOI: 10.33395/sinkron.v8i3.12473.

I. Zuhdi, L. M. Furkan, and H. Hilmiati, “Pengaruh Live Streaming Shopping dan Motivasi Belanja Hedonis terhadap Impulsif Buying pada Pengguna Aplikasi TikTok Shop di Kota Mataram,” J. Ris. Pemasar., Vol. 2, No. 3, pp. 66–72, 2023, DOI: 10.29303/jrp.v2i3.3286.

R. C. Kawet, I. D. Palandeng, and R. V. Pitta, “Pengaruh Online Customer Review, Online Customer Rating, dan Cash on Delivery terhadap Keputusan Pembelian Produk pada E-Commerce Tokopedia,” JMBI UNSRAT (Jurnal Ilm. Manaj. Bisnis dan Inov. Univ. Sam Ratulangi)., Vol. 11, No. 1, pp. 1170–1183, 2024, DOI: 10.35794/jmbi.v11i1.56516.

I. W. Sanjaya, M. R. Indah, D. R. Kurniawan, D. Saputra, and Ariansyah, “Analisis Tingkat Kepuasan Pengguna Layanan Tiktok Shop menggunakan System Usability Scale (SUS),” J. JUPITER, Vol. 15, No. Oktober 2023, pp. 951–959, 2021, DOI: 10.5281/zenodo.10068748.

G. Shmueli and O. R. Koppius, “Predictive Analytics in Information Systems Research,” MIS Q., Vol. 35, No. 3, pp. 553–572, Sep. 2011, DOI: 10.2307/23042796.

G.-D. Nguyen and T.-H. T. Dao, “Factors Influencing Continuance Intention to use Mobile Banking: An Extended Expectation-Confirmation Model with Moderating Role of Trust,” Humanit. Soc. SCI. Commun., Vol. 11, No. 1, p. 276, Feb. 2024, DOI: 10.1057/s41599-024-02778-z.

A. A. Daneji, A. F. M. Ayub, and M. N. M. Khambari, “The Effects of Perceived Usefulness, Confirmation and Satisfaction on Continuance Intention in using Massive Open Online Course (MOOC),” Knowl. Manag. E-Learning An Int. J., Vol. 11, No. 2, pp. 201–214, Jun. 2019, DOI: 10.34105/j.kmel.2019.11.010.

R. Alfiyaturrohmah, M. Prabantarikso, and B. M. Simatupang, “The Effect of Expectation Confirmation and Perceived Usefulness on Continuance Intention of QRIS Mobile Banking Users: The Mediating Role of Customer Satisfaction,” KALBISOCIO J. Bisnis dan Komun., Vol. 13, No. 1, pp. 7–19, 2026.

A. A. M Ayyoub, B. A. Abu Eidah, Z. N. Khlaif, M. Ahmad EL-Shamali, and M. R. Sulaiman, “Understanding Online Assessment Continuance Intention and Individual Performance by Integrating Task Technology Fit and Expectancy Confirmation Theory,” Heliyon, Vol. 9, No. 11, p. e22068, Nov. 2023, DOI: 10.1016/j.heliyon.2023.e22068.

F. M. Asyraf, A. Anofrizen, S. Syaifullah, and E. Saputra, “The Impact of User Satisfaction on the Intention to Continuously use the Threads Application: An ECM approach,” SISTEMASI, Vol. 14, No. 2, pp. 674–686, Mar. 2025, DOI: 10.32520/stmsi.v14i2.5027.

I. Z. Abdillah and H. N. Fadhilla, “Pengaruh Perceived Usefulness, Confirmation, dan Satisfaction terhadap Continuance Intention Penggunaan Fitur Live Streaming Shopee Eiger,” J. Manaj. Terap. dan Keuang., Vol. 15, No. 1, 2026, DOI: https://doi.org/10.22437/jmk.v15i01.52916.

M. Sokolova and G. Lapalme, “A Systematic Analysis of Performance Measures for Classification Tasks,” Inf. Process. Manag., Vol. 45, No. 4, pp. 427–437, Jul. 2009, DOI: 10.1016/j.ipm.2009.03.002.

Haibo He and E. A. Garcia, “Learning from Imbalanced Data,” IEEE Trans. Knowl. Data Eng., Vol. 21, No. 9, pp. 1263–1284, Sep. 2009, DOI: 10.1109/TKDE.2008.239.

F. M. Megahed, Y.-J. Chen, A. Megahed, Y. Ong, N. Altman, and M. Krzywinski, “The Class Imbalance Problem,” Nat. Methods, Vol. 18, No. 11, pp. 1270–1272, Nov. 2021, DOI: 10.1038/s41592-021-01302-4.

J. F. Hair, B. J. Babin, R. E. Anderson, and W. C. Black, Multivariate Data Analysis, 8th Editio. United Kingdom: Annabel Ainscow, 2018.

J. Brooke, “SUS: A Quick and Dirty Usability Scale,” in Usability Evaluation in Industry, P. W. Jordan, B. Thomas, B. A. Weerdmeester, and I. L. McClelland, Eds., Taylor & Francis, 1996, pp. 189–194. DOI: 10.1201/9781498710411-35.

A. Bangor, P. Kortum, and J. Miller, “Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale,” JUS J. Usability Stud., Vol. 4, No. 3, pp. 114–123, 2009.

W. H. DeLone and E. R. McLean, “The DeLone and McLean Model of Information Systems Success: A Ten-Year Update,” J. Manag. Inf. Syst., Vol. 19, No. 4, pp. 9–30, 2003, DOI: 10.1080/07421222.2003.11045748.

V. Venkatesh, M. G. Morris, G. B. Davis, and F. D. Davis, “User Acceptance of Information Technology: Toward a Unified View,” MIS Q. Manag. Inf. Syst., Vol. 27, No. 3, pp. 425–478, 2003, DOI: 10.2307/30036540.

M. D. Kurnia, T. Tukiyat, and M. Makhsun, “Penerapan Metode Naïve Bayes dan Weighted Product untuk Prediksi Lanjut Studi Peserta Didik,” J. Teknol. Sist. Inf. dan Apl., Vol. 6, No. 4, pp. 730–739, 2023, DOI: 10.32493/jtsi.v6i4.33538.

S. S. Hasibuan, A. Angraini, E. Saputra, and M. Megawati, “Sentimen Analisis terhadap Fitur TikTok Shop menggunakan Naïve Bayes dan K-Nearest Neighbor,” J. Media Inform. Budidarma, Vol. 8, No. 1, pp. 303–311, 2024, DOI: 10.30865/mib.v8i1.7238.

N. Hajli, “Social Commerce Constructs and Consumer’s Intention to Buy,” Int. J. Inf. Manage., Vol. 35, No. 2, pp. 183–191, Apr. 2015, DOI: 10.1016/j.ijinfomgt.2014.12.005.

K. Z. K. Zhang and M. Benyoucef, “Consumer Behavior in Social Commerce: A Literature Review,” Decis. Support Syst., Vol. 86, pp. 95–108, Jun. 2016, DOI: 10.1016/j.dss.2016.04.001.

N. Japkowicz and S. Stephen, “The Class Imbalance Problem: A Systematic Study,” Intell. Data Anal., Vol. 6, No. 5, pp. 429–449, 2002.




DOI: https://doi.org/10.32520/stmsi.v15i4.6261

Article Metrics

Abstract view : 4 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.