Implementasi Teknologi OCR dan Deep Learning pada Aplikasi Mobile untuk Otomatisasi Pencatatan Keuangan Pribadi Berbasis Struk
DOI:
https://doi.org/10.35870/jtik.v10i2.5230Keywords:
OCR, Deep Learning, Mobile, Expense Classification, Personal Finance AutomationAbstract
Personal financial management still faces limitations in both manual recording and conventional applications, such as low consistency and bias in expense categorization. This study develops a mobile application for personal finance automation using the waterfall method, integrating Optical Character Recognition (OCR) and Deep Learning to automatically record and classify expenses. The dataset consists of 900 images of local transaction receipts with varying print conditions. Text extraction is performed using a Convolutional Recurrent Neural Network (CRNN) and compared with the baseline Tesseract OCR. For expense classification, a CNN model with EfficientNet fine-tuning is applied Evaluation results show significant improvements with a character accuracy of 97.05%, word accuracy of 92.1%, and an F1-score of 82%. Transaction input time was reduced by an average of 62% compared to manual recording. A usability test using the System Usability Scale (SUS) with 36 respondents yielded a score of 70.069. The main contribution of this study is the integration of adaptive OCR and deep learning–based classification in the context of Indonesia’s local financial management.
Downloads
References
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023). Explainable artificial intelligence (XAI): What we know and what is left to attain trustworthy artificial intelligence. Information Fusion, 99, 101805. https://doi.org/10.1016/j.inffus.2023.101805.
Ardito, L., Coppola, R., Malnati, G., & Torchiano, M. (2020). Effectiveness of Kotlin vs. Java in Android app development tasks. Information and Software Technology, 127, 106374. https://doi.org/10.1016/j.infsof.2020.106374.
A-Sawaareekun, C., & Lipikorn, R. (2025). Menu item extraction from Thai receipt images using deep learning and template-based information extraction. In Proceedings of the 2024 6th International Conference on Information Technology and Computer Communications (pp. 107–113). ITCC ’24. Association for Computing Machinery. https://doi.org/10.1145/3704391.3704407.
Attanayaka, B., & Nawinna, D. (2023). WONGA: The future of personal finance management - A machine learning-driven approach for predictive analysis and efficient expense tracking. https://doi.org/10.1109/INCET57972.2023.10170209.
Efrian, M., & Latifa, U. (2022). Image recognition berbasis convolutional neural network (CNN) untuk mendeteksi penyakit kulit pada manusia. Power Elektronik: Jurnal Orang Elektro, 11(July), 276. https://doi.org/10.30591/polektro.v12i1.3874.
Grados-Espinoza, J., & Velasquez-Jimenez, L. (2025). Design and implementation of personal finance software for controlling financial efficiency. 73(2), 107–118.
Imawan, R., Putra, W. P., Alqahtani, R., Milakis, E. D., & Dumchykov, M. (2025). Enhancing financial literacy in young adults: An Android-based personal finance management tool. Journal of Hypermedia & Technology-Enhanced Learning, 3(1), 64–89. https://doi.org/10.58536/j-hytel.166.
Jange, B., Pendi, I., & Susilowati, E. M. (2024). Peran teknologi finansial (fintech) dalam transformasi layanan keuangan di Indonesia. Indonesian Research Journal on Education, 4(3), 1199–1205. https://doi.org/10.31004/irje.v4i3.1007.
Luthfi Firdaus, A., Kurnia, M. S., Shafera, T., & Istalama Firdaus, W. (2021). Implementasi optical character recognition (OCR) pada masa pandemi Covid-19. JUPITER: Jurnal Penelitian Ilmu dan Teknologi Komputer, 13(2), 188–194. https://doi.org/10.5281/3912.jupiter.2021.10.
Mardiani, N., & Juwita, K. (2024). Aplikasi LIKU (Literasi Keuangan) berbasis Android untuk meningkatkan financial literacy pelaku UMKM. Jurnal Informatika Ekonomi Bisnis, 6(3). https://doi.org/10.37034/infeb.v6i3.987.
Nistrina, K., & Sahidah, L. (2022). Unified modelling language (UML) untuk perancangan sistem informasi penerimaan siswa baru di SMK Marga Insan Kamil. J-SIKA: Jurnal Sistem Informasi Karya Anak Bangsa, 4(1), 17–23.
Panchal, S. B. (2024). Transforming money management: Analyzing the impact of technology on personal finance.
Permana, A. A., & Prakoso, A. B. (2023). Perancangan sistem informasi antrian jasa service menggunakan metode iteratif berbasis website. Format: Jurnal Ilmiah Teknik Informatika, 11(2), 100. https://doi.org/10.22441/format.2022.v11.i2.001.
Pratomo, D. N., Kusumaning Putri, D. U., & Azhari, A. (2022). Implementasi optical character recognition berbasis deep learning untuk ekstraksi data sertifikat tanah. Jurnal Informatika: Jurnal Pengembangan IT, 7(3), 131–134. https://doi.org/10.30591/jpit.v7i3.3657.
Purwanto, P., Safitri, D. Y., & Pudail, M. (2023). Edukasi pencatatan laporan keuangan sederhana bagi pelaku usaha mikro, kecil dan menengah (UMKM). As-Sidanah: Jurnal Pengabdian Masyarakat, 5(1), 1–14. https://doi.org/10.35316/assidanah.v5i1.1-14.
Raup, A., Ridwan, W., Khoeriyah, Y., Supiana, S., & Zaqiah, Q. Y. (2022). Deep learning dan penerapannya dalam pembelajaran. JIIP - Jurnal Ilmiah Ilmu Pendidikan, 5(9), 3258–3267. https://doi.org/10.54371/jiip.v5i9.805.
ROSE, O. (2025). Android Jetpack Compose UI components handbook. Onyx Rose.
Santoso, I. B., Aji, I. P., Franskusuma, S., Putri, K. A., Ardharani, Y., Mujiastuti, R., Ambo, S. N., Meilina, P., Rosanti, N., & Amri, N. (2025). Educating on the application of TensorFlow in artificial intelligence, machine learning, and deep learning. Society: Jurnal Pengabdian Masyarakat, 4(2), 318–325. https://doi.org/10.55824/jpm.v4i2.547.
Satav, M. S., Varade, T., Kothavale, D., Thombare, S., & Lokhande, P. (2020). Data extraction from invoices using computer vision. In 2020 IEEE 15th International Conference on Industrial and Information Systems (ICIIS), 316–320. https://doi.org/10.1109/ICIIS51140.2020.9342722.
Subur, J., Suryadhi, M. T., Al Hafizh, N. R., & Reza, M. (2024). Pemanfaatan teknologi computer vision untuk deteksi ukuran ikan bandeng dalam membantu proses sortir ikan. CYCLOTRON, 7(01), 52–60. https://doi.org/10.30651/cl.v7i01.21239.
Terven, J., Córdova-Esparza, D.-M., & Romero-González, J.-A. (2023). A comprehensive review of YOLO architectures in computer vision: From YOLOv1 to YOLOv8 and YOLO-NAS. Machine Learning and Knowledge Extraction. https://doi.org/10.3390/make5040083.
Zainuddin, F. F., P, F. C. E., Dhyaksa, P. S., Ardiansyah, M. B., Buana, P. A., & Penulis Korespondensi. (2025). System usability scale (SUS): Analisis pengalaman pengguna pada portal penerimaan mahasiswa baru Universitas Semarang. Jurnal Komputer dan Teknologi Sains (KOMTEKS), 4(1), 23–28.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Suhandana Ariawan Andi, Moh Alfaujianto, Susana Dwiyulianti
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
1. Copyright Retention and Open Access License
Authors retain copyright of their work and grant the journal non-exclusive right of first publication under the Creative Commons Attribution 4.0 International License (CC BY 4.0).
This license allows unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
2. Rights Granted Under CC BY 4.0
Under this license, readers are free to:
- Share — copy and redistribute the material in any medium or format
- Adapt — remix, transform, and build upon the material for any purpose, including commercial use
- No additional restrictions — the licensor cannot revoke these freedoms as long as license terms are followed
3. Attribution Requirements
All uses must include:
- Proper citation of the original work
- Link to the Creative Commons license
- Indication if changes were made to the original work
- No suggestion that the licensor endorses the user or their use
4. Additional Distribution Rights
Authors may:
- Deposit the published version in institutional repositories
- Share through academic social networks
- Include in books, monographs, or other publications
- Post on personal or institutional websites
Requirement: All additional distributions must maintain the CC BY 4.0 license and proper attribution.
5. Self-Archiving and Pre-Print Sharing
Authors are encouraged to:
- Share pre-prints and post-prints online
- Deposit in subject-specific repositories (e.g., arXiv, bioRxiv)
- Engage in scholarly communication throughout the publication process
6. Open Access Commitment
This journal provides immediate open access to all content, supporting the global exchange of knowledge without financial, legal, or technical barriers.
