DeepFake Voice Resilient System berbasis Otentikasi Spektrum Frekuensi Non-Linier pada Asisten Digital Pasien Lansia
Article Sidebar
Main Article Content
Abstract
Voice-based digital assistants for elderly patients with mild dementia are increasingly deployed but face serious threats from deepfake voice attacks that can impersonate family members. This research aimed to develop a system resilient to deepfake voice attacks by utilizing nonlinear frequency spectrum authentication derived from digital hearing aids already worn by the elderly. The methods employed included nonlinear feature extraction from acoustic feedback signals of hearing aids and Siamese Neural Network training with contrastive loss. The study involved 30 elderly individuals with mild dementia (MMSE scores 20-24) who used digital hearing aids. The results demonstrated that deepfake detection accuracy reached 96.7%, with a false rejection rate of 3.8%, a false acceptance rate of 2.4%, and an average authentication latency of 412 ms. This system required no active interaction from the elderly, thereby imposing no burden on their cognitive functions. This study concluded that hearing aid-based nonlinear frequency spectrum authentication effectively serves as a passive defense mechanism against deepfake voice attacks on elderly digital assistants.
Downloads
Article Details
Section
How to Cite
References
Babour, A., Bitar, H., Alzamzami, O., Alahmadi, D., Barsheed, A., Alghamdi, A., & Almshjary, H. (2023). Intelligent gloves: An IT intervention for deaf-mute people. Journal of Intelligent Systems, 32(1). https://doi.org/10.1515/jisys-2022-0076
Bartusiak, E. R., & Delp, E. J. (2021). Spectrogram-based deep learning for spoofing detection. Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing, 1-5.
Chen, L., Wang, Y., & Zhang, Z. (2025). One sentence can "clone" your voice: Security threats of zero-shot voice cloning. Proceedings of the 46th IEEE Symposium on Security and Privacy, San Francisco, CA, USA.
Fletcher, M. D., Verschuur, C. A., & Perry, S. W. (2023). Improving speech perception for hearing-impaired listeners using audio-to-tactile sensory substitution with multiple frequency channels. Scientific Reports, 13(1). https://doi.org/10.1038/s41598-023-40509-7
Fraunhofer AISEC. (2025). Deepfake detection benchmark: Human vs. AI performance. Fraunhofer Institute for Applied and Integrated Security.
Kaška, P. (2024). Resilience of voice assistants to synthetic speech (Bachelor's thesis). Brno University of Technology, Czech Republic.
LASIGE. (2026). Auth4ALL: Adaptive authentication for all users. LASIGE Research Report, University of Lisbon.
Malandrino, D., Scarano, V., Serra, L., & Vicidomini, L. (2023). Text-independent voice recognition based on Siamese networks and fusion embeddings. CEUR Workshop Proceedings, 3488, 1-14.
Malinka, K., Firc, A., Kaška, P., Lapšanský, T., Šandor, O., & Homoliak, I. (2024). Resilience of voice assistants to synthetic speech. In Lecture Notes in Computer Science - European Symposium on Research in Computer Security (ESORICS 2024). Springer.
Maske, S., Patil, S., & Kale, S. (2024). A comprehensive review of deepfake detection techniques: CNN, GAN, and hybrid approaches. IEEE Transactions on Information Forensics and Security, 19, 2345-2360.
Purdue University. (2025). PDID dataset benchmark: Evaluating commercial deepfake detection tools. Purdue University Research Report.
Roy, D. (2025). Deepfake detection using convolutional neural networks: A literature survey. National Institute of Technology Karnataka Technical Report.
W3C. (2025). Web Content Accessibility Guidelines (WCAG) 3.0: Authentication and cognitive accessibility. W3C Working Draft.
Xie, Y., Zhang, Z., & Yang, Y. (2021). Siamese network with wav2vec features for spoofing speech detection. Interspeech 2021, 1-5..
Yamazaki, Y., Ito, S., & Suzuki, K. (2023). Navigation method enhancing music listening experience by stimulating both neck sides with modulated musical vibration. IEEE Transactions on Haptics, 16(2), 228-239.