Legal Liability of AI Developers for Individual Interactions with Chatbots Resulting in Death: A Comparative Study

Khairunnisa Maharani; Sinta Dewi Rosadi; Nella Sumika Putri

doi:10.56444/jidh.v10i1.6768

Legal Liability of AI Developers for Individual Interactions with Chatbots Resulting in Death: A Comparative Study

Khairunnisa Maharani, Sinta Dewi Rosadi, Nella Sumika Putri

Sari

The development and use of Artificial Intelligence (AI) in platforms such as chatbots has raised ethical and legal concerns. This study aims to examine the legal responsibilities of AI developers, focusing on the case of Garcia Vs. Character Technologies, in which a teenager’s death was allegedly influenced by interaction with an AI chatbot. This study compares Indonesian and European Union legal frameworks on AI developer liability for harms arising from chatbot interactions. This study is a legal study that uses a normative juridical approach by conducting a review of legislation and comparing laws with other countries to answer the research questions. The results show that the regulation of AI ethics in Indonesia is still developing and relies on the Electronic Information and Transactions Law and the Personal Data Protection Law, which do not specifically regulate AI, while the European Union already has more comprehensive regulations. The responsibilities of AI developers, such as those in chatbot platforms, include ensuring system security from physical or psychological impacts, protecting personal data, and ensuring fair and transparent algorithms. Universally, both in Indonesia and the European Union, developers are required to comply with the principles of safety, fairness, and human rights by conducting testing to avoid bias and negative impacts on users.

Kata Kunci

AI Developer, Artificial Intelligence, Character.AI, Chatbot, Legal Liability.

Teks Lengkap:

PDF (English)

Referensi

Sheikh, H., Prins, C. & Schrijvers, E. Artificial Intelligence: Definition And Background. In 15–41 (Springer, 2023). https://doi.org/10.1007/978-3-031-21448-6_2

Wilsdon, L. Carissa Véliz, Privacy Is Power: Why And How You Should Take Back Control Of Your Data . International Data Privacy Law 12, (2022). https://doi.org/10.1093/idpl/ipac007

Bryson, J. J. Towards A New Enlightenment? A Transcendent Decade The Past Decade And Future Of Ai’s Impact On Society, (2018). Https://Static1.Squarespace.Com/Static/5e13e4b93175437bccfc4545/T/67057b00d95cad198c5575da/1728412417139/The-Past-Decade-And-Future-Of-Ai-Impact-On-Society-Bbva.Pdf .

European Parliament&Council. Artificial Intelligence Act. (Eu) 2024/1689. Official Journal Of The European Union (2024).

Kevin Caridad. When The Chatbot Becomes The Crisis: Understanding Ai-Induced Psychosis. Papsychotherapy.Org, (2025) Https://Www.Papsychotherapy.Org/Blog/When-The-Chatbot-Becomes-The-Crisis-Understanding-Ai-Induced-Psychosis.

Kejaksaan Negeri Sukoharjo. Kitab Undang-Undang Hukum Pidana (Kuhp)Kitab Undang-Undang Hukum Pidana .

Sartor, Giovanni. The Impact Of The General Data Protection Regulation (Gdpr) On Artificial Intelligence. (European Parliament, 2020). https://dx.doi.org/10.2861/293

Sonata, D. L. Metode Penelitian Hukum Normatif Dan Empiris: Karakteristik Khas Dari Metode Meneliti Hukum. Fiat Justisia:Jurnal Ilmu Hukum 8, (2015). https://dx.doi.org/10.25041/fiatjustisia.v8no1.283

Raharjo, B. Teori Etika Dalam Kecerdasan Buatan (Ai). Yayasan Prima Agus Teknik (2019). https://penerbit.stekom.ac.id/index.php/yayasanpat/article/view/463

Tri Setia Ningrum, Vika Dias Anspratiwi & Muhamad Wahyudi. Etika Dan Dampak Sosial Dari Penerapan Artificial Intelligence Dalam Sistem Informasi Manajemen. Jurnal Riset Sistem Informasi 2, 06–15 (2025). https://doi.org/10.69714/26ypxk67

Haikcal, E., El-Hady, F., Zenrif, M. F. & History, A. Pandangan Islam Terhadap Etika Kecerdasan Buatan (Artificial Intelligence) Dalam Kehidupan Sehari-Hari. Jurnal Penelitian Ilmu Sosial Dan Kegamaan Islam 21, (2024). https://doi.org/10.19105/nuansa.v21i2.16613

Haris, M. T. A. R. & Tantimin, T. Analisis Pertanggungjawaban Hukum Pidana Terhadap Pemanfaatan Artificial Intelligence Di Indonesia. Jurnal Komunikasi Hukum (Jkh) 8, (2022). https://doi.org/10.23887/jkh.v8i1.44408

Undang-Undang Republik Indonesia Nomor 19 Tahun 2016. Undang-Undang Republik Indonesia 19 Tahun 2016 Tentang Atas Undang-Undang I1 Tahun 2oo8 Informasi Dan Transaksi Elektronik. (2016).

Kemenkominfo. Peraturan Menteri Komunikasi Dan Informatika Indonesia 5 Tahun 2020 Sistem Elektronik Lingkup Privat. (2020).

Ye, X., Yan, Y., Li, J. & Jiang, B. Privacy And Personal Data Risk Governance For Generative Artificial Intelligence: A Chinese Perspective. Telecomm Policy 48, (2024). https://doi.org/10.1016/j.telpol.2024.102851

Cath, C. Governing Artificial Intelligence: Ethical, Legal And Technical Opportunities And Challenges. Philosophical Transactions Of The Royal Society A: Mathematical, Physical And Engineering Sciences Vol. 376 (2018) Https://Doi.Org/10.1098/Rsta.2018.0080.

Ayala-Rivera, V., Portillo-Dominguez, A. O. & Pasquale, L. Gdpr Compliance Via Software Evolution: Weaving Security Controls In Software Design. Journal Of Systems And Software 216, (2024). https://doi.org/10.1016/j.jss.2024.112144

Varsha. How Can We Manage Biases In Artificial Intelligence Systems – A Systematic Literature Review. International Journal Of Information Management Data Insights 3, (2023). https://doi.org/10.1016/j.jjimei.2023.100165

Cheong, B. C. Transparency And Accountability In Ai Systems: Safeguarding Wellbeing In The Age Of Algorithmic Decision-Making. Frontiers In Human Dynamics 6, (2024). https://doi.org/10.3389/fhumd.2024.1421273

United States District Court Middle District Of Florida Orlando Division. Garcia V. Character Technologies, Inc, (2024) Https://Ncdoj.Gov/Wp-Content/Uploads/2023/09/54-State-Ags-Urge-Study-Of-Ai-And-Harmful-Impacts

United States District Court Middle District Of Florida Orlando Division. Garcia V. Character Technologies, Inc. Case No. 6:24-Cv-1903-Acc-Dci. (2025).

Menteri Komunikasi Dan Informatika Republik Indonesia. Surat Edaran Komunikasi Dan Informatika Indonesia 9 Tahun 2023 Kecerdasan Artifisial. (2023).

Custers, B., Lahmann, H. & Scott, B. I. From Liability Gaps To Liability Overlaps: Shared Responsibilities And Fiduciary Duties In Ai And Other Complex Technologies. Ai Soc 40, 4035–4050 (2025). https://doi.org/10.1007/s00146-024-02137-1

Potkewitz, M. & Narayanan, S. A Risk-Based Approach To Assessing Liability Risk For Ai-Driven Harms Considering Eu Liability Directive Self-Assessment Of Liability Risk For Ai-Driven Harms. (2023) Doi: Https://Doi.Org/10.48550/Arxiv.2401.11697

Pfeiffer, M. First, Do No Harm: Algorithms, Ai, And Digital Product Liability. Https://Cupr.Rutgers.Edu/About/ (2023) Doi: Https://Doi.Org/10.48550/Arxiv.2311.10861

Langer, M., Lazar, V. & Baum, K. How To Test For Compliance With Human Oversight Requirements In Ai Regulation? (2025) Doi: Https://Doi.Org/10.48550/Arxiv.2504.03300

Jeon, J. Standardization Trends On Safety And Trustworthiness Technology For Advanced Ai. (2024) https://doi.org/10.48550/arXiv.2410.22151

Raji, I. D., Xu, P., Honigsberg, C. & Ho, D. Outsider Oversight: Designing A Third Party Audit Ecosystem For Ai Governance. In Aies 2022 - Proceedings Of The 2022 Aaai/Acm Conference On Ai, Ethics, And Society 557–571 (Association For Computing Machinery, Inc, 2022) https://doi.org/10.1145/3514094.3534181

Inovasi dan Regulasi: Tantangan Hukum di Indonesia dalam Menanggapi Perkembangan Kecerdasan Buatan Generatif. (2025). Jurnal Penelitian Multidisiplin Internasional , 3 (6), 52-59. https://doi.org/10.62504/jimr1267

DOI: http://dx.doi.org/10.56444/jidh.v10i1.6768