Exploring policy and regulations of clinical AI systems: Views from patients with chronic diseases

Background: Artificial Intelligence (AI) has become a transformative force in healthcare, offering opportunities to enhance patient care, improve efficiency, and reduce costs. However, patients' perspectives, which greatly influence the acceptance and implementation of AI technologies, remain under-researched. Objective: This study explores patients with chronic conditions’ perspectives on clinical AI systems, focusing on their concerns, government involvement, accountability for potential AI error, and preferences between AI and doctor recommendations. These insights are crucial for tailoring AI technologies to meet patients' needs and expectations and better engage patients in adopting new technologies. Method: This study conducted an online open-ended survey with valid responses from 140 patients with chronic conditions, exploring four aspects of clinical AI perspectives. The data was systematically coded and analyzed using an inductive thematic analysis approach to identify emergent themes. Result: The majority of participants expressed concerns about the implementation of AI in healthcare (92.86 %), with the top worries including lack of human touch (22.86 %), potential AI bias and fairness (16.43 %), and over-dependence on AI (16.43 %). Regarding responsibility for potential treatment damages, 37.14 % of participants believed that physicians should bear the responsibility, 16.43 % considered AI developers accountable, and 1.42 % viewed the government as the responsible party. Furthermore, 44.57 % suggested that responsibility should be shared among stakeholders. In terms of government role, 51.43 % saw regulation and monitoring as key responsibilities, while 8.57 % perceived no government role in AI healthcare. Finally, around 80 % of patients preferred treatment recommendations from care providers over AI. Conclusion: The findings suggest patients are looking for a balanced approach between technology and human involvement, with clear accountability and proper regulation. Though most prefer human doctors, an openness to AI's potential indicates an evolving perception. This underscores the need for a governance-inclusive and patient-centric strategy that addresses these aspects to ensure successful AI integration in healthcare. Lay Summary: This study explores the opinions of chronic patients on using AI in healthcare. It found that while patients appreciate the potential benefits of AI, they have concerns about losing the personal touch of human doctors, potential biases, and over-reliance on technology. They also believe that accountability for AI errors should be shared among doctors, developers, and the government. The findings highlight the need for careful integration of AI in healthcare, with clear regulations and a focus on patient safety to build trust and acceptance.