By Isaac Stovall
The emergence of Artificial Intelligence and Machine Learning (“AI/ML”) in recent years has drastically reshaped a wide range of industries, with the healthcare field being no exception. Healthcare systems have been implementing AI/ML technologies for assistance with clinical decision-making, improvement of the efficiency of administrative tasks, and as a tool for diagnostic and medical testing.[1]
Because healthcare is a heavily regulated space, it is therefore crucial for health systems to ensure that they are remaining compliant of such requirements as they continue integrating AI/ML into their ecosystem. This blog post discusses the interplay between some of these laws and regulations and the use of AI/ML in healthcare settings.
First, the Health Insurance Portability and Accountability Act (“HIPAA”) governs data privacy and security for patients by “covered entities,” which include healthcare providers, plans, and clearinghouses.[2] While this would not include AI developers and vendors, HIPAA requirements of patient data protection and limits on disclosure without patient authorization apply to “business associates,” which are entities that a covered entity may contract with to carry out certain functions, which could include many of the tasks that AI/ML models are already being used for today. Experts have highlighted a few compliance issues that could arise when integrating AI/ML models into healthcare systems, including that healthcare AI/ML models are trained on patient datasets, and the collection by a third-party vendor may make such data vulnerable to being intercepted by malicious actors.[3] Additionally, although such patient data is de-identified, scholars have discussed the issue of this data being re-identified when large tech companies gain access to it when they themselves integrate AI/ML models into their own technologies.[4]
Other various laws could potentially implicate the use of AI/ML technologies in healthcare systems, including the Federal Food, Drug, and Cosmetic Act (FDCA).[5] These technologies will often be used in a manner that classifies them as a “medical device” for the purposes of the FDCA, which, in pertinent part, defines medical devices as technologies that can be used in the diagnosis or treatment of disease or other conditions.[6] Therefore, AI developers will need to make submissions for approval by the Food and Drug Administration (“FDA”). The FDA has issued various documents to guide AI vendors as they develop and market their devices, and the agency has already approved the use of over 1,000 AI technologies as medical devices. [7]
AI/ML technologies hold great promise in revolutionizing healthcare delivery. Remaining cognizant of HIPAA, FDCA, and other laws and regulations that govern health systems will be essential in maximizing the potential of these models in this space.
Isaac Stovall is a second-year law student from Nashville, Tennessee. Before law school, he completed a Fulbright research grant in Brazil on Health Economics and received a Master of Public Health from Boston University.
[1] See Shiva Maleki Varnosfaderani & Mohamad Forouzanfar, The Role of AI in Hospitals and Clinics: Transforming Healthcare in the 21st Century, 11 Bioengineering, March 2024, at 337, 337.
[2] 45 C.F.R. § 160.103 (2024).
[3] See DOUGLAS MCNAIR & W. NICHOLSON PRICE II, HEALTH CARE ARTIFICIAL INTELLIGENCE: LAW, REGULATION, AND POLICY, IN ARTIFICIAL INTELLIGENCE IN HEALTH CARE: THE HOPE, THE HYPE 1, 222 (Michael Matheny, Sonoo Thadaney Israni, Mahnoor Ahmed & Danielle Whicher eds., 1990).
[4] See Delaram Rezaeikhonakdar, AI Chatbots and Challenges of HIPAA Compliance for AI Developers and Vendors, 51 J. L. MED. ETHICS, March 2024, at 988, 991.
[5] See FDA Issues Comprehensive Draft Guidance for Developers of Artificial Intelligence-Enabled Medical Devices, UNITED STATES FOOD AND DRUG ADMIN. (Jan. 6, 2025), https://www.fda.gov/news-events/press-announcements/fda-issues-comprehensive-draft-guidance-developers-artificial-intelligence-enabled-medical-devices
[6] 21 U.S.C. § 321(h)(1)
[7] See U.S. FOOD & DRUG ADMINISTRATION, ARTIFICIAL INTELLIGENCE/MACHINE LEARNING (AI/ML)-BASED SOFTWARE AS A MEDICAL DEVICE (SAMD) ACTION PLAN , https://www.fda.gov/media/145022/download?attachment;%20https://www.fda.gov/regulatory-information/search-fda-guidance-documents/artificial-intelligence-enabled-device-software-functions-lifecycle-management-and-marketing (last visited Sep. 24, 2025); U.S. FOOD & DRUG ADMINISTRATION, PROPOSED REGULATORY FRAMEWORK FOR MODIFICATIONS TO ARTIFICIAL INTELLIGENCE/MACHINE LEARNING (AI/ML)-BASED SOFTWARE AS A MEDICAL DEVICE (SAMD) – DISCUSSION PAPER AND REQUEST FOR FEEDBACK, https://www.fda.gov/media/122535/download (last visited Sept. 24, 2025).
