Absence of AI hospital rules worries nurses

For nurse Judy Schmidt, the beeping monitors hooked up to critical patients at the Community Medical Center in Toms River, New Jersey, were just a normal part of the whirlwind of activity in the intensive care unit.

But looking back on her work about a decade ago, Schmidt said she realizes those machines were using early versions of artificial intelligence to help analyze and track the patients' health.

Artificial intelligence has been used in health care settings for years, even before the public became familiar with the technology, said Schmidt, CEO of the New Jersey State Nurses Association, a professional organization.

Today, some electronic health records are programmed to alert providers when patients could be having symptoms of a major illness. And in medical education, professors are depending more on simulations using mannequins, such as those programmed to mimic a birth, she said.

But the fast-paced development of these systems — to the point where robotics are being used in surgery — raises practical and ethical questions for the providers who work with that technology, Schmidt said.

Some experts say AI technology can improve the health care industry by automating administrative work, offering virtual nursing assistance and more. AI systems can predict whether a patient is likely to get sicker while in the hospital. Virtual assistant chatbots in telehealth services enable remote consultations. And more health care providers could start using robotics in the examination room.

But some nurses are concerned that the scarcity of laws regarding AI's use in hospitals and beyond means a lack of protections for individuals who could suffer from the technology's mistakes.

“In the long run, whatever artificial intelligence we use, it's still the human — the person — that has to take that data, and the interpretation of that data in some respects, and apply it to the real person that's in the bed, the nursing home or the home of that person,” Schmidt said.

State legislators are lagging on creating regulations for the use of AI, said Richard Ridge, an assistant professor of nursing at the University of Virginia. As the technology becomes more advanced, most health care workers are relying on policies set by their own hospital or practice, which can vary.

Legislators not only need to educate themselves about AI but also consider protections for patients within systems that use the technology, said Ridge, who added that nurses should be a part of those conversations.

“The value nurses bring to the table in any health care discussion is helping policymakers and decisionmakers see things from the patient's point of view and the patient's perspective,” Ridge said.

“I wouldn't want to read something [ policy] about AI in health care and it not have anything to say about nurses,” added Ridge, who also heads a panel on workforce issues for the professional group the Virginia Nurses Association.

Lawmakers in several states have introduced bills on artificial intelligence in health care, but a Stateline survey found only one that has been enacted: a Georgia law that allows the use of artificial intelligence devices in eye exams.

One Pennsylvania bill that's sitting in a House committee would require insurers to disclose whether they are using AI-based algorithms when reviewing claims to determine whether medical procedures are necessary.

Pennsylvania state Rep. Arvind Venkat, a Democrat sponsoring the bill and a physician, said the growth of artificial intelligence means it can be used to determine whether treatments or medications aren't covered by a patient's insurance.

“One of the problems we've seen with AI is that the data goes into the AI platform, it makes a decision, and it gets spit out, but that decision is only as good as the data being used to train the platform,” Venkat said. “Existing biases are being reinforced by the use of artificial intelligence, and especially in the area of health insurance.”

An Illinois bill would set the maximum number of patients that may be assigned to a registered nurse in specified situations. For health care facilities that use AI, nurses could override the technology's recommendations if they deem it in the patient's best interest

he American Nurses Association's code of ethics, followed by all nurses in the country, states that advanced technologies, including AI, do not replace nursing skills or judgement.

In a position statement, the organization said nurses “are responsible for being informed about and ensuring the appropriate use of AI” for their patients. It also said it's essential for nurses to be a part of efforts to advocate for an AI governance framework that holds technology developers accountable.

Dan Weberg, the vice president of the American Nurses AssociationCalifornia and an expert in the connection between technology and nursing, said rapid advances in AI are making the issues more complicated.

“We've been using algorithms and machine- generated insights for a number of years,” Weberg said, “but now, it's sort of getting more pressing with the complexity. It's getting more refined with more tools and that kind of stuff.”

Albert Fox Cahn, the executive director of the Surveillance Technology Oversight Project, a nonprofit organization that advocates for privacy rights in the use of new technologies, said that in the absence of federal rules, he hopes state and local policymakers will begin to create policies modeled after that of the European Union.

The EU AI Act, which is set to become the world's first set of laws to govern artificial intelligence, could become the global standard for AI governance. It attempts to define artificial intelligence and would set rules for regulating the technology across the EU, including prohibited AI practices.

While acknowledging that the technology has major benefits, the legislation establishes rules for public and private entities — including the health care sector — to use risk assessments, testing and more to ensure AI systems are working properly and protecting the rights of its users.

The EU's artificial intelligence liability directive, which was proposed in September 2022, would ease the burden of proof for victims to show damage caused by an AI system.

It's a really alarming moment for people in technology policy, Cahn said. There are new AI systems being deployed across industries, including health care, but without the laws in place to protect individuals in case something goes wrong, he said.

That doesn't mean AI systems should be scrapped, Cahn said, but ignoring the dangers of these systems would be a mistake. Policymakers should look at the impact of AI from every standpoint, he added, including the datasets used to train artificial intelligence that could hold implicit biases and lead to discrimination.

One challenge as AI advances is maintaining trust between providers and patients. Many patients fear that they're dealing with a robot rather than their practitioner, said Jennifer Shepherd, vice president of the Virginia Nurses Association. Providers must work with AI systems from a human- centered perspective, she added.

“One of the things we've thought about and what a lot of our focus is on is instead of just calling it `AI in health care' or `evidencebased AI,' what if we start using the term `human-centered AI'?” said Shepherd. “Focusing in on that, it's not so scary.”