AI technology that listens to and transcribes patients’ consultations with doctors can put them at risk by getting the names of drugs and illnesses wrong, an NHS watchdog has warned. In one case a woman was left badly shaken when the AI scribe’s summary of her conversation wrongly said she had demyelination – serious nerve damage that can lead to multiple sclerosis. It was only when the patient, an NHS health professional, queried the AI tool’s record of the result of her MRI scan that the hospital corrected it to what it should have been – “null demyelination”.
Healthwatch England has highlighted the case and other mistakes by AI scribes to show that the technology, which the NHS is rolling out rapidly, is a potential threat to patient safety. Unless detected, errors could end up in patients’ medical records and affect their care, it said. In another case, an AI scribe confused the drug the GP had prescribed with a different one of a similar name – a blunder which again the patient, rather than the doctor, identified.
On another occasion an AI-generated summary letter did not say that the hospital consultant had told the patient to seek a repeat prescription from their GP for their migraine, which could have left them unable to get their medication. Healthwatch, the statutory NHS patient champion, has heard “multiple stories from patients who have noticed these errors when a health professional hasn’t”, it said. It is central to the planned “big shift” for the NHS from being an analogue to a digital-based service.
A Healthwatch spokesperson said: “Healthcare has never been error-free. But our findings show the urgent need for clarity over how patients can report and get corrected any mistakes made by AI scribing tools or the professionals that use them.” Rachel Power, chief executive of the Patients Association, said: “Trust and confidence in this technology depend on good communication and genuine partnership with patients and right now both are missing.” Healthwatch added that it was “worrying” that the Medicines and Healthcare products Regulatory Agency has decided not to classify AI scribes as medical devices, which means there will be no England-wide oversight to ensure they are safe to use and effective. GPs and hospital doctors in England are already using 27 different AI scribes.
Ministers have been warned that the NHS and medics could be sued over mistakes made by AI. AI’s accuracy was in the spotlight recently when patients in Rotherham complained to their local Healthwatch that an AI receptionist used by some local GP practices did not understand their strong Yorkshire accents. Dr Shier Ziser Dawood, a GP in London, last year warned in a leading medical journal that AI scribes may prove “a double-edged sword” for family doctors.
She recounted that an AI scribe said she had told her patient to “continue their Prozac” even though she had not prescribed or discussed that drug with them. That is an example of what are known as “hallucinations”, where AI scribes refer to something that was not raised during the consultation. She warned that the need for doctors to review all transcripts in order to check for errors meant that AI tools are not yet proving time-saving.
Extract — continue reading at the source.
