Study finds AI rarely recommend unnecessary medical care, but still make mistakes

Artificial intelligence (AI) models rarely suggest medical tests or treatments that patients do not need. However, they can still make mistakes in some medical situations, Ben-Gurion University of the Negev in Israel said on Tuesday.

In a study published in the Annals of Internal Medicine, researchers tested six commercial AI models from Google, Anthropic and OpenAI. They wanted to find out whether the models could recognise when medical checks, tests or treatments were being used more than necessary.

The researchers compared the models' answers with advice from the Choosing Wisely initiative. This initiative aims to reduce medical tests, procedures and treatments that are not needed, while recognising that some may be right for a patient depending on their situation.

The results showed that the models suggested medical action that might not have been needed in 5.4 per cent of cases where doctors should have held back. In comparison, they advised against a medical action that was actually needed in only 0.6 per cent of cases.

The researchers created 225 medical scenarios based on 15 Choosing Wisely recommendations. For each recommendation, 10 scenarios called for doctors to avoid unnecessary action, while five called for action. The six commercial AI models were tested three times on every scenario, producing a total of 4,050 responses.

The researchers found that suggestions for possibly unnecessary care mainly appeared in two areas: ultrasound scans of the thyroid for a small thyroid lump, smaller than one centimetre, found by chance; and routine electrocardiography (ECG), a test that records the heart's electrical activity, in adults who had a low risk of heart and blood vessel disease.

The findings come as doctors increasingly explore how AI tools can help them make medical decisions. The results suggest that commercial AI models can often recognise when unnecessary care should be avoided, although how well they do this depends on the medical situation.

The researchers warned that the study tested answers to single questions, rather than longer conversations. It also used carefully prepared scenarios instead of real medical appointments. An AI-based system first judged the answers, and doctors then reviewed them and settled any disagreements.

Because of these limits, the findings may not show how the models perform in everyday medical practice or in chatbots that patients use directly.

The researchers concluded that, in the scenarios they studied, the AI models rarely advised against medical action that was needed. However, their suggestions for care that might not be necessary remained a concern in certain medical situations.