Stanford details ICU-monitoring AI that links reduced patient activity to delirium
TRACER lets doctors and nurses look at a patient's behavior throughout their hospital stay. Its question-answering test covered only two patients, and the results do not show how accurately it can diagnose a condition.
An AI system developed by Stanford researchers turns video from intensive care units (ICUs) into a searchable timeline of patient behavior. It is called TRACER. It is designed to help medical teams look for possible signs of delirium, not to diagnose the condition for them.
Two cameras record each patient's hospital bed all day and night. TRACER checks the video once every minute and tracks 15 basic actions. These include talking, moving a hand toward the face, and lying still. The system combines these observations with electronic medical records. Doctors can then ask how a patient's behavior changed over several days, compare activity with treatments, and open video clips with timestamps to check what happened.
The research found a possible connection between patient activity and delirium. Across 280 patients and about 60,000 hours of video, patients who tested positive for delirium showed less purposeful activity. This included talking to others and looking at other people. However, this is a connection between two things, not proof that TRACER can reliably identify delirium or improve patient health.
The system recognized actions with an accuracy of up to 84.8 percent. Its question-answering scores ranged from 61 percent to 76.9 percent, depending on how the system was set up. Importantly, the question-answering test used questions about only two patients. This means the results do not show whether it would work just as well for many other patients.
The researchers also say they did not use patient data to train the system. The main question now is whether doctors can get useful and trustworthy answers when using it with a much wider range of patients. For now, doctors—not the AI—make the final assessment.
An AI research system can turn days of intensive-care video and medical records into a timeline that doctors can search and examine. In a November 20 report, Stanford HAI described TRACER, a research project designed to help doctors spot possible signs of delirium. The researchers found that patients who tested positive showed less purposeful activity. But this does not mean the system has been proven to diagnose delirium.
The project began with a problem: doctors usually check ICU patients only a few times each day. This leaves long periods when changes in a patient's condition may go unnoticed. Delirium is a state of confusion linked to lasting problems with thinking and a higher risk of death. It can be hard to spot during short medical visits. It is especially difficult to assess patients who cannot answer questions.
The researchers wanted to use movement, and a lack of movement, as another way to understand a patient's condition. TRACER combines video from cameras beside the bed with electronic medical records to follow behavior over time. The research paper, submitted on October 15, explains how the system organizes these observations and finds the evidence behind them. Its purpose is to support doctors, not replace their judgment.
Turning continuous video into an hourly record
Two cameras point toward the ICU bed and record high-quality video at 15 frames per second, all day and night. The second camera provides another view, making it less likely that a staff member or the patient will block what the cameras can see. However, recording and studying video all the time creates a large amount of data. The researchers therefore designed the system to process less of the footage.
TRACER checks the video once every minute and tracks 15 basic actions. These are small behaviors, such as talking, moving a hand toward the face, or lying still. The system puts these observations into hourly reports and combines them with medical records. An AI agent powered by a language model then uses tools to search the reports, find medical information, and examine relevant video clips.
Doctors can use the system to ask how a patient's behavior changed over several days. They can look at an activity timeline and compare it with medicines and treatments. They can also open video clips from specific times, including clips marked for a person to review. This lets doctors go back from a summary and check what the patient actually did on camera.
What the system's test scores tell us
The researchers tested two different abilities: recognizing actions and answering questions about a patient's hospital stay. To test action recognition, they collected 397 one-minute video clips. People had labeled the actions in these clips using the system's 15 action categories.
The question-answering test included 144 multiple-choice questions based on information from doctors. The questions used 264 hours of video and medical records from just two ICU patients. Because the test included so few patients, it does not show whether the system can answer questions accurately for a much larger and more varied group.
The Stanford HAI report highlights an action-recognition score of about 85 percent and a question-answering score of 61 percent. However, the research paper shows that question-answering results changed depending on which version of the AI agent was used. So 61 percent is not one fixed score that applies to every version of TRACER. Neither of these tests measures how accurately the system can identify patients with delirium.
A separate study looked at 280 patients and about 60,000 hours of camera footage. Patients who tested positive for delirium showed much less purposeful activity. This included looking at other people and communicating with them. The finding shows a connection between the behaviors recorded by TRACER and delirium assessments. But it does not prove that using TRACER can find delirium earlier or help patients recover.
The researchers say TRACER does not need patient data to train it. No patient data was used to train the system. The team hopes this will help avoid building systems that work well in one hospital but poorly in others. They are continuing to improve the research prototype. Doctors will use the reports and video to make their assessments. The AI system will not make those decisions for them.
Sources
- arxiv.orgTRACER: A Multimodal Harness for Patient Care in Hospitals
- hai.stanford.eduAI-Enabled Video System Monitors ICU Patients for Early Signs of Delirium | Stanford HAI
