GPT-4o-Based Physician Assistant: Why Record Analysis Did Not Improve Outcomes in Kenyan Clinics

A GPT-4o-based physician assistant is a software tool for supporting clinical decisions. It analyzes medical records, checks a specialist’s reasoning, and offers recommendations. A good answer to an individual medical question does not yet prove that the system will improve patients’ health in a real clinic.

The trial in Kenya clearly demonstrates the difference between these tasks. The AI provided doctors with additional information, but after two weeks adverse outcomes were recorded in 2,2% of patients in the assistant group and 2% of patients in the control group. The difference was 0,2 percentage points. This alone does not prove that the system caused harm: reaching such a conclusion requires confidence intervals, prespecified outcome criteria, and a complete statistical analysis.

Testing an answer and testing its impact on a patient are different tasks

A model can be tested on clinical questions, its answers can be compared with physicians’ conclusions, and the completeness of its recommendations can be assessed. Such a test answers the question of how well the system reasons in a given scenario. It does not show whether treatment will become safer if the recommendations are integrated into a clinic’s workflow.

A clinical outcome is built from several links. The physician must see the prompt, correctly understand its limitations, agree with it, choose an appropriate action, explain it to the patient, prescribe treatment, and ensure follow-up. An error or delay at any stage can reduce the benefit of good analysis to zero, even when the recommendation itself was substantively correct.

Even a conclusion that experts consider correct does not guarantee benefit to the patient. The system may have detected a risk but displayed it in an inconvenient interface; the physician may not have trusted the recommendation, lacked time to check the sources, or encountered an absent procedure for further action; the patient, in turn, may not have followed the prescription, returned for a follow-up appointment, or received the necessary medication.

How the Trial in Kenya Was Structured

The trial described involved 16 clinics, about 9 500 patients, and more than 100 healthcare workers. One group of specialists received a GPT-4o-based assistant, while the control group worked without such a tool. The researchers compared adverse outcomes after two weeks.

The cautious conclusion from the data presented is this: in this scenario, the assistant did not improve the measured outcome. The slight increase in the measure in the AI group cannot be called proven harm without statistical analysis, because random fluctuations, differences between clinics, and patients’ baseline characteristics may have affected the final result.

What the Physician Assistant Actually Did

The assistant did not replace a medical worker with an autonomous diagnosis. According to the trial description, specialists received analysis of medical records, checks of their own decisions, and recommendations. This role is closer to that of a second reader or consultant who helps identify missed information and consider additional options.

The strength of the approach is clear: the system can quickly process a large volume of text and compare symptoms, medical history, examination results, and prescribed actions. In an overloaded clinic, such a function could theoretically reduce the likelihood that a physician will miss an important detail.

A recommendation in an interface does not yet become part of treatment. The physician makes decisions amid queues, limited time, incomplete records, and varying data quality. If the assistant regularly produces too many insignificant warnings, specialists begin filtering the messages or stop viewing them as useful signals. An overly confident formulation, by contrast, increases the risk of excessive trust.

Why Useful Prompts Did Not Produce a Better Outcome

The workflow may have absorbed the benefit

Introducing an assistant changes not only the content of a consultation but also the sequence of actions. The healthcare worker must open the system, wait for the analysis, read the recommendation, compare it with the record, and decide exactly what to change. When these steps take time or duplicate existing documentation, a useful function becomes an additional burden.

And what happens if the recommendation arrives after the appointment has ended? The model may have correctly pointed to a risk, but the clinic may already have been unable to change the examination, treatment, or follow-up plan. Evaluating an assistant should include not only the quality of the prompt but also when it appears in the workflow.

Trust in the system is not the same as trust in an individual recommendation

A physician may view AI as a useful reference source but disagree with its conclusion in a particular case. Such a decision does not necessarily indicate a poor model. The specialist has observations that are absent from the medical record: the patient’s appearance, intonation, behavioral characteristics, family circumstances, and information shared during the conversation.

The opposite problem arises when a confident response style creates an illusion of reliability. The model may construct a plausible explanation from incomplete or erroneous data, and a physician under time pressure may fail to notice the weak point. Balancing distrust and excessive trust remains a separate implementation challenge.

The quality of medical records limits the quality of analysis

AI sees only what has made it into the available records. A missed allergy, an outdated medication list, a mistaken examination date, or an incomplete description of symptoms changes the context on which the recommendation is based. A model can process incorrect input data without error and still lead the physician to an unsuccessful decision.

Studies using electronic medical records also show a separate risk of patient identification errors. In an experiment with intentionally substituted record headers, agents often continued working without noticing that the data belonged to another person. This was a test scenario, not an observation of real treatment cases, but it shows why verifying the patient’s identity and the integrity of the record must remain a separate control.

A recommendation does not control subsequent actions

Even a correct decision requires execution. The medication must be prescribed, its availability checked, the regimen explained, follow-up scheduled, and the patient contacted if their condition worsens. A physician assistant can remind people of a risk but cannot guarantee that all participants in the process will carry out the necessary actions.

This is particularly noticeable in clinics where patients face financial, transportation, and organizational constraints. Improving the wording of a recommendation does not eliminate a lack of medication, a long journey to the hospital, or the inability to attend a follow-up appointment.

Why AI Implementation Adds New Points of Failure

Before the assistant appeared, an error could arise during history-taking or treatment selection. After implementation, a chain is added: uploading the record, matching the patient, extracting data, generating an output, displaying a warning, interpreting it by the physician, and documenting the decision. Each new link requires verification, and a failure at one stage can change the outcome of the entire procedure.

• The system may receive an incomplete or outdated record.
• The model may assign priorities incorrectly among risks.
• The interface may hide an important warning among secondary messages.
• The physician may miss the prompt because of workload or accept it without sufficient verification.
• The clinic may lack a procedure for reviewing disputed recommendations and recording incidents.

This list does not mean that medical AI is useless. It shows why demonstrating a strong answer and conducting a clinical trial represent different levels of evidence. A clinic manager must assess not only the quality of the model but also how it works in a specific organization, with its records, procedures, and staff workload.

What Metrics Does a Clinic Need Besides Accuracy?

Answer accuracy remains an important characteristic, but it does not describe the assistant’s full effect. Evaluation should include medical, organizational, and social measures. Their set depends on the task, specialty, clinic level, and risk to the patient, so no single universal set of metrics is appropriate here.

• Patient outcomes: complications, repeat visits, hospitalizations, adherence to follow-up timelines, and adverse events.
• Treatment process: time to prescribing, completeness of the examination, correctness of the medication regimen, and the proportion of decisions changed after review.
• Specialist workload: appointment duration, time spent on documentation, number of notifications, and the subjective assessment of the prompts’ usefulness.
• Data reliability: record-matching errors, omissions, outdated information, and cases of incorrect interpretation.
• Equity of access: differences in outcomes among linguistic, age, social, and medical groups.
• Economics: costs of implementation, training, support, and incident review compared with the measured benefit.

When a clinic reports only high accuracy on test questions, the manager cannot see whether real decisions and outcomes have changed. A pilot project should define its target measures, control group, observation period, experiment-stopping procedure, and those responsible for analyzing adverse events in advance.

How to Safely Implement a Physician Assistant

It is more reasonable to begin with tasks where an error is reversible and a specialist can easily check the result. These include preparing a brief record summary, finding missing information, translating medical text into plain language, and compiling a list of questions for a consultation. Automatically prescribing treatment or changing a record without specialist review requires considerably stricter controls.

1. Define a specific clinical task and the system’s scope of application.
2. Check the quality of input data and the rules for access to medical information.
3. Assign a physician to approve every recommendation that affects treatment.
4. Conduct testing on local scenarios before expanding use.
5. Record missed signals, false warnings, and identification errors.
6. Compare results with a comparable group without an assistant.
7. Discuss the results with physicians, administrators, and data-protection specialists.

Responsibility for the clinical decision must remain with an authorized healthcare worker and be established in the clinic’s internal rules. The model’s name does not replace medical qualifications, and an audit does not turn a recommendation into a proven prescription.

What This Means for Patients

Patients can use AI to prepare for a visit, clarify unfamiliar terms, and compile a list of questions. This scenario helps them gather their medical records and avoid forgetting an important detail during a conversation with a physician.

An AI assistant does not replace clinical care. The system’s response should not be considered a diagnosis or a basis for changing treatment independently simply because the recommendation sounds convincing. If symptoms are rapidly worsening or appear dangerous, seek medical care through the established pathway rather than waiting for confirmation from a chat system.

The expectation of personalized services like ChatGPT Health does not change this rule. According to the capabilities stated by the developer, such products are intended to work with a user's medical information and help prepare for a conversation with a doctor, not to replace diagnosis and treatment. The availability of features, connection of medical records, and data protection modes depend on the region and the specific product, so before uploading documents, you need to study the terms for processing sensitive information.

FAQ

Did the trial prove that GPT-4o is dangerous for medicine?

No. It showed that in the described scenario, the assistant did not improve unfavorable outcomes: 2,2% versus 2%. A small difference by itself does not prove harm without a full statistical analysis and an assessment of the quality of how the trial was conducted.

Why doesn't a good model response guarantee a benefit?

The response must enter the workflow, be correctly understood by the doctor, and lead to timely action. The result is also affected by the completeness of the record, the availability of treatment, adherence to the prescription, and subsequent monitoring.

Should clinics completely abandon medical AI?

That conclusion does not follow from the described result. A clinic should choose a limited task, assign human oversight, check data security, and measure real outcomes rather than limit itself to demonstrating the accuracy of responses.

Can a patient upload their medical records to an AI service?

The decision depends on the terms of the specific service and legislation concerning medical confidentiality and personal data. Before uploading, you need to check who processes the documents, how to revoke access, where the information is stored, and whether it is used to train models.

The main conclusion: an assistant for doctors based on GPT-4o may improve access to information and support clinical reasoning, but only a real trial in treatment settings shows whether it changes patients' health outcomes. The described experiment in Kenya did not detect such an improvement. Between strong analysis of a record and a better outcome lies a long chain of human decisions, data quality, and organizational procedures.