Use of Artificial Intelligence Models for Veterinary Triage
| Field | Value | Language |
| dc.contributor.author | Wong, Arlene | |
| dc.contributor.author | Slapeta, Jan | |
| dc.contributor.author | Livingstone, Samantha | |
| dc.coverage.spatial | Australia: New South Wales, Sydney | en_AU |
| dc.coverage.temporal | 1 November 2018 and 31 October 2019 | en_AU |
| dc.date.accessioned | 2024-08-06T23:08:45Z | |
| dc.date.available | 2024-08-06T23:08:45Z | |
| dc.date.issued | 2024-08-07 | |
| dc.identifier.uri | https://hdl.handle.net/2123/32901 | |
| dc.description.abstract | OBJECTIVE To assess the capability of ChatGPT and nurses in accurately triaging emergency patients compared to veterinarians. METHODS Retrospective observational study using cases of canine patients presenting at a private veterinary specialist and emergency hospital between November 2018 and October 2019. Given clinical signs and history, each patient was assigned to one of five triage categories (“0 minutes”, “15 minutes”, “30-60 minutes”, “120 minutes”, and “240 minutes” waiting times). Triages were performed by three veterinarians, two nurses, ChatGPT-3.5 and ChatGPT-4.0. Statistical analysis was used to assess how often triage by ChatGPT and nurses agreed with veterinarian triages. RESULTS There was moderate-to-substantial agreement in triages between veterinarians (kappa-statistics between 0.49 and 0.66). Relative to the median veterinarian triage, ChatGPT has high sensitivity in identifying severe emergencies, correctly prioritizing around 80-90% of critical cases. However, ChatGPT also over-triaged, categorizing around 60% of non-urgent cases as needing to be seen immediately. ChatGPT’s triage performance was comparable to the performance of nurses, with the latter correctly identifying 87% of critical cases. When we complemented nurses’ triage with ChatGPT by using ChatGPT as a tool to flag severe cases (“0 minutes”), nurses’ triage sensitivity rose to 95%. CONCLUSIONS AND CLINICAL RELEVANCE These results suggest that artificial intelligence models have the potential to be an effective tool for flagging severe cases for immediate attention and complementing triage by nurses. However, the tendency to over-triage non-urgent cases may lead to increased pressure on emergency clinic resources. | en_AU |
| dc.language.iso | en | en_AU |
| dc.rights | Creative Commons Attribution 4.0 | en_AU |
| dc.subject | Emergency triage | en_AU |
| dc.subject | artificial intelligence | en_AU |
| dc.subject | ChatGPT | en_AU |
| dc.title | Use of Artificial Intelligence Models for Veterinary Triage | en_AU |
| dc.type | Dataset | en_AU |
| dc.subject.asrc | ANZSRC FoR code::30 AGRICULTURAL, VETERINARY AND FOOD SCIENCES::3009 Veterinary sciences::300901 Veterinary anaesthesiology and intensive care | en_AU |
| dc.identifier.doi | 10.25910/rbk1-p833 | |
| dc.description.method | Our retrospective observational study used canine emergency cases between 1 November 2018 and 31 October 2019 from the University Veterinary Teaching Hospital Sydney (UVTHS) emergency department. Human and animal ethical approval were deemed not necessary by the Research Integrity & Ethics Administration, Research Portfolio, University of Sydney (14 June, 2023). The study focused on the initial triage assessment at the time when a patient presents to the emergency clinic, prior to a full physical exam and further diagnostics made by veterinarians. Therefore, the data included information on initial vitals, such as heart rate, pulse rate, respiratory rate, color and character of mucous membranes, capillary refill time, demeanor and body condition score. Diagnostics typically done after the triage has been completed, such as hematology, biochemistry, and radiographs, were not used in the study. For each case, we recorded the main body system associated with the primary reason for the visit and clinical signs on presentation. The five broad categories were cardiorespiratory, circulatory, neurological and musculoskeletal, gastrointestinal and trauma. We used standard formulae to calculate the required target sample size to measure the share of cases where ChatGPT matches veterinarian triage with 90% level of confidence and 10% acceptable error. Based on the conservative assumption that the expected outcome is 0.5, a required sample size of 68 cases per body system category was calculated, which gives a total sample size of 340 cases across the five categories. A random selection procedure within each body system category was used to ensure a representative sample of cases by category. Investigating variation in triage performance by body system category is important for determining whether AI models are useful across all cases, or only for certain types of cases. Each canine case was retrospectively triaged by three veterinarians, two nurses, ChatGPT-3.5 and ChatGPT-4.0. Given patient history and presenting signs, the veterinarians, nurses and ChatGPT were prompted to retrospectively assign cases into one of five categories for target waiting time until the patient is seen by a veterinarian using Tables A1 and A2 of Ruys et al. (2012) and UVTHS definitions for the “blue” category. The 5 triage categories are: “Red” (0 minutes, immediate), “Orange” (15 minutes, very urgent), “Yellow” (30-60 minutes, urgent), “Green” (120 minutes, standard), and “Blue” (240 minutes, non-urgent). Reference: Ruys LJ, Gunning M, Teske E, Robben JH, Sigrist NE. Evaluation of a veterinary triage list modified from a human five-point triage system in 485 dogs and cats. J Vet Emerg Crit Care (San Antonio). 2012;22(3):303-312. doi:10.1111/j.1476-4431.2012.00736.x. | en_AU |
| dc.relation.other | Research & Enquiry Funds, Sydney School of Veterinary Science | |
| usyd.faculty | SeS faculties schools::Faculty of Science::University of Sydney School of Veterinary Science | en_AU |
| workflow.metadata.only | No | en_AU |
Associated file/s
Associated collections