Use of Artificial Intelligence Models for Veterinary Triage
Access status:
Open Access
Type
DatasetAbstract
OBJECTIVE
To assess the capability of ChatGPT and nurses in accurately triaging emergency patients compared to veterinarians.
METHODS
Retrospective observational study using cases of canine patients presenting at a private veterinary specialist and emergency hospital between ...
See moreOBJECTIVE To assess the capability of ChatGPT and nurses in accurately triaging emergency patients compared to veterinarians. METHODS Retrospective observational study using cases of canine patients presenting at a private veterinary specialist and emergency hospital between November 2018 and October 2019. Given clinical signs and history, each patient was assigned to one of five triage categories (“0 minutes”, “15 minutes”, “30-60 minutes”, “120 minutes”, and “240 minutes” waiting times). Triages were performed by three veterinarians, two nurses, ChatGPT-3.5 and ChatGPT-4.0. Statistical analysis was used to assess how often triage by ChatGPT and nurses agreed with veterinarian triages. RESULTS There was moderate-to-substantial agreement in triages between veterinarians (kappa-statistics between 0.49 and 0.66). Relative to the median veterinarian triage, ChatGPT has high sensitivity in identifying severe emergencies, correctly prioritizing around 80-90% of critical cases. However, ChatGPT also over-triaged, categorizing around 60% of non-urgent cases as needing to be seen immediately. ChatGPT’s triage performance was comparable to the performance of nurses, with the latter correctly identifying 87% of critical cases. When we complemented nurses’ triage with ChatGPT by using ChatGPT as a tool to flag severe cases (“0 minutes”), nurses’ triage sensitivity rose to 95%. CONCLUSIONS AND CLINICAL RELEVANCE These results suggest that artificial intelligence models have the potential to be an effective tool for flagging severe cases for immediate attention and complementing triage by nurses. However, the tendency to over-triage non-urgent cases may lead to increased pressure on emergency clinic resources.
See less
See moreOBJECTIVE To assess the capability of ChatGPT and nurses in accurately triaging emergency patients compared to veterinarians. METHODS Retrospective observational study using cases of canine patients presenting at a private veterinary specialist and emergency hospital between November 2018 and October 2019. Given clinical signs and history, each patient was assigned to one of five triage categories (“0 minutes”, “15 minutes”, “30-60 minutes”, “120 minutes”, and “240 minutes” waiting times). Triages were performed by three veterinarians, two nurses, ChatGPT-3.5 and ChatGPT-4.0. Statistical analysis was used to assess how often triage by ChatGPT and nurses agreed with veterinarian triages. RESULTS There was moderate-to-substantial agreement in triages between veterinarians (kappa-statistics between 0.49 and 0.66). Relative to the median veterinarian triage, ChatGPT has high sensitivity in identifying severe emergencies, correctly prioritizing around 80-90% of critical cases. However, ChatGPT also over-triaged, categorizing around 60% of non-urgent cases as needing to be seen immediately. ChatGPT’s triage performance was comparable to the performance of nurses, with the latter correctly identifying 87% of critical cases. When we complemented nurses’ triage with ChatGPT by using ChatGPT as a tool to flag severe cases (“0 minutes”), nurses’ triage sensitivity rose to 95%. CONCLUSIONS AND CLINICAL RELEVANCE These results suggest that artificial intelligence models have the potential to be an effective tool for flagging severe cases for immediate attention and complementing triage by nurses. However, the tendency to over-triage non-urgent cases may lead to increased pressure on emergency clinic resources.
See less
Date
2024-08-07Funding information
Research & Enquiry Funds, Sydney School of Veterinary Science
Licence
Creative Commons Attribution 4.0Faculty/School
Faculty of Science, University of Sydney School of Veterinary ScienceShare