ChatGPT shows promise in answering patients’ questions to urologists

The ChatGPT chatbot exhibits potential as a time-saving instrument for responding to affected person questions despatched to the urologist’s workplace, suggests a research within the Urology Practice journal.

The synthetic intelligence (AI) instrument generated “acceptable” responses to almost one-half of a pattern of real-life affected person questions, in keeping with the brand new analysis by Michael Scott, MD, a urologist at Stanford University School of Medicine.

“Generative AI applied sciences might play a worthwhile function in offering immediate, correct responses to routine affected person questions—doubtlessly assuaging sufferers’ issues whereas liberating up clinic time and sources to handle different advanced duties,” Dr. Scott feedback.

Can ChatGPT precisely reply questions from urology sufferers?

ChatGPT is an revolutionary massive language mannequin (LLM) that has sparked curiosity throughout a variety of settings, together with well being and drugs. In some current research, ChatGPT has carried out nicely in responding to numerous forms of medical questions, though its efficiency in urology is much less well-established.

Modern digital well being document (EHR) programs allow sufferers to ship medical questions on to their medical doctors. “This shift has been related to an elevated time burden of EHR use for physicians with a big portion of this attributed to affected person in-basket messages,” the researchers write. One research estimates that every message in a doctor’s inbox provides greater than two minutes spent on the EHR.

Dr. Scott and colleagues collected 100 digital affected person messages requesting medical recommendation from a urologist at a males’s well being clinic. The messages have been categorized by kind of content material and issue, then entered into ChatGPT. Five skilled urologists graded every AI-generated response when it comes to accuracy, completeness, helpfulness, and intelligibility. Raters additionally indicated whether or not they would ship every response to a affected person.

Findings help ‘generative AI expertise to enhance medical effectivity’

The ChatGPT-generated responses have been judged to be correct, with a median rating of 4.0 on a five-point scale; and intelligible, common rating 4.7. Ratings of completeness and helpfulness have been decrease, however with little or no potential for hurt. Scores have been comparable for various kinds of query content material (signs, postoperative issues, and so forth.).

“Overall, 47% of responses have been deemed acceptable to ship to sufferers,” the researchers write. Questions rated as “straightforward” had the next fee of acceptable responses: 56%, in comparison with 34% for “troublesome” questions.

“These outcomes present promise for the utilization of generative AI expertise to assist enhance medical effectivity,” Dr. Scott and co-authors write. The findings “recommend the feasibility of integrating this new expertise into medical care to enhance effectivity whereas sustaining high quality of affected person communication.”

The researchers notice some potential drawbacks of ChatGPT-generated responses to affected person questions, “ChatGPT’s mannequin is skilled on info from the Internet generally, versus validated medical sources,” with a “threat of producing inaccurate or deceptive responses.” The authors additionally spotlight the necessity for safeguards to make sure affected person privateness.

“While our research gives an attention-grabbing start line, extra analysis shall be wanted to validate using LLMs to reply to affected person questions, in urology in addition to different specialties,” Dr. Scott feedback. “This shall be a doubtlessly worthwhile well being care utility, significantly with continued advances in AI expertise.”