Carlos Solis
Who would have guessed—young people think artificial intelligence gives better answers to mental-health questions than psychiatrists. At least, that’s what participants in a study carried out by researchers from the Norwegian University of Science and Technology and the SINTEF Foundation for Industrial and Scientific Research believe.
Results published in the journal DIGITAL HEALTH show that young people found ChatGPT’s answers more useful and more relevant than professionals’ responses.
A group of healthcare workers also rated the answers, and they too found the language model’s advice more convincing.
“Both professionals and young people thought ChatGPT provided advice they perceived as relevant, responsive, and easy to understand,” said Marita Skjuve, a researcher at SINTEF.
How the study tested ChatGPT against clinicians
Skjuve and her colleagues selected real mental health questions that were worrying young Norwegians. For each question there were two anonymous responses: one generated by ChatGPT (GPT-4) and one written by a medical professional.
The team recruited 123 young people and 31 healthcare workers to rate the answers on the following criteria:
- accuracy
- relevance
- clarity
- usefulness
The quantitative data showed that both young people and healthcare workers rated answers from both sources similarly on accuracy, clarity, and usefulness. However, the young volunteers judged ChatGPT’s answers to be more relevant and more useful. In other words, that group preferred the chatbot’s clear, practical advice.
“We noticed young people like ChatGPT’s answers a bit more because they’re clearer and feel more useful. Also, ChatGPT is pretty good at giving clear, digestible answers in bullet-point form.” — Marita Skjuve, researcher at SINTEF.
A chatbot with impostor syndrome
Experts who reviewed the findings expressed concern that the AI model appears to be attempting to make diagnoses.
While not all healthcare workers are allowed to make diagnoses or provide medical care, ChatGPT isn’t constrained by any rules.
Still, Skjuve’s team suggests AI could become a useful aid for clinicians, helping clinics scale services and reach more patients, particularly in mental health.
The scientists believe the future lies in hybrid services where AI and medical staff collaborate closely. That collaboration will be useful because AI doesn’t always grasp context and has a tendency to make up answers.
Overall, the researchers weren’t very surprised by the results. “In other studies we have seen that artificial intelligence is often perceived as more responsive than medical staff. It is friendly and sympathetic toward patients,” recalled Marita Skjuve.
Photo: pixabay.com
