Categories: Technology

Study finds that including evidence in questions confuses ChatGPT and lowers accuracy, according to Tech News

Reading Time: < 1 minute

A recent study has found that asking ChatGPT, an AI-powered chatbot, health-related questions with evidence included can confuse the bot and reduce its accuracy in providing answers. Researchers from the Commonwealth Scientific and Industrial Research Organisation (CSIRO) and The University of Queensland (UQ) in Australia conducted the study to investigate the impact of evidence on ChatGPT’s responses.

The study involved presenting ChatGPT with 100 health-related questions, ranging from the effectiveness of zinc in treating the common cold to the ability of drinking vinegar to dissolve a stuck fish bone. The questions were presented in two formats – either as a standalone question or with supporting or contrary evidence included.

The results showed that ChatGPT’s accuracy dropped from 80% when presented with question-only prompts to 63% when evidence was included in the prompts. The researchers hypothesized that the evidence added “too much noise,” leading to a decrease in accuracy.

Lead researcher Bevan Koopman, a CSIRO Principal Research Scientist and Associate Professor at UQ, emphasized the need for continued research on the use of large language models like ChatGPT for health-related inquiries. He highlighted the importance of informing the public about the potential risks associated with relying on AI-powered tools for health information.

The study was presented at the Empirical Methods in Natural Language Processing (EMNLP) conference in December 2023, underscoring the significance of understanding the limitations and effectiveness of AI-powered chatbots in providing accurate health information.

Team@GQN.

Share
Published by
Team@GQN.

Recent Posts

Salesforce Developer

Job title: Salesforce Developer Company: Han Staffing Job description: salesforce apex visual Job Description:Our client…

1 year ago

JAVA DEVELOPER

Job title: JAVA DEVELOPER Company: Han Staffing Job description: End Client: WELLSFARGO Title: Java Developer…

1 year ago

Jr. Full Stack Developer

Job title: Jr. Full Stack Developer Company: Leidos Job description: DescriptionJob Description:The Leidos Decision Advantage…

1 year ago

Jr. Full Stack Developer

Job title: Jr. Full Stack Developer Company: Leidos Job description: DescriptionJob Description:The Leidos Decision Advantage…

1 year ago

Principal Software Developer

Job title: Principal Software Developer Company: Oracle Job description: Job Description:As a member of the…

1 year ago

Sr Alfresco Developer- Lead

Job title: Sr Alfresco Developer- Lead Company: InterSources Job description: Job Title: Sr Alfresco Developer-…

1 year ago