Photo Credit: Delmaine Donson

The following is a summary of “Three versions of an atopic dermatitis case report written by humans, artificial intelligence, or both: identification of authorship and preferences.” published in the November 2024 issue of Allergy and Immunology by Bianchi et al. 

AI use in scientific writing grew, raising concerns about authorship identification, writing efficiency, and content quality.  

Researchers conducted a retrospective study to explore the real-world impact of ChatGPT, a large language model, in a simulated publication scenario. 

They recruited 48 individuals from 3 medical expertise levels: medical students, residents, and experts in Allergy or Dermatology. Each participant evaluated 3 blinded versions of an atopic dermatitis case report: 1 human-written (HUM), 1 AI-generated (AI), and 1 combined written (COM). They assessed authorship, ranked preferences, and graded 13 quality criteria for each version while recording the time taken to generate each manuscript. 

The results showed authorship identification accuracy was 33%, with experts (50.9%) performing better than residents (27.7%) and students (19.6%, P<0.001). Participants preferred AI-assisted versions (AI and COM) over HUM (P<0.001), with COM scoring highest in quality. COM and AI reduced writing time by 83.8% and 84.3%, respectively, and improved quality by 13.9% (P<0.001) and 11.1% (P<0.001). However, experts gave the lowest scores for references in the AI manuscript, which could impact its publication. 

Investigators found that while AI-assisted writing can save time, human oversight was essential to ensure accuracy and ethical standards. The study emphasized the need for transparency in AI use and the potential benefits of human-AI collaboration in scientific writing. 

Source: jaci-global.org/article/S2772-8293(24)00169-3/fulltext