TY - JOUR
AU - Ermis, Serhat
AB -  Clinical Relevance Recently, studies have been carried out on the accuracy of large language models, which are increasingly used, but the reproducibility of the answers given by these language models is as important as their accuracy. Background To evaluate the accuracy and reproducibility of the responses of ChatGPT-4.0 to queries about intraocular lenses (IOLs) over a six-month interval. Methods This observational, cross-sectional study was conducted using 45 open-ended questions related to IOLs, categorised into three difficulty levels: easy, moderate, and difficult. Two evaluators independently posed these questions to ChatGPT-4.0 at two separate time points (April and October 2024) using the ‘new conversation’ feature. Responses were analysed for word count, character count, and plagiarism rates, and evaluated on a 5-point Likert scale for accuracy, relevance, and consistency. Results A total of 90 responses were analysed. Accuracy rates were 93.3%, 86.7%, and 80% for easy, moderate, and difficult questions, respectively. Significant increases in word and character counts were observed between the initial and final evaluations (p < 0.0001). Likert scores improved significantly over time (p = 0.046). ICC analysis showed moderate consistency (0.639) overall, with excellent reproducibility (1.00) for easy questions, acceptable (0.788) for moderate, and moderate (0.550) for difficult questions. Conclusions ChatGPT-4.0 demonstrated significant improvements in accuracy and response length over time. While the model exhibited high reproducibility for easy questions, its performance declined with increased question difficulty. Despite its limitations, ChatGPT-4.0 shows promise for use in medical information dissemination, highlighting the need for further studies to enhance its reliability and accuracy. 
TI - Responses of ChatGPT-4 on intraocular lenses: an evolving artificial intelligence assessment
JF - Clinical and Experimental Optometry
DO - 10.1080/08164622.2025.2515970
DA - 2025-06-28
UR - https://www.deepdyve.com/lp/taylor-francis/responses-of-chatgpt-4-on-intraocular-lenses-an-evolving-artificial-rZU0N0ED0D
SP - 1
EP - 5
VL - OnlineFirst
IS - 
DP - DeepDyve
ER -