Evaluating Guideline Adherence in Gemini-Powered Dental Trauma Workflows: Standalone Gemini Chat vs. Document-Grounded NotebookLM

Abstract

Aim: The aim of this study was to compare the accuracy and inter-account consistency of two Google Gemini–powered, user-facing workflows for dental trauma decision support: standalone Gemini chat and NotebookLM, a document-grounded work-flow that generates responses grounded in uploaded European Society of Endodontology and International Association of DentalTraumatology guideline documents, when answering dichotomous (yes/no) clinical questions on the management of trauma-tized permanent teeth.Methodology: A cross-sectional simulation was conducted using 99 dichotomous (yes/no) questions derived from the EuropeanSociety of Endodontology and International Association of Dental Traumatology guidelines. Three academic endodontists sub-mitted each question to Gemini and NotebookLM using three independent Google accounts, generating 297 responses per work-flow. Accuracy was defined as exact agreement with guideline-based answers, and consistency as the proportion of identicalresponses across the three trials. Statistical analyses included Wald and Wilson 95% confidence intervals, Fleiss' kappa for inter-account agreement, and Pearson's chi-squared tests to compare proportions.Results: Gemini demonstrated an overall accuracy of 83.83% (95% CI: 75.08–90.47) and a consistency of 74.74% (κ = 0.84).NotebookLM showed higher accuracy (92.93%; 95% CI: 85.97–97.11) and perfect consistency (100%; κ = 1.00). While the dif-ference in accuracy did not reach statistical significance (p = 0.076), NotebookLM exhibited significantly greater consistency(p < 0.001).Conclusions: The responses generated from the guidelines were highly consistent with both workflows. Document groundingmay enhance repeatability and alignment with guideline-derived decision points for structured dichotomous inquiries, as evi-denced by NotebookLM's ability to achieve complete inter-account consistency and to quantitatively increase accuracy. Theseresults are the outcome of workflow-level benchmarking; therefore, clinical utility cannot be inferred solely from them; profes-sional oversight and additional validation remain necessary before any clinical application.