Semiautomated screening in living guideline maintenance: a simulation study of 90 machine learning-prioritized screening system configurations (protocol).

Abstract

Background Living guidelines are an emerging approach to ensuring guideline recommendations remain relevant to the needs of stakeholders and frontline clinical care. However, development and maintenance remain manual. Single-database workflows involving an overarching search and triage of articles by a single-guideline development team have emerged in this context to improve the efficiency of evidence synthesis. However, there has been limited investigation in how technological enablers such as machine learning (ML)-prioritized screening may be integrated to further compound work savings. This stage I registered report details the protocol for a retrospective simulation study aiming to simulate and evaluate 90 ML-prioritized screening system configurations to streamline the title and abstract screening phase of living guideline development. Methods A total of five feature extraction techniques (Term frequency inverse document frequency, Word2Vec, SentenceBERT, Specter2, and BioLinkBERT), two classification algorithms (logistic regression, support vector machines), three training strategies (no retraining, adaptive retraining, and incremental retraining) and three stopping rules (data driven, time-driven, and statistical base criteria) will be simulated and evaluated. Each combination will be evaluated based on capacity to save work over sampling, at a target recall of 95%. Computation bottlenecks within the system that may adversely affect user experience and limit adoption by guideline development teams will also be investigated. Lastly, risk-calibrated threshold values targeting recall of 95% for various stopping rules will be investigated. The study will be conducted on a dataset generated from the International 2023 Polycystic Ovary Syndrome (PCOS) guidelines, and the simulation will be conducted on consumer level hardware to better simulate real world scenarios. Conclusion This study will inform ongoing work transitioning the International 2023 PCOS guidelines into a living format. The results will also provide guideline development teams at large with empirically derived guidance for adopting pragmatic, single database ML-prioritized screening into their own guideline maintenance or development workflows. It will also provide a realistic estimate for potential work savings. With these insights, the study will advance both the theoretical understanding and practical implementation of ML-assisted evidence synthesis, informing how living guidelines can remain sustainable into the future.Copyright © 2026 The Author(s).