Development and validation of a search strategy and an automated classifier for retrieving temporomandibular disorders studies

The objective was to develop and evaluate a comprehensive search strategy (SS) and automated classifier (AC) for retrieving temporomandibular disorders (TMD) research articles. An initial version of SS and AC was created by compiling terms from various sources, including previous systematic reviews (SRs) and consulting with TMD specialists. Performance was assessed using the relative recall (RR) method against a sample of all the primary studies (PS) included in 100 TMD-related SRs, with RR calculated for both SS and AC based on their ability to capture/classify TMD PSs. Adjustments were made iteratively. A validation was performed against PSs included in all TMD-relevant SRs published from January to April 2023. The analysis included 1271 PSs from 100 SRs published between 2002–2022. The initial SS had a relative recall of 89.34%, while the AC detected 70.05% of the studies. After adjustments, the fifth version reached 99.5% and 89.5% relative recall, respectively. Validation with 28 SRs from 2023 showed a search strategy sensitivity of 99.67% and AC sensitivity of 88.04%. In conclusion, the proposed SS demonstrated excellent performance in retrieving TMD-related research articles, with only a small percentage not correctly classified by the AC. The SS can effectively support evidence synthesis related to TMD, while the AC can aid in creating an open-access, continuously updated digital repository for all relevant TMD evidence.

Temporomandibular disorders; Research methodology; Evidence-based dentistry; Systematic review; Automated classification

[1] Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLOS Medicine. 2010; 7: e1000326.

[2] Rada G, Pérez D, Araya-Quintanilla F, Ávila C, Bravo-Soto G, Bravo-Jeria R, et al. Epistemonikos: a comprehensive database of systematic reviews for health decision-making. BMC Medical Research Methodology. 2020; 20: 286.

[3] Saltaji H, Cummings GG, Armijo-Olivo S, Major MP, Amin M, Major PW, et al. A descriptive analysis of oral health systematic reviews published 1991–2012: cross sectional study. PLOS ONE. 2013; 8: e74545.

[4] Bassani R, Pereira GKR, Page MJ, Tricco AC, Moher D, Sarkis-Onofre R. Systematic reviews in dentistry: current status, epidemiological and reporting characteristics. Journal of Dentistry. 2019; 82: 71–84.

[5] European Centre for Disease Prevention and Control; Hocking L, Parkinson S, Adams A, Nielsen E, Ang C, Gomes HDC. Use and impact of new technologies for evidence synthesis: literature review and qualitative data collection. European Centre for Disease Prevention and Control: Stockholm. 2022.

[6] Klasser GD, Romero M, American Academy of Orofacial Pain. Orofacial pain: guidelines for assessment, diagnosis, and management. Seventh edition. Quintessence Publishing: USA. 2023.

[7] Peck CC, Goulet J, Lobbezoo F, Schiffman EL, Alstergren P, Anderson GC, et al. Expanding the taxonomy of the diagnostic criteria for temporomandibular disorders. Journal of Oral Rehabilitation. 2014; 41: 2–23.

[8] Bond EC, Mackey S, English R, Liverman CT, Yost O. Committee on Temporomandibular Disorders (TMDs): from research discoveries to clinical treatment, board on health sciences policy, board on health care services, health and medicine division, national academies of sciences, engineering, and medicine. In Bond EC, Mackey S, English R, Liverman CT, Yost O (eds.) Temporomandibular disorders: priorities for research and care. National Academies Press: USA. 2020.

[9] Orofacial T. International classification of orofacial pain, 1st edition (ICOP). Cephalalgia. 2020; 40: 129–221.

[10] Schiffman E, Ohrbach R, Truelove E, Look J, Anderson G, Goulet J, et al. Diagnostic criteria for temporomandibular disorders (DC/TMD) for clinical and research applications: recommendations of the international RDC/TMD consortium Network and orofacial pain special interest group. Journal of Oral & Facial Pain and Headache. 2014; 28: 6–27.

[11] Wielandt V, Oyarzo JF, Verdugo F. Description of the commonly used terms in TMD systematic reviews: TMD LOVE study. Journal of Dental Research. 2022; 101: 1254.

[12] Github repository. Available at: https://github.com/dperezrada/keywords2vec (Accessed: 12 March 2023).

[13] Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, et al. An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Medical Research Methodology. 2006; 6: 33.

[14] Hamada Y, Kondoh T, Holmlund AB, Yamamoto M, Horie A, Saito T, et al. Inflammatory cytokines correlated with clinical outcome of temporomandibular joint irrigation in patients with chronic closed lock. Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology, and Endodontology. 2006; 102: 596–601.

[15] Visser A, Kroon GW, Naeije M, Hansson TL. EMG differences between weak and strong myogenous CMD patients and healthy controls. Journal of Oral Rehabilitation. 1995; 22: 429–434.

[16] Lyrio AO, da Cruz SS, Gomes-Filho IS, Silveira VSS, Souza ES, Batista JET, et al. Validation of a search strategy for randomized clinical trials related to periodontitis. Archives of Public Health. 2021; 79: 43.

[17] Nylander E, Ramstrand N, Hjort M, Rusaw DF. Development and validation of a sensitive MEDLINE search strategy to identify literature relevant to limb prostheses. Prosthetics & Orthotics International. 2021; 45: 289–294.

[18] Selva A, Solà I, Zhang Y, Pardo-Hernandez H, Haynes RB, Martínez García L, et al. Development and use of a content search strategy for retrieving studies on patients’ views and preferences. Health and Quality of Life Outcomes. 2017; 15: 126.

[19] Fiorini N, Canese K, Starchenko G, Kireev E, Kim W, Miller V, et al. Best match: new relevance search for PubMed. PLOS Biology. 2018; 16: e2005343.

[20] Kiester L, Turp C. Artificial intelligence behind the scenes: PubMed’s best match algorithm. Journal of the Medical Library Association. 2022; 110: 15–22.

[21] Jin Q, Shin A, Lu Z. LADER: log-augmented dense retrieval for biomedical literature search. 2023. Available at: http://arxiv.org/abs/2304.04590 (Accessed: 21 November 2023).