Level of evidence in wrist ligament repair and reconstruction research: a systematic review

Protocols

This systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement (Moher et al., 2009) and follows The Cochrane Handbook for systematic Reviews of Interventions.

Eligibility criteria

Therapeutic studies that report on wrist ligament (SLL, LTL and TFCC/DRUJ) and instability repair (suture and re-insertion) and reconstruction with clinical outcome measurements related to the ligament repair or reconstruction were included. Articles with mixed cohorts with some patients operated on with ligament repair and others operated on with other methods were included. Expert opinions, case reports and technical reports on surgery for wrist ligament injuries and instability were also included. Studies on anatomical reconstruction of the TFCC and DRUJ (Adams & Berger, 2002), other ligament reconstructions and debridement plus pinning/shrinkage were included.

Studies on animals, cadavers, diagnostic tools, economics, epidemiology, imaging, diagnostic or anatomic studies without clinical outcome; rehabilitation protocols and studies on skeletally immature populations were excluded. Studies on perilunate dislocations, midcarpal instability, ulnar shortening osteotomy, bone corrective surgery, capsulodesis without concomitant ligament repair, pinning only, arthroscopic or open debridement alone without repair or shrinkage, interosseous membrane (IOM) reconstruction, salvage procedures (4-Cornerfusion, Proximal Row Carpectomy etc), partial fusions and implant arthroplasty were excluded. The purpose of the study was to analyse the level of evidence of studies on repair (suture and reinsertion) and reconstruction exclusively. Studies not written in English, comments, authors´ reply, letters to the editor, book chapters and instructional courses were also excluded.

No outcome measures were extracted from the studies, as this was not the aim of this systematic review.

Information sources and search

An electronic literature search of articles published between January 1, 1985 and May 24, 2016, in PubMed, Embase, and Cochrane Library was carried out in the end of May 2016, by an expert in electronic search strategies, at the Sahlgrenska University Hospital Library. Therapeutic studies written in English that report on wrist ligament repair and reconstruction with clinical outcome measurements related to the ligament repair or reconstruction were included, as well as LoE V publications on wrist ligament repair (suture and reinsertion) and reconstruction. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (Moher et al., 2009) checklist guided the extraction and reporting of data. Categorization and implementation of the level of evidence and journal distribution were performed. The systematic electronic search was updated on April 28, 2017, in order to identify newly published studies that were eligible for inclusion.

No manual search of recently published articles in pertinent journals was undertaken. Corresponding authors were not contacted for additional information. The complete electronic search strategies (initial and updated) are described in the Additional file 1.

The impact factors of the journals were acquired from the Thomson Reuters Journal Citation Reports (JCR) database (ISI Web of Knowledge, http://www.webofknowledge.com). The time span in which the impact factors were able to be acquired was from 2008 to 2015/16. The Eigenfactor (Bergstrom et al., 2008) of the journals were acquired from http://www.eigenfactor.org. The time span in which the Eigenfactors were able to be acquired was from 1997 to 2013.

Study selection

The first (JKA) and second (BR) author performed the study selection. All articles, generated by the electronic search, were screened by reading the title and abstract. The articles were validated in duplicate. The first author double-checked the extracted data by processing the included studies once again, with full agreement with the prior data selection and extraction.

If initial screening failed to provide sufficient information for the purpose of inclusion or exclusion, the full text of the article was always assessed by paper copies. In terms of 35 studies, it was necessary to retrieve full text to be able to decide about inclusion or exclusion. The investigator was not blinded to the names of authors or journals during the screening and data-extraction process.

Data extraction and data items

Study characteristics

A total of 1889 studies were analyzed, of which 362 were included. Figure 1 shows the flowchart of inclusion and exclusion of this systematic review.

Outcome measures

Level of evidence

Analysis of the study types revealed that expert opinions, LoE V (n = 203) was the most frequent study type. Randomized controlled trials (level of evidence I) and level of evidence (LoE) II studies represented 0% (n = 0) of the sample. Thus, not one LoE I or II study could be found during 30 years of research in the area of wrist ligament injury surgery. LoE III studies represented 3.0% (n = 11), LoE IV studies represented 40.9% (n = 148), and LoE V 55.1% (n = 203). The distribution (%) of the included studies in terms of level of evidence is displayed in Fig. 2. Disagreement between the level of evidence as assessed by the publishing journal and the level of evidence as assessed by the researchers was noticed in two cases.

In general the included studies included small sample sizes (in average 23 patients) and rather short follow up (in average 37 months) – only 2% displayed long-term follow up (> 10 years) and 7% mid-term follow up (6–8 years). 91% of the included studies displayed short-term follow up (2 years). Over 80% of the LoE IV studies were retrospective.

Distribution of publications

Six journals; J Hand Surg Am, J Hand Surg Br/Eur, Hand Clin, Arthroscopy, Tech Hand Up Extrem Surg and J Wrist Surg represented 56.4% (n = 204) of the included studies. The distribution of publications in different journals (numbers, percentage) and their impact factors are displayed in Fig. 3 and Table 2. In fact, three journals; J Hand Surg Am, J Hand Surg Br/Eur and Hand Clin represented 40% of the included studies (n = 145).

Journal	Impact factor (2015)	Number (Total n = 362)	%
J Hand Surg Am (JHSA)	1.64	72	19.9
Hand Clin	0.70	46	12.7
J Hand Surg Eur/Br (JHSE)	1.87	28	7.7
Arthroscopy (A-scopy)	3.72	21	5.8
Tech Hand Up Extrem Surg (THUES)	1.36 (2010)	20	5.5
J Wrist Surg (JWS)	No data	17	4.7
Subtotal		204	56.4
Others		158	43.6

Trends

Only 21.5% (n = 78) of the studies (66 authors) were published before 2000, in which 0% represented LoE I and II studies, 1.4% (n = 1) represented LoE III studies, 31.9% (n = 23) represented LoE IV studies and 66.7% (n = 48) represented LoE V. Almost four fifthts of the included studies were published between 2000 and 2016. There was no trend toward higher level of evidence over time. The updated electronic search performed on April 28, 2017 did not generate any new studies with high level of evidence (LoE I-II) eligible for inclusion.

Studies on TFCC (n = 117) and TFCC/DRUJ (n = 65) as well as SLL (n = 126) dominate in the literature. The distribution of studies on different wrist ligaments injuries and injury combinations are displayed in the Venn diagram below (Fig. 4).

The Impact factor and Eigenfactor for the most common journals are shown in Fig. 5a and b. No data were available for Tech Hand Up Extrem Surg and J Wrist Surg.

In general the Impact factor was relatively low, slightly increasing over years. The Eigenfactor followed the Impact factor.

Geographic distribution

Level of evidence

This systematic review shows that most therapeutic studies on repair and reconstruction of wrist ligament injuries were of low level of evidence (LoE IV-V). Analysis of the study types revealed that expert opinions, level of evidence V (55.9%, n = 203) was the most frequent study type and no studies of LoE I-II were found between 1985 and April 2017.

In the present study, in general, the Impact factor was relatively low, slightly increasing over years. The Eigenfactor followed the Impact factor. Eigenfactor (Bergstrom et al., 2008) score is correlated with the impact factor and the total citation count for medical journals, but these metrics provide significantly different information. For a given number of citations, citations from more significant journals will result in a higher Eigenfactor score. The result in the present study indicates that citations were given in similar journal that publications about wrist ligament surgery are published in.

Distribution

Frequency tables for geographic distribution revealed that the United States was the country with the highest number of publications, especially in terms of the two most common journals (J Hand Surg Am – 65% and Hand Clin – 87%). Frequently recurrent authors were common. Studies with LoE V were particularly common in Hand Clin (89%) and Tech Hand Up Extrem Surg (75%).

J Hand Surg Br/Eur, J Wrist Surg and to some extent also Arthroscopy displayed a more diversified geographic frequency distribution and a relatively lower rate of LoE V studies.

To be able to recommend surgery and what kind of procedure in specific cases, the hand surgery community needs larger studies with a randomized prospective assessment. A more diversified geographic distribution in some journals could also be desirable and worthwhile.

Up to this date, the patients´ needs and the surgeon’s specific skills seem to have had a vast impact in choosing type of surgical treatment for SLL, LTL and TFCC/DRUJ injuries (Garcia-Elias et al., 2006; Kitay & Wolfe, 2012; van de Grift & Ritt, 2016; Anderson et al., 2008; Zimmerman & Jupiter, 2014). A surgical method that works in the hands of one surgeon may not work in the hands of others. Therefore studies at a high level of evidence are needed to evaluate different therapeutic methods in terms of wrist ligament injuries. Every clinician has to continuously critically review and re-evaluate his decision of surgical options, especially as we conclude that the level of evidence - in terms of our common recommendations - are really low.

Larger prospective randomized controlled trials (RCTs) are needed to evaluate the outcome of wrist ligament surgery, in terms of minimizing bias and confounding factors. Multi-center studies or non-inferiority studies could also be an option, saving time and costs in the study setting, compared to RCTs.

The literature in terms of wrist ligament repair and reconstruction area has increased dramatically over the last 15 years. Almost 80% of the included studies were published between 2000 and 2016. This systematic review showed that there was no clear trend toward higher level of evidence over time.

As 56.5% of the studies were published in six of the 74 indexed registered orthopaedic journals - and 40% in three of those journals - this could indicate a risk of publication bias.

Limitations

The search was limited to papers written in English indexed in PubMed, Embase, and Cochrane library. Studies in other languages and cited in other databases were therefore not included in this review. The data extraction was not performed in a blinded fashion, that is, by blacking out authors, title, and so on.

To our knowledge, this is the first systematic review on categorizing the study type and level of evidence of studies on repair and reconstruction of SLL, LTL and TFCC/DRUJ injuries by applying the level of evidence rating system proposed by the Oxford Centre for Evidence-Based Medicine with this extensive electronic search. This extensive search, of course, creates a large number of abstracts to analyze, which creates the possibility that some studies were classified into the wrong category. However, our results are so clear-cut, that wrong categorization in some few cases, with the highest probability, would not affect our conclusion.