The rate of publication of free papers at the 2008 and 2010 European Society of Sports Traumatology Knee Surgery and Arthroscopy congresses
© The Author(s). 2017
Received: 2 March 2017
Accepted: 27 April 2017
Published: 10 May 2017
The purpose of this study was to evaluate the frequency with which free papers presented at the 2008 and 2010 European Society of Sports Traumatology Knee Surgery and Arthroscopy (ESSKA) congress were ultimately published in peer-reviewed journals. Moreover, this study evaluated whether any correlations exist between the level of evidence of the free papers and their frequency of publication or the impact factor of the journals in which they are published.
Free papers presented at the 2008 and 2010 ESSKA congresses were included for assessment. Clinical papers (observational studies and trials involving direct interaction between an investigator and human subjects) were graded for level of evidence by two independent reviewers. A comprehensive strategy was used to search the databases PubMed, Ovid (MEDLINE), and EMBASE for all publications corresponding to the included free papers.
Three hundred-ninety presentations were evaluated, of which 215 (55%) were ultimately published in a peer-reviewed journal within five years of the presentation date. The mean time from presentation to publication was 16 months (SD 25 months). There was no significant difference in the distribution of the level of evidence between studies that were ultimately published, versus those that were not published (n.s.). The level of evidence of the published study was not a significant predictor of the impact factor of the journal in which it was published (n.s.). Presentations were most commonly published in Knee Surgery, Sports Traumatology, Arthroscopy (24%) and The American Journal of Sports Medicine (22%).
Free papers at the 2008 and 2010 ESSKA congress were published at a frequency that is comparable to that at other orthopaedic meetings. The publication rate was similar across all levels of evidence. Further encouragement of manuscript preparation and submission following these meetings could help to ensure important research findings are disseminated to large audiences.
To ensure that important research findings ultimately have an impact on clinical decisions, their timely dissemination to target audiences is critical. Often the results of the latest research trials are initially presented at scientific meetings before being published as full-text articles in peer-reviewed journals. The European Society of Sports Traumatology, Knee Surgery and Arthroscopy (ESSKA) biannual congress is one of the major venues for leading orthopaedic surgeons specializing in degenerative joint disease and sports medicine to present and learn about the latest techniques and research in these fields. The level of evidence of research presented at the ESSKA congress has demonstrated significant improvement over the past decade, with a high proportion of level I studies being presented (Kay et al. 2016a). While these meetings provide its attendees with important information in terms of the latest advancements, the findings that are presented are not usually implemented into clinical practice until after they are published in peer-reviewed journals and disseminated to much larger audiences (de SA et al. 2015). It is therefore critical that these presentations are followed up with manuscript preparation, and ultimately publication. The publication rate of presentations at other orthopaedic meetings such as the American Academy of Orthopaedic Surgeons (AAOS) (49%), Arthroscopy Association of North America (AANA) (55%) and the American Shoulder and Elbow Surgeons (ASES) (49%), suggest that a significant proportion of presentations at these meetings may not ultimately be widely disseminated to impact clinical decisions (Kay et al. 2016b; Kay et al. 2016c; Voleti et al. 2013). Moreover, there are conflicting results with respect the impact of the level of evidence of the presentations on the publication rate at these meetings. The publication rate of presentations at the ESSKA congress has not been evaluated to date.
The purpose of this study was to evaluate the frequency with which free papers presented at the 2008 and 2010 European Society of Sports Traumatology Knee Surgery and Arthroscopy (ESSKA) Congress were ultimately published in a peer-reviewed journal. Moreover, this study evaluated whether any correlations exist between the level of evidence of the free papers and their frequency of publication or the impact factor of the journals in which they are published.
Study eligibility and assessment
The methodology used in the present study is similar to that used in a previous study to assess the publication rate of presentations at the American Association of North America (AANA) meetings (Kay et al. 2016c). Inclusion criteria included free papers presented at the 2008 and 2010 ESSKA congresses, as these years would allow for evaluation of all corresponding papers published within five years of presentation. The abstracts of the free papers presented at the ESSKA biannual congresses have been published electronically by Knee Surgery, Sports Traumatology, Arthroscopy (KSSTA). Two reviewers (J.R. & M.M.) comprehensively searched PubMed, MEDLINE, and EMBASE using a modified version of the search strategy described by Bhandari et al. (Bhandari et al. 2002). For each free paper presented, the first, second, and last author were searched using these databases. Next, the Boolean operator ‘AND’ was used to combine this search with additional key words from the title or abstract of the study until the search yielded only one result. If the identified publication was published less than five years after the date of presentation, this publication was recorded and its information was abstracted. A 5-year time period was chosen as previous studies have demonstrated satisfactory accuracy using this time frame (Bhandari et al. 2002). All peer-reviewed journals, including open access journals, were included in the assessment. The abstracts were then screened independently and in duplicate by the reviewers for those deemed as clinical studies. More specifically the free papers were assessed for any trial or observational study that included direct interaction between an investigator and their human subjects. While these studies may ultimately have important clinical implications, they are unable to be classified into a level of evidence according to the AAOS classification scheme (Wright 2005). The included abstracts were then graded by the two reviewers independently for level of evidence using the AAOS classification scheme (Wright 2005).
Data extraction and statistical analysis
The relevant study data was extracted from the included free papers including the title, authors, location, sample size, journal of publication (if applicable), and joint of focus. These data were recorded in Microsoft Excel 2015 spreadsheets (Microsoft, Redmond, WA, USA). The Web of Science database was used to obtain the impact factors of each journal. In order to account for variation by year, the impact factor corresponding to the year of publication were used for the analysis. Kappa (κ) statistics were determined for the abstract screening and level of evidence evaluation stage to assess inter-reviewer agreement. The proportions of free papers that were published was determined by level of evidence and year of presentation. Means and standard deviations were calculated for the time to publication and impact factors. Chi-squared tests were used in order to test for changes in the proportion of published free papers and student t-tests as well as one way ANOVA tests were used when comparing the mean values of quantitative data. A p-value of 0.05 or less was considered to be significant. All statistics were calculated using Minitab ® statistical software version 17 (Minitab Inc., State College, USA).
The most important finding in the present study was identifying that more than half of the presentations at the 2008 and 2010 ESSKA congresses have ultimately been published in peer-reviewed journals. There was no effect of level of evidence on the publication rate, with similar frequencies of presentations published across all levels of evidence. Moreover, there was no association between the level of evidence and the impact factor of the journal in which the study was published. Presentations were most commonly published in Knee Surgery, Sports Traumatology, Arthroscopy followed by The American Journal of Sports Medicine.
The critical concepts of knowledge translation involve the synthesis and exchange of ethically and methodologically sound research in order to improve the health of patients (Graham et al. 2006). To accomplish these important goals, it is vital that important research findings are disseminated to large audiences so that they have the opportunity to influence clinical decisions. Publication of research findings in recognized peer-reviewed journals is a requisite step in this process. While the publication rate of presentations at the ESSKA congress has not yet been evaluated to date, the rate of publication of presentations at other orthopaedic meetings have been previously studied. The publication rate of presentations at the AAOS meetings was assessed for the time periods 1990–1992, 1993, 1996, and 2001. The publication rates of presentations over those time periods were respectively reported as 46%, 44%, 34%, and 49% (Voleti et al. 2013; Bhandari et al. 2002; Hamlet et al. 1997; Murrey et al. 1999). The publication rate of presentations at the Society of Military Orthopaedic Surgeons meetings from 1999 to 2003 was reported as 44% (Fuller et al. 2012). Presentations at the 1997 and 1999 International Society of Arthroscopy, Knee Surgery & Orthopaedic Sports Medicine (ISAKOS) meetings were published with frequencies of 35% and 39% respectively (Eck 2005). The publication rate of 55% identified in the present study is greater than, although comparable, to these previously reported rates. Overall, 12% of the presentations at the ESSKA congresses had been published before the date of the presentation. This number may be one indication of the originality of the presentations at a meeting. This number is slightly higher than the 3% and 7% of papers that were published before their corresponding presentations at the 2005–2010 ASES and 2006–2010 AANA meetings, respectively. One possible explanation for this discrepancy may relate to the bi-annual nature of the ESSKA congress which allows a longer time period for the presentations to get published before they are actually presented (Kay et al. 2016b; Kay et al. 2016c).
Conflicting results have been reported with respect to the association between the level of evidence and the publication rate of presentations at orthopaedic meetings. Voleti at al. reported a significantly improved publication rates in presentations of higher levels of evidence at the 2001 AAOS meeting (Voleti et al. 2013). On the other hand, no such correlation was identified at either the 2006–2010 AANA meetings or at the 2005–2010 ASES meetings (Kay et al. 2016b; Kay et al. 2016c). The present study found no association between the level of evidence and the publication rates of free papers presented at the 2008 and 2010 ESSKA meetings. The level of evidence of a study is one method of screening its quality. The assessment of level of evidence in orthopaedic research has been published by the AAOS, in a system adopted by that described in the Journal of Bone and Joint Surgery (Wright 2005). The system assigns a level of evidence to each study with methodological designs such as high quality randomized controlled trials graded as a level I, whereas case series’ and reports are assigned a level IV. The idea of such a system would be that research that has been conducted using study designs that inherently limit the chance of bias would be more likely to have meaningful clinical impact than research conducted using a study design that is less rigorous. However, Marx et al. have demonstrated that the level of evidence of a paper may not provide a direct reflection of its clinical importance (Marx et al. 2015).
This study is the first to evaluate the frequency with which presentations at the ESSKA biannual congress have been published in peer-reviewed journals. Strengths of this study include the use of multiple databases and an almost perfect level of inter-reviewer agreement at both the screening (κ = 0.96) and level of evidence evaluation (κ =0.87) stages. Although multiple databases were used, there is a possibility that some full-text publications were not identified by the search strategy, particularly if the title of the presentation had changed when the manuscript was being written. Thus, the publication rate identified by the present study is likely lower than the true frequency with which presentations at the ESSKA congress are published. However, a change in title would likely have a small effect on the ability to identify the project, considering the search strategy involved searching the first and last authors of the study rather than the title. Moreover, the results are encouraging considering a similar publication rate to that identified at other orthopaedic meetings (Kay et al. 2016b; Kay et al. 2016c). The methodological quality of the study design used by the presentations was assessed based on the abstracts alone. Given the word constraints inherent in the short abstracts, the assessment in the present study may not have provided a completely thorough representation of the methodological quality of the studies. Although it has been found that some orthopedic presentations are included at multiple meetings, (Bhandari et al. 2005) the present study only considered the presentations at the ESSKA congress. It is possible that presentations at multiple meetings could impact the quality of the research being presented. Non-clinical studies including cadaveric studies, animal studies and technique demonstrations were not included for assessment in the present study. Although these studies provide critical information for those attending the ESSKA congresses, and they may ultimately have important clinical impact, they cannot be graded for level of evidence using the AAOS classification system and would not provide for a meaningful comparison.
Although the proportion of free papers presented at the 2008 and 2010 ESSKA congress that were ultimately published is similar to the frequency identified at other orthopaedic meetings, about half of the free papers that were presented remain unpublished. Moreover, there was no association between the level of evidence of the free papers and the rate at which the presentations were published. One possible explanation for these findings would be that the free papers that are unpublished are not, in fact, being rejected for publications, but rather they are not being submitted as full-text manuscripts to peer-reviewed journals by the respective authors. Sprague et al. studied presentations that remained unpublished after AAOS meetings, and found that only 25% of these studies were rejected for publication by a peer-reviewed journal (Sprague et al. 2003). The other 75% of unpublished papers were never submitted for publication, with the most commonly given reasons being “insufficient time to prepare a manuscript”, “co-authors moving or changing institutions”, and “manuscript still in progress” (Sprague et al. 2003). If similar factors have been preventing the authors of free papers from the ESSKA congresses to prepare manuscripts for submission, there may be important clinical findings that are never disseminated to large audiences and impacting clinical decisions. It is therefore important to ensure that the latest research findings presented at the ESSKA congress are submitted for peer-reviewed publication to allow for the widespread dissemination of the highest quality research. One potential policy that might, if implemented, improve the publication rate of presentations at the ESSKA congress involves encouraging the submission of a complete manuscript accompanying each abstract or within a set period of time following each presentation. Future research might evaluate such strategies and effects on the publication rate at subsequent meetings.
Free papers at the 2008 and 2010 ESSKA congress were published at a frequency that is comparable to that at other orthopaedic meetings. The publication rate was similar across all levels of evidence. Moreover, there was no association between the level of evidence of a paper and the impact factor of the journal in which it was published. Further encouragement of manuscript preparation and submission following these meetings could help to ensure important research findings are disseminated to large audiences.
This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.
JK lead author, contributed to all elements of the study; specifically: study design; primary literature reviewer; responsible for executing the search for abstracts; data abstraction, analysis and presentation; primary manuscript writer and responsible for all encompassing and all subsequent revisions throughout the editing process. MM contributed to study design, reviewer responsible for grading level of evidence, and contributed to data abstraction, manuscript preparation and revision. JR contributed to study design, reviewer responsible for grading level of evidence, and contributed to data abstraction, manuscript preparation and revision. DdeSA served as a third reviewer to resolve any discrepancies between the two primary duplicate literature reviewers; contributed to study design, data analysis, and manuscript preparation. NS content expert, contributed to study design, data analysis, and manuscript preparation and revision. RS content expert, contributed to study design, data analysis, and manuscript preparation and revision. JK content expert, contributed to study design, data analysis, and manuscript preparation and revision ORA Study supervisor and content expert; contributed to all elements of the study, specifically: study design; development of the literature search strategy and grading process; assisted with data analysis and presentation; manuscript preparation and revision. All authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
- Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S, Schemitsch EH (2002) An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg 84:615–621View ArticlePubMedGoogle Scholar
- Bhandari M, Patenall V, Devereaux PJ, Tornetta P, Dirschl D, Leece P, Ramanan T, Schemitsch EH (2005) An observational study of duplicate presentation rates between two national orthopedic meetings. Can J Surg 48:117–122PubMedPubMed CentralGoogle Scholar
- de SA D, Thornley P, Evaniew N, Madden K, Bhandari M, Ghert M (2015) CHAracteristics of research studies that iNfluence practice: a GEneral survey of Canadian orthopaedic Surgeons (CHANGES): a pilot survey. Springerplus 4:62View ArticlePubMedGoogle Scholar
- Eck JC (2005) Publication rates of abstracts presented at Biennial Meetings of the International Society of Arthroscopy, Knee Surgery and Orthopaedic Sports Medicine. Knee Surg Sport Traumatol Arthrosc 13:426–429View ArticleGoogle Scholar
- Fuller B, Dowd T, Masini B, Gerlinger T (2012) Publication rates of abstracts presented at the annual meeting of the Society of Military Orthopaedic Surgeons. J Surg Orthop Adv 21:88–91View ArticlePubMedGoogle Scholar
- Graham ID, Logan J, Harrison MB, Straus SE, Tetroe J, Caswell W, Robinson N (2006) Lost in knowledge translation: time for a map? J Contin Educ Health Prof 26:13–24View ArticlePubMedGoogle Scholar
- Hamlet WP, Fletcher A, Meals RA (1997) Publication patterns of papers presented at the Annual Meeting of The American Academy of Orthopaedic Surgeons. J Bone Joint Surg 79:1138–1143View ArticlePubMedGoogle Scholar
- Kay J, Memon M, Rogozinsky J, Simunovic N, Seil R, Karlsson J, Ayeni OR (2016a) Level of evidence of free papers presented at the European Society of Sports Traumatology, Knee Surgery and Arthroscopy congress from 2008 to 2016. Knee Surg Sport Traumatol Arthrosc. doi:10.1007/s00167-016-4391-8 Google Scholar
- Kay J, Memon M, de Sa D, Duong A, Simunovic N, Athwal GS, Ayeni OR (2016b) Five-year publication rate of clinical presentations at the open and closed American shoulder and elbow surgeons annual meeting from 2005–2010. J Exp Orthop 3:21View ArticlePubMedPubMed CentralGoogle Scholar
- Kay J, Memon M, de SA D, Duong A, Simunovic N, Ayeni OR (2016c) Does the Level of Evidence of Paper Presentations at the Arthroscopy Association of North America Annual Meeting From 2006–2010 Correlate With the 5-Year Publication Rate or the Impact Factor of the Publishing Journal? Arthroscopy 33:12–18View ArticlePubMedGoogle Scholar
- Marx RG, Wilson SM, Swiontkowski MF (2015) Updating the Assignment of Levels of Evidence. J Bone Joint Surg 97:1–2View ArticlePubMedGoogle Scholar
- Murrey DB, Wright RW, Seiler JG, Day TE, Schwartz HS (1999) Publication rates of abstracts presented at the 1993 annual Academy meeting. Clin Orthop Relat Res 359:247–253Google Scholar
- Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta P, Cook DJ, Dirschl D, Schemitsch EH, Guyatt GH (2003) Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg 85:158–163View ArticlePubMedGoogle Scholar
- Voleti PB, Donegan DJ, Kim TWB, Lee G-C (2013) Level of evidence: does it change the rate of publication and time to publication of American Academy of Orthopaedic Surgeons presentations? J Bone Joint Surg 95, e2View ArticlePubMedGoogle Scholar
- Wright JG (2005) Levels of evidence and grades of recommendations. AAOS Bull 53 http://www2.aaos.org.libaccess.lib.mcmaster.ca/bulletin/apr05/fline9.asp Accessed: 21 September 2016