This process has clearly designated steps to identify primary studies and the methods which will be employed to assess their methodological quality, the way in which data will be extracted, and the statistical techniques that will be used in the synthesis and reporting of that data.
The need for rigor in the production of systematic reviews has led to the development of a formal process for their conduct. Although there is an increasing requirement for randomized controlled trials to guide healthcare decision-making, the synthesis of the results of more than one RCT in a systematic review can summarize the effects of their individual outcomes and provide numerical answers about the effectiveness of a particular intervention.Ī systematic review is a summary of the medical literature that uses explicit methods to systematically search, critically appraise, and synthesize the data on a specific topic. Randomized controlled trials (RCTs) play a pivotal role in medical research and are widely considered to be the best way of achieving results that can genuinely increase our knowledge about treatment effectiveness. The strongest features of the app, identified and reported in user feedback, were its ability to help in screening and collaboration as well as the time savings it affords to users. Feedback from users, obtained mostly through the app web site and a recent survey, has highlighted the ease in exploration of searches, the time saved, and simplicity in sharing and comparing include-exclude decisions. In addition, around 75% of the respondents mentioned that screening and labeling studies as well as collaborating on reviews to be the two most important features of Rayyan.Īs of November 2016, Rayyan users exceed 2000 from over 60 countries conducting hundreds of reviews totaling more than 1.6M citations. The survey respondents reported 40% average time savings when using Rayyan compared to others tools, with 34% of the respondents reporting more than 50% time savings. Post rollout user experiences and a reflexive response by the developers enabled real-time modifications and improvements. The “suggestions” and “hints,” based on the “prediction model,” appeared as testing progressed beyond five included studies. The second review (1030 records) required several iterations to identify the previously identified 11 trials. The “taster” review (273 records) allowed a quick overview of Rayyan for early comments on usability.
Pilot testing of Rayyan focused on usability, accuracy against manual methods, and the added value of the prediction feature. We also conducted a survey of Rayyan’s users and collected feedback through a built-in feature. Different features of Rayyan were tested using these two reviews. Their searches, with 1030 records and 273 records, were uploaded to Rayyan. For the beta testing phase, we used two published Cochrane reviews in which included studies had been selected manually.
We developed Rayyan ( ), a free web and mobile app, that helps expedite the initial screening of abstracts and titles using a process of semi-automation while incorporating a high level of usability. Automation of systematic reviews is driven by a necessity to expedite the availability of current best evidence for policy and clinical decision-making. Filtering of searches is time consuming, and no single method fulfills the principal requirements of speed with accuracy.
Synthesis of multiple randomized controlled trials (RCTs) in a systematic review can summarize the effects of individual outcomes and provide numerical answers about the effectiveness of interventions.