Pragmatic Free Trial Meta
Pragmatic Free Trail Meta is an open data platform that allows research into pragmatic trials. It collects and distributes cleaned trial data, ratings, and evaluations using PRECIS-2. This allows for a variety of meta-epidemiological analyses that compare treatment effect estimates across trials of various levels of pragmatism.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. The term "pragmatic", however, is a word that is often used in contradiction and its definition and assessment need further clarification. The purpose of pragmatic trials is to guide clinical practice and policy decisions, not to confirm the validity of a clinical or physiological hypothesis. A pragmatic study should strive to be as close as possible to real-world clinical practices which include the recruiting participants, setting, design, delivery and execution of interventions, determination and analysis outcomes, and primary analysis. This is a major distinction between explanation-based trials, as described by Schwartz & Lellouch1 which are designed to prove the hypothesis in a more thorough manner.
Trials that are truly pragmatic should not attempt to blind participants or the clinicians, as this may cause bias in estimates of the effect of treatment. Pragmatic trials will also recruit patients from different health care settings to ensure that their results can be applied to the real world.
Finally, pragmatic trials must be focused on outcomes that matter to patients, such as the quality of life and functional recovery. This is particularly important in trials that require the use of invasive procedures or could have dangerous adverse impacts. The CRASH trial29 compared a two-page report with an electronic monitoring system for patients in hospitals with chronic cardiac failure. The catheter trial28 however utilized symptomatic catheter-related urinary tract infections as its primary outcome.
In addition to these features, pragmatic trials should minimize the requirements for data collection and trial procedures to cut costs and time commitments. In the end, pragmatic trials should aim to make their findings as applicable to current clinical practice as is possible. This can be achieved by ensuring that their analysis is based on the intention-to treat method (as described in CONSORT extensions).
Despite these criteria, many RCTs with features that defy the concept of pragmatism have been mislabeled as pragmatic and published in journals of all kinds. This could lead to misleading claims of pragmatism and the usage of the term must be standardized. The development of a PRECIS-2 tool that can provide a standardized objective evaluation of pragmatic aspects is a good start.
Methods
In a pragmatic study the aim is to inform clinical or policy decisions by demonstrating how an intervention could be integrated into routine care in real-world settings. Explanatory trials test hypotheses about the causal-effect relationship in idealized conditions. In this way, pragmatic trials could have a lower internal validity than studies that explain and are more susceptible to biases in their design, analysis, and conduct. Despite their limitations, pragmatic research can be a valuable source of data for making decisions within the context of healthcare.
The PRECIS-2 tool evaluates an RCT on 9 domains, ranging from 1 to 5 (very pragmatic). In this study, the recruitment, organisation, flexibility: delivery, flexible adherence and follow-up domains were awarded high scores, however, the primary outcome and the method for missing data were below the limit of practicality. This suggests that a trial can be designed with effective practical features, but without harming the quality of the trial.
It is hard to determine the level of pragmatism in a particular trial because pragmatism does not have a binary characteristic. Some aspects of a study can be more pragmatic than other. A trial's pragmatism could be affected by modifications to the protocol or logistics during the trial. Koppenaal and colleagues found that 36% of 89 pragmatic studies were placebo-controlled, or conducted prior to licensing. 프라그마틱 슬롯 found that the majority were single-center. They aren't in line with the standard practice and can only be referred to as pragmatic if their sponsors accept that these trials are not blinded.
A common feature of pragmatic studies is that researchers try to make their findings more relevant by studying subgroups of the trial sample. However, this can lead to unbalanced results and lower statistical power, thereby increasing the likelihood of missing or misinterpreting the results of the primary outcome. This was a problem during the meta-analysis of pragmatic trials because secondary outcomes were not adjusted for differences in covariates at the time of baseline.
In addition, pragmatic trials can also present challenges in the collection and interpretation of safety data. This is due to the fact that adverse events are usually self-reported and are prone to reporting delays, inaccuracies or coding deviations. It is essential to improve the accuracy and quality of outcomes in these trials.
Results
While the definition of pragmatism may not mean that trials must be 100% pragmatic, there are some advantages to including pragmatic components in clinical trials. These include:
By including routine patients, the results of trials can be more quickly translated into clinical practice. However, pragmatic trials can also have disadvantages. The right kind of heterogeneity, for example could allow a study to generalise its findings to many different patients or settings. However the wrong type of heterogeneity could decrease the sensitivity of the test and, consequently, reduce a trial's power to detect even minor effects of treatment.
A variety of studies have attempted to categorize pragmatic trials with various definitions and scoring systems. Schwartz and Lellouch1 have developed a framework that can differentiate between explanation studies that prove a physiological hypothesis or clinical hypothesis, and pragmatic studies that guide the selection of appropriate treatments in real world clinical practice. Their framework comprised nine domains that were scored on a scale ranging from 1 to 5, with 1 indicating more explanatory and 5 suggesting more pragmatic. The domains included recruitment, setting up, delivery of intervention, flexible compliance and primary analysis.
The original PRECIS tool3 featured similar domains and a scale of 1 to 5. Koppenaal et al10 developed an adaptation of the assessment, dubbed the Pragmascope, that was easier to use for systematic reviews. They found that pragmatic reviews scored higher on average in all domains, but scored lower in the primary analysis domain.
The difference in the main analysis domain could be explained by the fact that most pragmatic trials process their data in an intention to treat manner, whereas some explanatory trials do not. The overall score was lower for pragmatic systematic reviews when the domains on the organization, flexibility of delivery and follow-up were merged.
It is important to remember that a pragmatic study does not mean that a trial is of poor quality. In fact, there is an increasing number of clinical trials that use the word 'pragmatic,' either in their title or abstract (as defined by MEDLINE, but that is neither sensitive nor precise). The use of these terms in titles and abstracts could indicate a greater understanding of the importance of pragmatism but it isn't clear if this is manifested in the contents of the articles.
Conclusions
In recent years, pragmatic trials are increasing in popularity in research because the value of real world evidence is increasingly recognized. They are randomized studies that compare real-world care alternatives to new treatments that are being developed. They are conducted with populations of patients more closely resembling those treated in regular care. This method is able to overcome the limitations of observational research for example, the biases that come with the reliance on volunteers and the lack of codes that vary in national registers.
Other advantages of pragmatic trials are the ability to utilize existing data sources, and a greater probability of detecting significant changes than traditional trials. However, these trials could have some limitations that limit their credibility and generalizability. For example the participation rates in certain trials could be lower than anticipated due to the healthy-volunteer influence and financial incentives or competition for participants from other research studies (e.g. industry trials). Many pragmatic trials are also limited by the need to enroll participants in a timely manner. Additionally certain pragmatic trials do not have controls to ensure that the observed differences aren't due to biases in trial conduct.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs self-labeled as pragmatic and that were published until 2022. The PRECIS-2 tool was used to determine pragmatism. It includes domains such as eligibility criteria as well as recruitment flexibility as well as adherence to interventions and follow-up. They discovered that 14 trials scored highly pragmatic or pragmatic (i.e. scoring 5 or more) in at least one of these domains.
Trials that have a high pragmatism score tend to have broader eligibility criteria than traditional RCTs which have very specific criteria that aren't likely to be found in the clinical setting, and contain patients from a broad variety of hospitals. The authors suggest that these characteristics can help make pragmatic trials more effective and relevant to everyday clinical practice, however they do not guarantee that a trial using a pragmatic approach is free of bias. Moreover, the pragmatism of trials is not a definite characteristic; a pragmatic trial that does not have all the characteristics of a explanatory trial can yield reliable and relevant results.