Journal of Clinical Trials
Open Access

A South East Asia Multi-Country Survey Assessing Awareness and Preparedness of the Clinical Investigation Staff on Risk Based Monitoring (RBM) Approach

Karthikeyan Kumar^1* and Manoj P Jadhav^{2 *}Correspondence: Karthikeyan Kumar, Clinical Trial Manager, Clinical Operations, Global CRO, Chennai, India, Email:

Author info »

Introduction

RBM is a relatively novel concept or adaptive methodology being discussed among clinical research community extensively. Clinical research industry had been historically used of 100% source data verification (SDV) during onsite monitoring visits done by the clinical study staff e.g. monitors, however it is costly (which includes personnel time, travel and other expenses); also delays in data checks and review, data interpretation, time investment on interim analysis, risk assessments and many other factors including assessment of the therapeutic benefits of the drugs. The emphasis on to refocusing efforts to overcome the current challenges on risk assessments and onsite monitoring, among various options there are several optimized approaches are encouraged and recommended by USFDA, EMA, MHRA as well as in E6 ICH (R2) called Risk Based Monitoring (RBM). As an initiative, the seminal work on RBM was conducted by the Clinical Trials Transformation Initiative (CTTI) 2009. This survey consisted of 55 questions, which involved only 65 responders from industry, which concluded that only 33% or less have adopted the centralized monitoring approach as against the traditional source data verification monitoring [1-22]. In 2013, MCC planned and conducted a survey,with 45 responders reported that upwards of 50% of industry stakeholders were using RBM approaches in one or other ways on data analytics. Moreover, it also was found that 10 to 30% of respondents planning to implement RBM within 12 months. Since then in last 3-4 years, the RBM adoption has steeply increased in part to growing use of electronic solutions and statistical assessments especially to improve the subjects ’ safety, quality of data, integrity and efficiency among sponsors oversight of clinical trials.

While looking up drafted guidance on RBM approach from major regulatory agencies of USFDA, EMA and MHRA, an idea gained traction to plan and conduct a survey in South East Asia region of India, Malaysia and Singapore on assessing the RBM awareness and how are we/clinical study site staff is prepared to adopt this guidance/methodology. The critical objective of this study was to ensure the generation of the high-quality data using RBM approach and its ultimate impact on roles on subject safety, data quality and trial cost implication.

In view of literatures, the awareness of RBM among developed countries is higher than emerging countries [7] however there are no large data available on RBM approach awareness and preparedness among clinical industry staffs in South East Asian countries of Malaysia, Singapore and India. Thus, it encouraged conducting the survey in Malaysia, Singapore and India to understand the current scenario of RBM approach and its awareness and preparedness.

Implementing and adapting RBM has many advantages. The effective risk management should have a detailed structured approach for risk identification, evaluation, analysis and control. Regulators demands that the risk management approaches should be sustainable, easily adaptable, reproducible, repeatable, and controllable to achieving quality outcomes. RBM approaches can also help to facilitate in many other aspects of clinical trials like site identification, selection, qualification, protocol design and development, and subject enrollments as well [18].

Materials and Methods

The methodology of descriptive survey research design was chosen with a primary objective of assessing the RBM awareness and preparedness. Also, emphasis was given with subset of questions relevant to trial oversight, quality, and safety and cost criterion. The survey was planned to have mixed mode (paper-toweb) self-completion questions involving online (desktop, laptop, tablet or smart phone) and pen- paper method for those professionals who either do not have internet access at that time or prefer to complete the survey in paper mode. The survey consisted of 18 closed- ended and 1 open ended question in English language related to safety, quality, cost, adoption and others. The survey questionnaire validated relevant to the objective and the variables planned for the study [9]. The participants, in the survey were clinical research staffs from Malaysia, Singapore and India and selected randomly have had a provision to specify opinions/suggestions with their experience relevant to RBM implementation and challenges. To improve the participants’ interest, questionnaire design and formatting was optimized and set out to circulate to each region simultaneously. Questionnaire responses obtained upon participants’ voluntariness and never been influenced however considering large target volume the participants were reminded.

Data integrity is a fundamental component of the study data and information security [20] and refers to accuracy and consistency of data collected for the study. Having said, the data collected for the survey maintained securely and compiled in excel timely. Data validation of participants ensured by their email address or by the paper copies collected personally. Out of 502 responses, 469 responses were electronic copies and 33 responses were paper copies. Data confidentiality is a property of data and considering the importance, de-identification achieved to protect the respondents’ personal data and labelled the responses as Response 1, 2, 3 and so on. Data validation/ Edit checks of de-identified database against labelled questionnaires were reviewed by third party who has had no conflict of interest in the survey for analysis.Variable significance was determined by ‘Chi Square’ or ‘Fisher’s exact test’. The optimum method of statistical analysis among the parameters was used for the survey hypothesis.

Results

The overall survey response rate was 9.54%, survey was sent out to 5258 clinical trial professionals and we received 502 completed responses. The characteristics are described in Table 1.

Table 1: Survey response rate across regions of Malaysia, Singapore and India.

Percentage of survey response for Malaysia (32.5%) was higher than Singapore (4.9%) and India (6.5%). Number of responses received for Singapore is not significant to compare withMalaysia andIndia. To avoid analysis bias, the survey data of Malaysia and Singapore (considered as Group 1) were combined to compare with India alone (considered as Group 2). In group 1 the received responses were n=220 and in group 2 the received responses were n=282. In Group 1, 32.2% of Investigators, 39% of study coordinators/nurses, 19% of CRO personnel and 9.5% of other clinical staffs including sponsors, QA Professionals, pharmacists, clinical auditor and others. In Group 2, 25.8% of Investigators, 23.7% of study coordinators/nurses, 32.6% of CRO personnel and 17.7% of other clinical staffs including sponsors, QA professionals, independent trial consultants, subject matter experts, site management organization (SMO) staffs, pharmacists, clinical data associates, resource planner, andothers.

Demographic data analysis

In the survey, 260 (51.79%) males and 242 (48.21%) females participated. The awareness of RBM among males 208 (80%) was slightly higher than females 181 (74.79%). We also found that awareness of RBM concept and its methodology was proportionate with number of clinical trials experience as such awareness rate for more than 10 trials was 40.24%, trials between 3 to 10 was 30.08% and less than 3 trials were 7.17%. In article by Caroline [6] stated that mostly the RBM model are being used in multi-centric/larger population trials rather in smaller population studies still prefers paper CRF considering the cost . To prove this, the survey results showed majorly RBM used in global trials (50.9%) rather than local trials (4.9%) but number of subjects involved, type of phase studies and relevant details were not in the survey scope. As a trend, global/multi centric studies are conducted mostly in phase II/phase III/Phase IV studies which requires larger population [23]. To interpret with survey response, global trials may consider as multi-centric studies to match the results with article by Caroline Hurely [6]. Survey results showed that 72.2% of trial adapted RBM, 27.5% of trial non-adapted RBM and 0.2% not aware of RBM involvement. Data of non- adapting RBM closely matched with Caroline Hurely [6] article. These data proven that at least 50% of trials were being used either one the RBM tools, the reason of lack of non-RBM trials could be lack of RBM knowledge, training in clinical units, cost factor on IT demands, lack of trained staffs and others [6].

RBM awareness data analysis

Overall RBM awareness rate was 77.49%, group 2 awareness (85.46%) significantly increased compared with group 1 (67.27%) which could be the chances of trials conducted in India were almost 200% more per trials registered in https:// clinicaltrials.gov/ when compared with group 1 (Malaysia+ Singapore). Awareness rate further scrutinized among investigator, coordinator/Research nurse, CRO personnel and other clinical staffs in percentages depicted in Figure 1. It revealed that awareness among CRO and Clinical staffs were much higher than investigators and study coordinators.

Figure 1: Percentage of RBM Awareness.

More efficient monitoring process can produce number of benefits including data integrity, trial subject safety, quality data, faster and better data analysis, lowering budgets and increased productivity. Additionally, it will help that data anomalies and frauds like non-random data generation, distribution and fabrication may be more easily detected by centralized monitoring techniques rather than traditional monitoring [33]. Above all also indicates that more effective oversight reinforces trial subject participant and protection.

Data analysis on preferred monitoring type

To our knowledge, this could be the first approach studied to understand the preferred monitoring type among these regions. Though regulatory agencies drafted RBM guidelines and CT industry emphasis on mixed model of traditional and centralized monitoring over the decades, the evidence of acceptance of mixed model proven in this survey and depicted in Figure 2. Preferred monitoring type was further classified individually among Investigators, study coordinators, CRO Personnel and other Clinical Staffs and the preference rate of hybrid monitoring was 75.69%, 63.4%, 89.55% and 87.32% respectively. The next preferred type of monitoring recommended by study coordinators was on-site monitoring and the preference rate was 32.03%.

Figure 2: Percentage of preferred monitoring type.

Data analysis of trial oversight, safety, quality, efficiency on RBM

Most respondents agrees that RBM helps to oversight the trial data timely, to mitigate the subject’s safety risks on real time basis, to improve the data quality via data integration and overall improves the trial efficacious and there was no significant value among Group 1 and Group 2 and the detailed classification is elaborated in Table 2 below.

Table 2: Contribution of RBM in trial oversight, subject safety, quality and efficiency.

Data analysis on RBM cost factor

Many articles and white papers conveyed that RBM reduces the trial cost by 15-30% [29-32]. Given this perception, survey results revealed that respondents agrees that RBM reduces monitoring cost by 55.78%, disagrees by 7.97%, neutral by 18.92% and not sure by 17.33%. Similar trends were seen for questions related to overall trial cost using RBM, respondents agrees that RBM reduces overall trial cost by 49%, disagrees by 8.57%, neutral by 23.51% and not sure by 18.92%. It proves widely accepted perception among clinical stakeholders not differentiated and an optimum utility of RBM methodologies proportionate with trial costs. There was significant difference (p<0.001) between the two groups based on monitoring resources and overall trial costs. The acceptance of reduces cost widely agreed by group 2 rather than group 1. At group 1, the percentage of RBM reduces resource cost to monitoring are 42.73% and at group 2 are 65.96%, likewise the percentage of RBM reduces overall costs at group 1 are 36.82% and at group 2 are 58.51%. Article by Caroline et al. [6] conveyed that RBM reduces the monitoring costs by 16% in Ireland, however in SEA regions the understanding is drastically increased up to 55.78%.

Data analysis of adoptability of RBM methodologies

Understanding of RBM methodologies mostly liaises among CRO personnel (78.36%) and other Clinical Staffs including sponsors, etc. (80.28%). To accept the fact that CRO personnel (53.73%) and other Clinical Staffs including sponsor, etc. (61.97%) were anticipating challenges in RBM since it is considered that developing, planning, utilizing the technologies, and managing RBM mainly lies with them. The understanding of RBM methodology among investigators and study coordinators/nurses was 52.76% and 66.67% respectively and anticipating challenges in RBM rate was comparatively less than CRO personnel and other Clinical Staffs which was 45.14% and 43.14%. This survey also revealed that overall 69.52% of respondents ready to adopt RBM methodologies, 67.73% understands how RBM methodologies supports in clinical trials, 70.92% agrees that RBM approaches may evolve drastically in future and 49.2% anticipates challenges in adapting and implementing RBM strategies.

Data analysis of Non-RBM vs. RBM trial groups

Curiously, the participants were also surveyed about the studies involved/being involved RBM and if so, type of the trial it is. Survey results showed that 362 respondents (72.2%) of trial adapted RBM, 138 respondents (27.5%) of trial non-adapted RBM and 1 respondent (0.2%) not aware of RBM involvement. Data of non-adapting RBM closely matched with Caroline Hurely [6] article. Mostly, RBM methodologies were involved in global trials of around 70.44%. Interestingly, RBM methodologies were also used in local trials of around 6.91% and of course the data of RBM methodologies involved in both trials showed as 22.65%. These data proven that at least 50% of trials were being used either one the RBM tools and may be the reason of lack of non-RBM trials could be lack of RBM knowledge, training in clinical units, cost factor on IT demands, lack of trained staffs and others [6].

It is clear evidence that the approach of RBM is steadily increasing in global trials i.e, which may involve larger group of population. The parameters of preferred type of monitoring, understanding of RBM relates for timely oversight, safety risks, overall trial quality, trial efficiency, and cost factor of monitoring resources and overall trial cost as well the impact of RBM understanding and its significance data were also analyzed separately between non-RBM and RBM experienced groups.

Analysis revealed that 80.94% of RBM experienced groups preferred hybrid monitoring and 68.84% of non-RBM experienced groups preferred hybrid monitoring, this interpretation determines that RBM experience users willing to adopt RBM more rather than non-RBM experience respondents. Other analyzed parameters are described in Figure 3.

Figure 3: Percentage of accepted parameters in RBM vs. Non-RBM trials.

The survey questions of RBM awareness, RBM tools experience and trials involved/involves RBM are interrelated each other and overall percentage rate is 77.49%, 81.87% and 72.11% respectively. Below 10% of deviation was noted in the related data which might be widely acceptable and the interpretation conveyed that still around 4% of stakeholders not truly understand the RBM tools/methodologies however aware about RBM; the same factor may also be applicable for the interrelations between RBM tools experience and trials involved/involves RBM. It is fact that only small group of stakeholders in each organization are being involved in RBM development and methodologies as well in many cases there might be a lack of internal knowledge sharing and between field monitors and remote monitors, it is utmost importance that industry encourages the RBM knowledge sharing throughout all team members.

Importance of Technologies in RBM

The use of technology is important in implementing RBM strategy. Pharma/CROs use various electronic systems of adopting RBM programs to reduce onsite monitoring and encourage remote monitoring with support of the central data analytics. Advantage of those systems is to set up an automatic flag/alerts to review the critical data or risk indicators.

In clinical research industry, regulatory authorities and all industry stakeholders needs everything to be budgeted, organized, managed, tracked, reported and user friendly. It can be implemented using RBM methodologies with help of the software tools to make the day to day activities easier. As per survey responses, the experience of RBM tools and technologies among the focused regions of Malaysia+Singapore and India were 83% and 85% respectively and survey outcome shown in Figure 4.

Figure 4: Experience of RBM tools and technologies.

The percentage of usage of RBM technologies showed concrete evidence that the importance of technology required for in RBM model. The USFDA and EMA drafted guideline proposes risk based monitoring (RBM) to build the monitoring components and trial management through quality by design and encourages sponsors to employ electronic systems to improve clinical oversight, prevent risks to data quality, safeguard critical processes to ensure trial subject safety and data integrity. Adapting RBM into practice requires enabling technologies that can provide centralized control and visibility, flexibility and immediate response. Survey revealed that technology already plays a central role in the conduct of clinical trials, ranging from centralized monitoring like Electronic Data Capture (EDC), Remote Data Capture (RDC), e-reports, Interactive Voice Response System (IVRS), Interactive Web Response System (IWRS), etc. to wider clinical trials management systems (CTMS).

Recently technologies are moving to e-diaries, e-consent, esignature and e-forms features on mobile devices. To have realtime and honesty data, the uses of technology is much essential. The benefit of e-forms on mobile device includes easy accessibility, real-time data capture, review critical and subject safety data, prevents loss of data, easy storage, durable, and goes paperless. By adapting electronic technologies in RBM mostly goes paperless considered the RBM is eco-friendly and environmental approach too. Benefits of RBM and its technology is well described but practically still there is a gap between the awareness on RBM and how the clinical investigation staffs are ready for it is questionable since only small group of staffs engaged in developing RBM methodologies in Pharma or CRO companies, and thus survey hypothesis evolved to understand more among clinical staffs.

RBM hierarchical survey data

RBM is a vast topic which includes several tools and methodologies such as CTMS, RDC, EDC, Analytical tools, Programmatic Data, IWRS, IVRS, e-reports, e-forms, e-diaries, Risk Assessments, Remote monitoring and others. Selection and prompt application of RBM methodologies is crucial for successful of RBM implementation. Below Figure 5 also represents the sequential of accepted parameters as per survey. First and foremost accepted parameter was timely oversight of the trial; efficiency of trial conduct, reduces safety risks and improving trial quality were the further accepted parameters. Even though, many articles and white papers conveyed that RBM reduces the trial cost [28] significantly however the trial cost holds the last accepted parameters in the survey. Furthermore studies are required to analysis and prove the fact.

Figure 5: RBM hierarchical survey data.

Discussion

As far as we know, this is the first study among SEA regions of Malaysia, Singapore and India to investigate the awareness and preparedness of RBM among Clinical Investigation Staffs. USFDA, EMA and MHRA already opened up an idea of new approach of remote monitoring as well latest ICH-GCP R2 emphasis the quality management via risk identification, risk evaluation, risk control, risk communication, risk review and risk reporting [4]; so that the RBM is likely to raise in future. The survey had a question on understanding of RBM approaches will evolve dramatically in future, overall response was fortunate and most of the clinical investigation staffs agrees that the RBM approaches will evolve significantly in future and the percentage rate of accepting this among Investigators, Coordinator/Research Nurse, CRO Personnel and Other Clinical Staffs includes sponsor, etc. is 65.28%, 64.05%, 79.1% and 81.69% respectively. The primary objective for this study was achieved to conclude that overall RBM awareness (77.49%) was well versed among these regions; even overall factor to adopt the RBM was also on significant rate (69.52%). Some of the interesting comments have been provided in the survey as below, which are necessary to focus on developing the RBM model for better perception of clinical trials.

“ I wouldn't think there would be major challenges in implementing RBM, especially if the Clinical Operations Managers/Line managers and Project Managers themselves embrace RBM and understand its advantages.”

“Identifying the correct risks, striking a balance between onsite and remote monitoring.”

"One of the main challenges would be organization of resources including developing and fine- tuning of monitoring platforms, and allocation of adequate number of staff”

“Though centralized monitoring is very much needed in large trials involving many centres, the cost of it would be great and on-site monitoring is necessary to ensure things are as reported.”

“There will be some challenges to adopt RBM at initial stage, but gradually RBM will give effective results in terms of data quality and timelines. RBM is evolving as a best monitoring methodology in Clinical Trials as it is effective in terms of Data quality and study timelines majorly. There are some challenges to start and implement the fundamentals and tools of RBM at initial stage but results of studies involving RBM are tremendous when it comes to data quality!”

“Challenged are proper training of investigator site staff and monitors and clear separation of duties between field monitor and RBM monitor. Also validated analytics support required for RBM.”

From the above comments, we could clearly see the RBM understanding among these regions is much encouraged, the challenges highlighted in RBM are lack of training among site staffs as well with monitors, clear understanding of RBM, potential identification of risk factors in RBM, allocating resources on developing and implementing the RBM technology, coordination between remote monitors and field monitors. The barriers of RBM listed in article by Caroline et al. [6] are coinciding with the challenges provided by the participants in these regions. Even though there have been difference in region, culture, work style, understanding, etc. but across worldwide the challenges may remain the same. The result of this survey reveals that through RBM by reducing costs without comprising the integrity or accuracy the efficiency of the trial can be increased. Now-a-days, more and more clinical trials in developed regions and emerging regions are conducting studies in accordance with the principles of adaptive design [7] and it supports to build on the idea of evaluating data as a trial progresses to decide whether to modify the study aspects in real time. Considering the participants’ rate of RBM acceptability, it is the time now to move away from the traditional approach of frequent on-site visits and accomplish required SDV toward a combination of activities, including centralized monitoring and data collection [13]. The objective of traditional monitoring approaches is to ensure trial subjects’ safety and quality data, which can be further enhanced without comprise in data integrity by adopting RBM strategy.

New monitoring approaches like triggered, tailored and targeted monitoring are enhanced and well supported by RBM methodologies. New methodologies are constantly changing to improve the data integrity like Phase 0 (exploratory studies) and Phase V (translational research studies) [15] and need of focus confined and systematic approach of RBM encourages it. Here are the some of the RBM tools are being used as provided by survey respondents are Merge eCOS, J-review, Thor, Spotfire, CITI program, Infosario, I-portal, Trial Insight, eSubjectDiaria. New innovation biotherapeutics of Biosimilars and Biobetters are also growing molecular research in trends and considering the huge population size in those research, the robust model to oversight the subjects ’ safety, data quality and integrity is essential which can be potentially regulated by RBM methodologies [8].

Conclusion

As a trend, the number of clinical trials in these regions is increasing steeply, and considering uniqueness of each trial and complex in terms of structure and data handling, it is crucial to build quality and subjects ’ safety into the trial process from planning to post-marketing follow-up. Even though, traditional monitoring with 100% SDV is no longer the recommended method for monitoring all type of trials and many sponsors, organizations still hesitate to transition to RBM from traditional method. USFDA, EMA, MHRA, ICH-GCP (R2) recognizes that an approach focused on the specific risks for clinical trial is more likely to ensure the safety of the patient rather than routine site visits and 100% SDV. It is very important to understand that RBM is an adoptive methodology and not a standard, with the common principles shared across regulatory authorities and industry. This survey explores alternative monitoring strategies, per se future of clinical trial monitoring and strongly recommends implementing hybrid monitoring (Remote Monitoring+On-site Monitoring) in an effort to clarify the confusion about RBM awareness in these regions. The results of this survey, the clinical stakeholders much aware and mostly agreed to adopt the RBM concept to prioritize the factors of timely oversight, safety, quality, efficiency as well for cost factor. Now, industry has to look how best to operationalize RBM in clinical studies. RBM could be a one door solution for data integrity, safety and wellbeing of trial subjects, and quality of the trial, RBM with a promising outlook can be a successful alternative monitoring approach in clinical trials. At any rate, the present study has opened up a new vista about RBM for further detailed studies in wider aspects.

Acknowledgements

It is indeed moment of great pleasure and pride to acknowledge with deep sense of gratitude and respect to my guide Dr. Manoj Jadhav for his valuable guidance, suggestions, help and constant encouragement throughout the course for the completion of my PhD dissertation work. I express my sincere thanks to Dr. Parthasarathy Rengarajan for his review to validate the edit checks of the compiled data for analysis. I also express my great thanks to Mr Kumar Naidu for his contribution in statistical analysis support and in addition, acknowledgement goes to University, teachers, mentors, student coordinators, and administrators for their support on this thesis work. Last but not least, my warmest grateful to my family members for their love and support throughout the project work.

Author´s Contributions

All authors contributed to the development of study planning, design, data review and approval of final manuscript. The author(s) also declare(s) that there is no conflict of interest.

Ethics and Consent to Participate

All study participants voluntarily participated and have not influenced in anyways.

References

1. EMA. Reflection paper on risk based quality management in clinical trials. 2013.
2. International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonized Tripartite- Guideline for Good Clinical Practice
3. International Council for Harmonization of Technical Requirements for Pharmaceuticals for Human Use (ICH). Integrated Addendum to ICH E6(R1): Guideline for Good Clinical Practice E6(R2);2016.
4. MRC/DH/MHRA Joint Project. Risk-adapted approaches to the management of clinical trials of Investigational Medicinal Products. 2011.
5. Caroline H, Carol S, Mike C, Patricia K, Emmy Ra, Joseph E, et al. Perceived barriers and facilitators to Risk Based Monitoring in Academic-Led Clinical Trials: A Mixed Methods Study Trials. 2017.
6. Prajna P, Jeroze D, Manoj J. Risk Based Monitoring (RBM): A global study focusing on perception and merits among clinical investigational sites. Contemporary Clinical Trials Communications. 2016; 155-160.
7. Verma S, Jain P, Chauhan P. Biobetters: The better biologics and their regulatory overview. International Journal of Drug Regulatory Affairs. 2016; 4(1), 13-20
8. Brosteanu OHP, Ihrig K, Ohmann C, Paulus U, Pfistner B, Schwarz G, et al. Risk analysis and risk adapted on-site monitoring in noncommercial clinical trials. Clin Trials. 2009;6(6): 585
9. Shukla BK, Khan MS, Nayak V. Barriers, adoption, technology, impact and benefits of risk based monitoring. Int J Clin Trials. 2016;3(1): 9
10. Hurley C, Shiely F, Power J, Clarke M, Eustace JA, Flanagan E, et al. Risk based monitoring (RBM) tools for clinical trials: A systematic review. Contemp Clin Trials. 2016;51: 15
11. Bakobaki JM, Rauchenberger M, Joffe N, McCormack S, Stenning S, Meredith S. The potential for central monitoring techniques to replace on-site monitoring: findings from an international multi-centre clinical trial. ClinTrials. 2012;9(2): 257
12. Mahan VL. Clinical Trial Phases. International Journal of Clinical Medicine. 2014; 1374-1383.
13. Clinical Trials Transformation Initiative. Quality Objectives of Monitoring: Effective and efficient monitoring as a component of quality assurance in the conduct of clinical trials.2019.
14. Clinical Trials Transformation Initiative. Results and Recommendations: Effective and efficient monitoring as a component of quality assurance in the conduct of clinical trials. 2019.
15. Mitchel J, Kim Y, Choi J. Evaluation of data entry errors and data changes to an electronic data capture clinical trial database. Drug Information Journal. 2011;45(4): 421-430.
16. Dillon C, Zhao W. A comparison of the effectiveness of on-site and central monitoring activities across six phase III multi-center clinical trials. Paper presented at: SCT Conference; May 20, 2014; Philadelphia, PA.
17. Catherine G, , Manizhe P, Sybil T, Scott P, Jeffrey T, Schouten MD. Ensuring Participant Safety and Trial Integrity with Clinical Trials Oversight. J Acquir Immune DeficSyndr. 2014; 65(0 1): S40
18. Oracle Health Sciences. Beyond SDV: Enabling Holistic, Strategic Risk-Based Monitoring, Oracle White Paper. March 2016.
19. Applied Clinical Trials, The current status of Risk-Based Monitoring by Linda B. Sullivan. 2015.
20. Applied Clinical Trials, The core of RBM in centralized monitoring by Martin Giblin.
21. Randall R. Stoltz. Risk-Based Monitoring: Implications of the US FDA Guidance for Pharmaceutical Physicians. Pharm Med. 2013; 27: 279
22. Trans Celerate Biopharma Inc. Risk-based Monitoring Methodology Position Paper
23. Vantage Biotrials, Inc. Quality by Design and Risk-Based Monitoring, white paper. 2015.
24. Risk-Based Monitoring: Understanding The Future Of Clinical Trial Monitoring By Lynn King, TKL Research Guest Column | July 29, 2015 Clinical Leader. 2015.
25. Ten proven ways to Reduce the Cost of Clinical Trials, eCOSeGuide. 2019.
26. Cutting Edge Information: Clinical Teams Should Re-Think Risk-Based Monitoring Costs to Improve Their Bottom Line by Sarah Ray.
27. Applied Clinical Trials: RbM Guidance Document. In: Moe A, Beat W , Johanna S, Peter S (eds) Ten Burning Questions about Risk-Based Study Management. 2015.
28. Dennis Constantinou Industry Poised for Significant Movement Towards RBM Adoption: Announces Results of Risk-Based Monitoring Survey(11 August edition)
29. Clinical Trials Arena Defining quality by design (QbD), risk-based monitoring (RBM) and centralized monitoring (CM) (06 Oct edition). 2015.
30. Usher R. PhRMA Bio Research Monitoring Committee Perspective on Acceptable Approaches for Clinical Trial Monitoring. Drug Inf J. 2010. 44: 477-483
31. Buyse. The Role of Biostatistics in the Prevention, Detection and Treatment of Fraud in Clinical Trials. Statistics in Medicine. 1999;18: 3435-3451.