AGU: Arabian Gulf University; CMMS: College of Medicine and Medical Sciences; HS: High Secondary School; HSS: High Secondary School Science; AGU-MCAT: Arabian Gulf University Medical College Admission Test; AGUT: AGU-MCAT Total Scores; AGUE: AGU-MCAT English Test; AGUS: AGU-MCAT Science Test; AGUI: AGU-MCAT Interview; Year1 AGPA: Year1 Average Grade Point Average; Phase1: Year 1 (Premedical); Phase2: Year 2-4 Preclinical; Phase3: Year5-6 Clinical Clerkship Phase; Bsc: Bachelor Of Medical Sciences; MD: Medical Doctor Degree

Opting to study medicine is limited in itself in many aspects because it is very competitive that only good school achievers could get access to admission. Moreover, the duration of study is too long to be a specialist in the field with an expensive carrier to opt for. The cost of studying medicine concerns the students, their supporters and the establishments that are investing in founding and running such schools being governmental or private. Therefore, it is out-most consequential to ascertain that the invested effort and money when made, would have a worthy outcome; i.e. production of safe competent doctors.

For that, and prior to acceptance of students in the medical school, there is a great need for developing a valid, accurate, substantial and solid instrument, which is not an easy task, that avails in testing the students’ competence and capabilities. Such evaluation device should be able to incorporate evaluation of both cognitive abilities and individual qualities.

Studies have demonstrated that for numerous explanations, the steady loss rate (students dropping out from advancing through their restorative studies and graduating as doctors) is high. This elevated rate does not only lead to waste of assets, as well as to mental effect and social disappointment of the pupils and their supporters. For that, the discovery of components or ascribes that contributes to students not finishing their medical course and getting their degree may undoubtedly help in diminishing such withdrawal impacts. Being able to predict medical students’ performance is vital to ensure the adequate supply of quality physicians. Never-the-less it is not a simple assignment to shoulder; however, the proper selection procedure for granting admission to students into college of medicine is an important ingredient that could aid in predicting the students’ outcome. The college admission committees usually confront the overwhelming assignment of selecting a few candidates, who are most likely to succeed in medical school, from a sizeable pool of seemingly suitable applicants [1]. Across the globe, most medical colleges use incredible exertion in selecting students from large qualified applicants. Such colleges are in a dynamic process of developing a tool ensuring the proper selection of students. Apart from MCAT, the UK Clinical Aptitude Test (UKCAT) was introduced in the UK in 2006 as an added tool for the selection of medical students. It tests mental ability in four distinct domains (Quantitative Reasoning, Verbal Reasoning, Abstract Reasoning, and Decision Analysis). Even so, its total score had little predictive value [2]. Vu et al. developed and designed a Medical Reasoning Aptitude Test (MRAT) for the assessment of the aptitude of the clinical problemsolving in medical school applicants, for the intent of applying it to predict medical school performance. They found that MRAT scores with the GPAs and MCAT scores increased the precision in identifying pupils who performed p006Forly or exceptionally well in the second year and in the clinical clerkship year [3]. Others opt for cognitive and non-cognitive assessments to make sure selecting students having the personal characteristics of importance in the practice of Medicine [4,5]. Studies have shown that not only MCAT scores, predicted performance of the majority of pupils in the first year of medical school [6], but other factors such as students’ gender and age have also implications. Gender and age were found to be related were older females perform better than older males in three grades of medical school that indicate a significant gender by age interaction as a prediction of academic performances [7,8].

Since the English language is used as a tool of instruction in most medical schools in the Middle East, it is expected that this reason also has an influence on the students’ performance. Reports from Saudi Arabia and Kuwait indicated that achievement in the premedical and medical years was positively correlated with good competency in the high school English courses [8,9].

Not only, admission test could predict students’ performances, but the students’ level in their secondary school is as much important. A study found that there is a high correlation between high school grades and later performance during the medical program [7]. Another study even suggested that it was important to have an idea of the high school that the students have studied in. It found that nearly twice as many students were forced out or withdrew in poor academic standing who attended undergraduate institutions in the lowest selectivity category [10].

Non-cognitive areas that could be tested by interview during admission are found to be related as well to students’ performances [11]. And due to that, the interview part of the admission test was found to play an important role in predicting future students’ performances at the medical school.

The main aim of this study which was implemented in the College of Medicine and Medical Sciences (CMMS) of the Arabian Gulf University (AGU) Bahrain, was to investigate the extent to which the AGU-MCAT scores supplemented by the power of the secondary school grades, predict success in the medical college or could predict early enough students’ dropout. It was also aimed at investigating the academic trend/s that has/had a major effect on students’ performance during their six years of longitudinal medicinal studies.

A retrospective cohort study of the achievement of 107 medical students after satisfying their admission requirements and were enrolled in College of Medicine & Medical Sciences of the Arabian Gulf University in Bahrain, during the academic year 2002-2003, have been studied and examined. Their academic performance was traced through their academic records starting from admission throughout Year 1 and then at Year 4 (the B.Sc. level) and Year 6 (during clerkship) and later in the final year’s examination (the MD).

For admission to CMMS, students have to meet the eligibility criteria, and then sit the admission test “AGU-MCAT” which consists, of three parts:

1- Sciences test (Multiple-choice question in physics, chemistry, mathematics and biology).

2- English test (testing students’ reading, writing and comprehending skills)

3- Interview (by two senior faculties and one senior student to check students’ attributes such as personality, self-dependency, passion and determination to study medicine, etc.).

A pre-designed formula is used for reaching the final results and hence deciding about the best candidates to be granted admission (It is the total of the grades of; high school, high school sciences, and grades obtained during the AGU-MCAT test). The MCAT procedure usually screens out 40 to 50% of the total applicants.

The CMMS is a problem-based school with a community-oriented curriculum that extends over six years divided into three phases [12];

• Phase I; is a one academic year of a credit-hour system where the average grade-point average (AGPA) is calculated at its end. Students in this phase are taught basic sciences courses such as Biology, Physics. Chemistry, Biostatistics and Epidemiology, besides English language, Islamic Studies, Psychosocial and Computer Sciences.

• Phase II; is the pre-clerkship phase that extends over the 3-year period where students are taught in a self-directed method using around one hundred community-oriented problems grouped into nine units. In addition, students are engaged in many activities such as research activity, professional skills, and community-related training programs in the primary health care facilities. At the conclusion of this phase, those students who pass the exams are granted B.Sc. in medical sciences and promoted to the clerkship phase.

• Phase III; is the clerkship phase that runs over two years where students rotate in the hospital in various major and minor clinical specialties and do their family medicine rotation in the primary health care centers. By the end of this phase, students sit the MD examination to be granted the degree.

Since all the variables of interest are quantitative, Pearson correlation coefficient was used to decide the linear relationship between the students’ performance at the end of each phase and their scores in; high school, AGU-MCAT, Year1 courses, B.Sc., Family Medicine clerkship and MD. Stepwise regression was used to find out what factors mainly predict the students’ performance in the medical study. ANOVA was used to examine if there was any relevant difference in the students’ performance in AGPA, BSC, Family medicine rotation and MD compared to their grades in the MCAT and High school general and science grades. P-value of 0.05 or less was considered as statistically significant.

Out of the 210, students applied for admission to the CMMS; 185 after initial screening were allowed to sit the AGU-MCAT test and only 107 were selected for admission. The academic achievements of these students in; HS, HSS, AGU-MCAT, end of the Year1 (AGPA), end of year 4 (B.Sc.), the clerkship rotation in Family Medicine (as an example of clinical rotations in Year 6) and their MD results, were studied, correlated and compared.

These students were followed up and found that 89 and 81 reached to the B.Sc. and MD level subsequently (Table 1). The overall attrition rate at the end of the medical studies was found to be 10.3% (11students were lost out of the total number enrolled in Year 1). It is equivalent to 12% from those who were supposed to be enrolled in Year 6).

Table 1: Highlights the failure rate during and at the end of each phase.

Pearson correlation

The Pearson correlation coefficient between student’s phases achievements and (HS, HSS, AGU-MCAT and Year 1 courses). The finding as stipulated in Table 2 shows:

Table 2: Pearson Correlation Coefficient between Students’ Achievements in Various Phases and (HS, HSS, AGU-MCAT and Year 1).

• A significant positive moderate correlation was found between HS, HSS and all components of AGU-MCAT except the Interview which is weak.

• There is a significant linear relationship between students’ performance in the B.Sc. phase and HS, AGU-MCAT components, AGPA and Year 1 courses.

• The students’ performance in Family medicine rotation is positively correlated with all the sub-scores except for the AGUS and Year 1 physics course.

• The performance in MD phase is no difference. It is positively correlated with all sub-scores except for the AGUI.

Stepwise regression analysis

To find what factors mainly predict the performance of students in each phase of the medical studies, a multiple linear regression (stepwise regression) analysis was used. The predicted factors of each phase with the corresponding value of R2, percent of variation explained by the predictors are shown in Tables 3-7.

Table 3: Multiple Linear Regression of HS, HSS with AGUT

Table 4: Multiple Linear Regression of HS, HSS, AGUS, AGUE and AGUI with Year 1 AGPA.

Table 5: Multiple Linear Regression of HS, HSS, AGUS, AGUE, AGUI, Year 1 AGPA and Year 1 Courses with B.Sc.

Table 6: Multiple Linear Regression of HS, HSS, AGUS, AGUE, AGUI, Year 1 AGPA, Year 1 Courses and B.Sc. with Family Medicine.

Table 7: Multiple Linear Regression of HS, HSS, AGUS, AGUE, AGUI, Year 1 AGPA, Year 1 Courses, B.Sc. and Family Medicine with MD.

AGU-MCAT total score: Table 3 shows the multiple linear regression analysis of HS, HSS with AGUT. From the table we notice that HS and HSS scores explained 42.4% of the variation in AGUMCAT score. The HS scores alone explained 42.3% of the variation in AGU-MCAT score while HSS scores explained 29.6% of the variation in AGU-MCAT score. Regression analysis shows that only HS grade is statistically significant as a predictor of AGU-MCAT score (p-value<0.001).

Year 1 AGPA: Multiple linear regression of HS, HSS, AGUS, AGUE and AGUI with Year 1 AGPA is shown in Table 4. The full regression model (Model 1) indicates that these factors explained 45.3% of the variation in AGPA grade (i.e., 45.3% of the variations in AGPA grade are due to these factors). Stepwise regression shows that 26.8% (compared to 45.3%) of the variation in AGPA is explained by HS (when only High School included within the model). While 37.6% (compared to 45.3%) of the variation in AGPA is explained by HS and AGUE, an added 10.8% of the variations are explained by AGUE if it is included into the model beside HS. 43.1% (compared to 45.3%) of the variation in AGPA is explained by HS, AGUE, AGUS, an extra 5.5% of the variation is explained by AGUS if it is included in the model beside AGUE and HS Science. Only 2.1% of the variation in AGPA is explained by HSS provided that HS, AGUE and AGUS are included in the model. We conclude that students’ Scores in HS, AGUE and AGUS can be used as predictors for the performance of the students at the end of Year 1. They explained about 43.1% of the variation in AGPA scores. The students’ score in AGUI does not affect the performance of students at the end of Year 1. It explained a maximum of 0.2% of the variation in AGPA taken into consideration that the AGUE, AGUS, HSS was included in the model.

B.Sc.: The multiple linear regression model relating B.Sc. to students’ scores in HS, HSS, AGU-MCAT, Year 1 courses and AGPA is shown in Table 5. The full regression model (model 1) indicates that these factors explained 72.8% of the variation in B.Sc. grade. Stepwise Regression shows that 54.9% (compared to 72.8%) of the variation in B.Sc. are explained by AGPA. While 67% (compared to 72.8%) of the variation in B.Sc. are explained by AGPA and Biostatistics (an added 12.1% of the variation is explained by Biostatistics if it is included within the model beside AGPA). However, 69.6% (compared to 72.8%) of the variation in B.Sc. are explained by AGPA, Biostatistics, Physics, (an additional 2.1% of the variation is explained by Physics if it is included within the model beside AGPA and Biostatistics. An extra of 2% of the variation is explained by Chemistry if it is included into the model beside AGPA, Biostatistics, Physics. We conclude that AGPA and Year 1 courses; Biostatistics, Physics and Chemistry are the most important subjects that can be used to predict the performance of students in B.Sc.

Family medicine: In Table 6, the multiple linear regression of HS, HSS, AGUS, AGUE, AGUI, Year 1 AGPA, Year 1 Courses and B.Sc. with Family Medicine is presented. The full regression model (model 1) indicates that these factors explained 41.5% of the variation in Family Medicine grade. Stepwise Regression shows 23.6% (compared to 41.5%) of the variation in Family Medicine is explained by B.Sc. 25.6% (compared to 41.5%) of the variation in Family Medicine is explained by B.Sc. and AGUI (an extra 2% of the variation is explained by AGUI if it is included into the model beside B.Sc.). Only 18.3% of the variation in Family Medicine is explained by HS, HSS, AGUS, AGUE and Year 1 courses. We conclude that B.Sc. and AGUI are significant predictors of the performance of students in Family Medicine Phase, they explained 25.6% of the variation in Family Medicine grade.

MD: The multiple linear regression model relating MD to students’ scores in HS, HSS, AGU-MCAT, Year 1 courses, AGPA, B.Sc. and Family Medicine is shown in Table 7. The full regression model (model 1) indicates that these factors explained 83.5% of the variation in the MD grades. Stepwise regression shows that 77.6% (versus to 83.5%) of the variation in MD is explained by B.Sc. While 80.8% (versus to 83.4%) of the variation in MD is explained by B.Sc. and Family Medicine. Family Medicine alone explained 33.2% of the variation in MD. Only 2.7% of the variation in MD is explained by HS, HSS, AGU-MCAT, AGPA any Year 1 courses. We conclude that B.Sc. and Family Medicine are significant predictors of the final grade of students in MD phase.

A summary of the factors predicting the performance of students in each phase of the medical studies is shown in Table 8.

Table 8: Stepwise Regression of Students’ Achievement at each phase.

The overall student attrition rate in the AGU by the end of the medical studies was found to be 10.3% (11 students were dropped out of the total number enrolled in year 1). It is equivalent to 12% from those who were enrolled in the final year of the clerkship phase (Y6). This dropout rate is very high when compared to other places. For example; in an Irish study, it was found to be 5.7% (45/779) [13], and In the UK, the overall average first-year dropout rate over the period 1980-92 was calculated to be 3.8% [14]. Even our figures were shown to be higher than those figures reported from developing countries such as Nigeria where a study in 2010 reported that 7.8% of the students admitted into preclinical class withdrew from their study of which 53.8% believed having the poor academic ability [15]. It was found that the largest attrition rate usually happens during the first year of medicine, which coincides with other reports. A UK study reported that the highest rates of attrition (46/1188, 4%) occurred during the initial two years (largely in the preclinical studies) [16]. Other investigators documented that the probability that a student would drop out of medical school during their first year was influenced significantly by both the subjects studied at A-level, and by the scores achieved. For example, achieving one grade higher in biology, chemistry or physics reduced the dropout probability by 0.38% points, equivalent to a fall of 10%. It was also found that males were about 8% more likely to drop out than females [14]. Students who persist in the course and perpetuate their medical vocation may be remotely more self-regulated, self-efficacious and higher achievers than their peers who drop out [17].

Conventionally medical school dropout has negative consequences on the society, patients, and the vocation. High attrition is very costly for any institution [18] and for the AGU it would not only play a major role in decrementing the engenderment rate of fresh graduate doctors who are highly needed in this component of the world but also has substantial resource implications for the Faculty. It deprives the university from a large income (around a BD 429000 equivalent to US$ 1,135,000) that is vital for its growth, expansion and development. The high attrition rate can also affect the academic reputation of a medical school and staff morals. More important, are the personal consequences of dropout for the student [13].

Although, Arulampalam et al. believed that the probability of dropping out depends largely on the personal characteristics of the student, including academic preparedness, they do think that one of the main causes of dropout students is fewer effective admission policies for the selection of suitable student [19]. They in another study while emphasizing the importance of students’ selection stated “if traditional entry requirements or standards are relaxed, then this is likely to have detrimental effects on medical schools’ retention rates unless accompanied by appropriate measures such as focused student support” [14]. Urlings et al. in 2011 reported that properly selected group of students received considerably higher mean grades on their first five clerkships, which could not be attributed to factors other than the selection procedure, and the actual dropout rate proved to be twice as low in the systematically selected group of students [20]. In the Denmark, a study reported that only the admission strategy, the type of qualifying examination and the priority given to the program on the national application forms contributed substantially to the dropout [19]. Another study from the Denmark found that selection of medical students from proper admission testing had a lower relative risk of dropping out of medical school within two years of admission [10,21].

Since, mental-health problems predominate in late course attrition and may have been undisclosed for some time [16], structured and reliable interview procedure is a very important component of any MCAT test. The MCAT verbal reasoning was found to be statistically significant in predicting first-year Podiatric medical students’ scores [10]. For that, Callahan et al. study in 2010 supported the short- and long-term predictive validity of the MCAT [22].

An effort was made in this study to find out whether students’ academic background such as their overall high school’s achievement or their science grades or English language proficiency has any effect on the AGU-MCAT. It was found that their performances in AGU-MCAT were positively correlated with their HS scores (r=0.380, p-value<0.001) and their HSS scores (r=0.280, p-value=0.003).

The performances of students in all AGU-MCAT components except Interview were positively correlated with the students’ HSS. The regression analysis has indicated that the students’ HS grade can be used to predict the students’ grade in AGUS, although the prediction power is weak (R²=14.6%).

The students’ performance at the end of Year 1 was found to be dependent on multiple factors but the HS grades, AGUS and the AGUE were the most important (R²=43.1%). Julian ER, in 2005 reported similar finding by stating that the “MCAT performs well as an indicator of academic preparation for medical school” but they think that it is independent of the school-specific GPAs [23].

Students’ scores in HS, AGUS and the AGUE can be used as predictors for the performance of the students at the end of Year1 (It explains about 43.1% of the variation in AGPA scores). Smith and Geletta in 2010 while studying the factors influencing medical student attrition found that the pre-admission variables, such as undergraduate grade-point average, ethnic origin and biological science scores, are statistically significant in predicting first-year of Podiatric medical school grade-point average [10]. Similarly McManus et al. in 2013 found that the General Certificate of Secondary Education results predicted undergraduate and post-graduate medical education outcomes [24]. However, Frischenschlager et al. warns against depending on the success in secondary school, as the only criterion for university admission despite its importance [25]. Other authors thought that psychosocial measures should be considered as significant and unique predictors of performance in medical school [26]. This statement is supported by a report from an Australian medical college that changed their selection processes as a result of their findings, and stated, “students’ academic performances during the medical program were explained very well by the selection criteria used, which were grade-point average (GPA) (most strongly) followed by the interview score” [27].

During the preclinical period students in the AGU are given large amounts of scientific material that they should; analyze, understand and at time memorize. Therefore, the wider their background of knowledge the better they are in this process. It was found that at the B.Sc. level (end of preclinical Phase), the students’ performance is dependent on many factors such as their achieved HS scores, AGU-MCAT and AGPA. Even so, the AGPA and some of Year 1 science courses such as Biostatistics, Physics and Chemistry were the most important subjects that could predict the performance of students at the B.Sc. A finding which might prove that science subjects are predictors of students’ performance in the preclinical year. However, Hall & Stocks in their study did not find any relationship between quantity of science-based undergraduate premedical education, and the performances of medical students in their preclinical years of medical school [28]. While Höschl and Kozený study stated that variables chosen from the assessment domains of high school performance, written entrance examination, admission interview, and personality traits may be significant predictors of academic success during the first three years of medical study [29].

No effect was found of HSS grades on the students’ performance at the B.Sc. Lipton et al. in 1988 also demonstrated similar results when stated that the school science subjects were of moderate value for the prediction of preclinical achievement [30].

The AGUI, since it tests students’ non cognitive attributes is found to be the least factor to affect the students’ achievements at B.Sc. Similar findings were reported elsewhere [31].

Although performance in the preclinical years is predicted by the grade-point average and MCAT scores, no such correlation exists for achievements in the clinical years, for postgraduate training, or as physicians [31]. During the clerkship phase, students are required to do a lot of interactions with patients, and for that they need to have good communication skills. We found that the AGUI and B.Sc. are significant predictors of the students’ performance in Family Medicine clerkship. Both factors explained 25.6% of the variation in Family Medicine rotation‘s grades. A positive trend for interview ratings with clinical performance was reported in the literature [32]. Students who perform well in the AGU-MCAT interview probably will develop better patient communication skills later. The school English was reported to be the most important predictor of performance in the clinical years, which is perhaps an indication of the value of communication skills [33]. However, we found that such relationship was very weak neither a relation was found with the AGUS. But, Shen and Comrey [33] warns against depending on the personality characteristics while deciding on accepting students by stating, “it is not realistic to use one or two personality traits to predict personal suitability on all medical performance measures. Various personality characteristics are incorporated in different types of medical performances” [31].

The MD is the final stage of the challenges for the medical students before they pass and be doctors. It is an overall exam that test; students’ scientific knowledge as well as their clinical skills. It was found that performance at this stage depended on students; scores in Family Medicine, B.Sc., Year 1 courses, AGPA, HS, HSS, AGUS and AGUE. However, the scores of the B.Sc. and Family Medicine were found to be the most significant predictors of the students’ final grades in the MD; they explained 80.8% of the variation in MD grades. Both these combined elements of professionalism, require scientific and clinical competencies.

In conclusion, medicine is a precious and an expensive career to opt for. Therefore, medical students’ selection procedures should be felicitous, systematic, scientific and most importantly independent. There is a need to identify the best applicants for medicine and to ensure that selection is fair and ethical. Utilizing a more comprehensive, more reliable and more authentic students’ selection method substantially decreases the attrition rate and increases the prediction of students’ success in the medical college. Many factors can be used as predictive tools for students’ performances during their medical studies. Since the AGU-MCAT scores, correlates with standard measures of academic success in the medical school, it is important that the admission procedures and its committee be highly independent, accurate and precise in its screening tools to select well-qualified applicants who not only do not drop out but excel in their studies.

Overall, the competitiveness of the college admission procedures and the courses taken was found to contribute significantly to the prediction of all measures of medical school performance.