ACC: Acceptable; ALP: Alkaline Phosphates; BUN: Blood Urea Nitrogen; CLIA: Clinical Laboratory Improvement Amendment; EPHI: Ethiopian Public Health Institute; EQA: External Quality Assessment; EQAS: External Quality Assessment Scheme; FGD: Focused Group Discussion; HDL: High Density Lipoprotein; Hgb: Haemoglobin; ISO: International Organization for Standardization; KII: Key Informant In; MOH: Ministry of Health; NEQAS: National External Quality Assessment Scheme; OSHA: Occupational Safety and Health Administration; PT: Proficiency Testing: PI: Principal Investigator; QMS: Quality Management System; SLIPTA: Stepwise Laboratory Quality Improvement Towards Accreditation; SGOT/AST: Serum Glutamate Oxalactate Transaminase/Aspartate Aminotransferase; SGPT/ALT: Serum Glutamate Pyruvate Transaminase/Alanine Aminotransferase; TSH: Thyroid Stimulating Hormone; UNACC: Unacceptable; VB: Variability of Bias; WHO: World Health Organization

Proficiency Test [PT] is a program in which objectively one or more samples are periodically sent to members of a group of laboratories for analysis and each laboratory results are compared with those of other laboratories in the group or with pre-assigned value and the feedback reported to the participating laboratories [1].

Before 60 years ago, laboratories in Philadelphia compared their result for Hgb testing, this initiative give birth to PT program. Quality improvement, satisfying accreditation requirement, satisfying payers requirement and positioning the laboratory on the competitive market place are the main reasons to PT in laboratory setting [2,3].

In Ethiopia, PT was started in 1992 at national reference HIV laboratory of Ethiopian public health institute (EPHI) for monitoring performance of HIV screening laboratories, then after at the end of 2001, 15 blood bank and regional laboratories was enrolled [4-6].

Quality is a vital in the diagnostic service of clinical laboratories for multiple perspectives like health improvement, customer satisfaction, cost, time and labour effectiveness, waste minimization, so it is very crucial to assess the utilization of PT scheme in the clinical laboratories it helps to look for the trend and the gap from participating laboratories in order to develop improvement plan by the authorized bodies for effective utilization and also increase the competence of participating laboratories by aware of their competence performance [7].

Participating in PT alone is not enough to improve the performance of the laboratory but evaluating the gap based on the feedback from PT provider and taking corrective action will improve the performance of the laboratory sustainably. Implementation of the corrective action is primarily the responsibility of the laboratory personnel and the management. Even if making participation in PT program is the mandatory part of implementation of accreditation/SLIPTA participant laboratories in developing countries has many challenges to utilize the feedbacks [8].

Until recently, however, the majority of Ethiopian public health laboratories delivered suboptimal service and were not in a position to contribute to a quality health system. Many performed poorly, hindered by dilapidated infrastructures, and poor development and implementation of quality management systems (QMS), including inadequate participation in external quality assessment (EQA) programs. Now, through strong commitment and leadership by the Federal Ministry of Health (FMOH) through the Ethiopian Public health Institute (EPHI), and the concerted effort of local and international partners, this has begun to change.

In recent years, public health laboratories in Ethiopia have begun to implement national and international QMS to provide quality laboratory services including participation in external quality assessment (EQA) programs. Participation in EQA/PT alone is not guarantee to insure the analytical quality but the laboratory management and the laboratory staffs need to work hard for improve the quality by filling the gap identified from EQA feedback but it is gap utilization of EQA in participant laboratories as indicated in the literature review part of this study [9].

Yet published data on utilization of PT result among participant laboratories in Addis Ababa governmental hospitals were limited. So the aim of this study was to assess the utilization and challenges of PT feedback among participant laboratories in Addis Ababa governmental hospitals and to provide information on to optimize assessing the trend of laboratories PT performance and identify challenges for effective utilization, plus to this, it helps as an information source for the policy makers to take action for the effective utilization of PT in order to minimize hazards due to analytical error [10-15].

Study design and settings

A Cross-sectional study design using qualitative and quantitative data collection approach was used from March to June 2014 in Addis Ababa, which is the capital city of Ethiopia, covers a landmass of 540 sqkm and a total population of 2.98 million according to 2008 central statistical agency report. It has 6 regional, 2 NGO supported, 30 private, 5 federal, 1 defense, 1 prison and 1 police hospitals laboratories; 70 (currently functional) public and 4 NGO-supported health centers laboratories, 7 public, 550 private and 31 NGO supported clinics laboratories [16].

The study area was chosen because it is most accessible area for good utilization of PT and also relatively high number of Participants laboratory in Ethiopia. Of 14 governmental hospitals found in Addis Ababa, 1 hospital was excluded using exclusion criteria and 1 hospital was piloted and the study was conducted in 12 governmental hospitals. Of which, 5 hospitals were from Addis Ababa city administration, 3 hospitals were from uniformed hospitals services, and 4 were from federal hospitals. All 12 hospitals were got PT from one world accuracy/digital PT through EPHI by postal services.

Sample size determination and sampling techniques

In this study qualitative data collection were conducted through FGD and KII using semi structured questioner. This tool is important to collect in-depth information from different key laboratory professionals engaged at different positions. For the quantitative study retrospective data was collected using customized formats. It was suitable for trend analysis of qualitative and quantitative laboratory test parameters and the remaining and also it used to assess participation rate [17-21].

12 government hospitals found in Addis Ababa was addresses through this study for quantitative data. 17 laboratory professionals among 10 hospitals of the 12 hospitals were participated on 2 FGD and 2 KII was done. Purposive sampling was used to include 12 from 14 government health facilities who were enrolled in EQA for 6 cycles (2012 and 2013 G.C).

Data on results of EQA samples for 20 commonly performed tests were collected retrospectively from laboratory PT feedback. SGOT, SGPT, ALP, BUN, Creatinine, TBI, DBI, WBC, RBC, Hgb, PLT, Morphology test: Parasitological-examination, Syphilis, HBsAg, HCV, HIV, CD4 count, AFB, Gram stain laboratory parameters were included in the study [22-24].

Qualitative data was collected using non-random sampling, a total of 17 laboratory head and quality officers were purposively participated on the FGD because of they have relatively more exposure in managing the utilization of PT in their facilities. And 2 key informants having good experience in mentoring hospitals under their offices for quality management system was addressed.

Operational definition

Analytical performance score: A value given by the PT provider to the participant laboratory for their PT performance in each cycle for each test parameter on the PT feedback.

Failure rate: The frequency of obey less than 80% analytical performance score in each test parameter.(80% performance score is a regulatory requirement of WHO-Afro).

Acceptable performance of participating laboratory: Defined a minimum achievement of 80% or more for each test run in proficiency testing program.

External Quality Assessment (EQA): A system for objectively checking the laboratory’s performance using an external agency or facility.

PT Challenges/unacceptable: Are the number of tests distributed in the panel of one test parameter in a single PT cycle. (e.g: AFB panel contain 5 slides (challenges) in a single cycle for a single participant.

Failure rate=

Cumulative analytical performance rate=

Participation rate=

Inclusion and exclusion criteria for the study

Those governmental hospital laboratories participated in PT scheme in 6 cycles of 2012 and 2013G.C and only volunteer hospitals and laboratory managers as well quality officers for this study were included. Hospital used for pilot study was excluded.

Data collection procedure

Retrospective data of 6 PT cycles (2012 and 2013 G.C) was collected for 20 test parameters by using piloted data collection template format. Data was collected from 12 government hospital laboratories as well the missed data were also obtained from EPHI archived PT feedback by decoding the name of health facility to secure the performance of individual hospital laboratory. Both the quantitative and qualitative data were collected from March to May 2014 G.C. Since focus group discussions and key informant interview was the research vehicle for this study, which provided rich, intensive information, this study employed such techniques to supplement and confirm the quantitative information as well as information on laboratory personnel, equipment, supply, administrative support towards utilization of PT. FGD and KII were used to collect information for identification of challenges by using detailed pre-tested semi-structured and open ended questionnaire. The data was collected by trained data collectors with the background of laboratory profession having one moderator one reporters’ and one supervisor. In addition, the data collection processes were supported by voice recorder.

Data quality

One day training was conducted for 5 data collectors (3 for quantitative data collectors, 2 for FGD and KII) and for 2 supervisor.

All data collection instruments were pre tested at St. Peter Hospital, one of government hospital laboratory and after we perform a pilot study, appropriate modification has been made. The quality of data entry was crosschecked by supervisors against the collected data and the quality of collected data was cross checked against the original data from EPHI, the quality of information jotted by the reporter during the KII and FGD was cross checked against the voice record. Participant on FGD and KII were laboratory professionals considering had better understanding on the points than other professionals.

Data processing and analysis

Responses and scores were entered and analyzed on a Microsoft Excel spreadsheet. For quantitative analyte graded as ACC (Acceptable) when the analyzed analytical value reported by the participated laboratory meet the established target value of the PT provider or UNACC (unacceptable)when the analyzed analytical value reported by the participated laboratory doesn’t meet the established target value of the PT provider this is true for both quantitative and qualitative test parameters. For all cycles (i.e. one cycle or an entire six cycles), the proportion of acceptable responses were calculated. Assessment of overall or cumulative performance in a component was performed by grouping all grading area responses together and calculating the proportion with acceptable scores was done using SPSS version 20 and those meet 80% and above are interpreted as acceptable analytical performance score or satisfy the regulatory requirement. Qualitative data obtained through the FGDs and KII were classified in major thematic areas and summarized accordingly.

Characteristics of participated institutions

From a total of 12 hospital participated proficiency testing program, of these 4 (33.3%) owned by federal ministry of health and 5 (41.7%) were under Addis Ababa Regional health Bureau plus 3 (17.0%) were uniformed service hospitals. A total of 12 (100%) laboratories participated in all six cycles from 2012 and 2013 G.C. All data of year 2012 and 2013 (6 cycles) PT feedback were collected. 17 individual study subject were participated on in two FGDs and 5 of them were female and 2 of the 3 eligible for in-depth interview were participated (Table 1).

Table 1: Characteristics of qualitative study subjects and laboratories participated in six cycle proficiency testing schemes from 2012-2013. Addis Ababa, Ethiopia.

From the participant 12 hospital laboratories in each cycle of 2012 and 2013 G.C., there were 5 PT challenges for each test parameter except CD4 (2 challenges) summarized in this study, from the 6 cycle of 2012 and 2013 G.C, only cycle 1/2013 shows relative better performance. Cumulative PT failure rates (UNACC) of 20 test parameter were 63.2, from these 63.8% for AFB, 50% for ALP, 48.6% for CD4, 68.1% for creatinine, 75% for Direct bilirubine,88.9% for gram stain, 68.1% for HBsAg and 39.9% for WBC was identified in this study as detail depicted at Table 2.

Table 2: Distribution of laboratory performances for 20 analyte in six cycles of proficiency testing program carried out during the years 2012–2013.

Trend of PT performance score using parameter for 6 cycles

When we explore the trend of cumulative analytical performance of the 20 test parameters of the participated 12 hospital laboratories, although there were high failure rate observed in the 2012 than 2013, no improvement project was observed across the 6 cycles. Relatively good performances were observed in cycle 1/2013 as shown Figure 1 below.

Figure 1: The trend of cumulative analytical performance for 20 test parameters among 12 hospital laboratories in each cycle of 2012 and 2013 G.C.

Trend of PT performance score by health facilities

The finding indicated that a total of 120 PT samples were distributed in the 6 cycles for each study hospitals. Those hospital laboratories coded as 105, 107, 108, 109 (4 laboratories) were accounts 41.23% failure rate among 12 hospital laboratories. As illustrated in the Figure 2 out of the 120 PT samples distributed throughout the 6 cycles for 20 test parameters for each hospital laboratory, only two hospital laboratories (code 102, 110) analytical performance score were 78 and 72 respectively, even though which did not meet the regulatory requirement, these results were relatively better as compared with the rest hospital laboratories (Figure 2).

Figure 2: Analytical performance score of 12 hospital laboratories through 6 PT cycles for 20 test parameters, (120 PT samples), in public hospitals, Addis Ababa, 2014.

Problems identified from PT feedback

The result of PT performance was categorized as acceptable or unacceptable for each test as of the sated criteria by PT providers. In this study, a total of 6984 PT challenges or Unacceptable results were noted, there were 5 challenges for each test parameter except CD4 which was committed for 2 challenges were identified, based on the data obtained, the failure rate from the total of 6, 984 unacceptable findings, the paramount problems were failure to participate, tests suspended during the test event, equipment failure and below liner limit of the analyte were 54.4%, 2.5%, 1.03% and 0.04 % respectively. A total of 240 PT samples were distributed for these 12 hospital laboratories for 20 tests, in each cycle, meaning 1440 total PT samples were circulated with in the 6 cycles. Based on this data, the participation rate was 645 out of expected 1440 throughout the 6 cycles which is 44.79% as described at Figure 3.

Figure 3: The cumulative participation rate of PT among 12 laboratories participated in 20 test parameters in 6 cycles (2012 and 2013 G.C), Addis Ababa, 2014.

Finding from focus group discussion and key informant indepth interview

In this study qualitative study was also applied to supplement the quantitative finding. In general two focus group discussions were conducted with laboratory managers and quality officers. A total of 17 participants were included during the discussion, of which 5 of them were female and also 2 key persons were interviewed to collect detailed information about utilization of PT, both key informants were male and from Addis Ababa regional laboratory and university partner with a practical back ground of laboratory profession.

Regarding the objective of PT. Almost all participants argue that PT is important “To identify the analytical problem and provide quality improvement for the identified gap to the maintain quality laboratory result for patient care.” in addition the participants also said that “PT is a tool for the identification of the competency of the laboratory personnel” “participating on PT and meeting 80% or more on two consecutive PT is a regulatory requirement for accreditation by ISO 15189 by ENAO.” Based on the above PT objectives majority of FGD participant were agree as PT is not meeting the objective in the situation of 12 hospital laboratories included in this study, whereas few participants are stated as they have good utilizations with some limitations.

The participants also asked challenges which can affect the utilization of PT in their situation. In this regard, the participants list different challenges as follows: “Even there is poor documentation, almost all laboratories did not identify their gap based on the PT feedback, they are not well understood the objective of the PT clearly.” There are also problem on the laboratory personnel “staff resistance for participating on PT is one of the challenge which is because of basic knowledge gap with most staffs and lack of experience on miss understandings the objectives of PT and not knowing how to interpret the feedback, how to summarize the challenges of the PT participation of their laboratory and also I belief that most of us have gap to develop the quality improvement plan so more is expected from the higher education’s teaching the laboratory profession and also the association must support the stakeholders for PT utilization”. The other identified challenge was “The reason for failure to participation of PT is due to equipment down time and reagent stock out”. Even if the above mentioned are the challenges on effective utilization of PT at all study subjects agree that the national central laboratory (EPHI) is not couching them sufficiently on effective utilization of the PT as our performance is not monitored by regulatory body in my opinion in addition to the participant laboratory the EPHI and regional laboratories are expected to identify our gap for effective utilization of PTP and early coaching.”

About their communication about PT feedback with internal and external clients, most of the study subjects were respond as they were not know their PT participation rate and there isno any system for their internal, external customers and their hospital administration “because of the management of the hospital is not aware of PT, they are not follow and support the laboratory for effective utilization of the PT.”

In this study, participants were also replied about the participation in PT should be voluntarily based or obligatory based, for this question, most of FGD participant agree as “participating in the PT should voluntarily based but indirect force is mandatory for effective approach. The type of indirect force should be creating competition among participant laboratories by awarding the best one, providing training for the staff, by increasing awareness of the administration… and the like.”

One of the KII participant put the reason for PT failure as follows “even if the major accountability for failure of PT utilization are the laboratory personnel’s working in each facility still EPHI is not invest more on mentoring, providing training, reviewing the performance status periodically rather than panels distribution”

Almost all participants agree that as “PT performance is not monitored by regulatory body of internal facility of external authorized body”. As that of the FGD participant both KII participant agree that “participating in the PT should voluntarily based but to improve the utilization more activities are expected from stakeholders and collaborators, strong supportive supervision by reviewing the PT feedback periodically is never missed”.

This study was focused to assess the utilization and identified challenges of PT feedback among participant laboratories in Addis Ababa governmental hospitals. The analyzed PT feedback of 2012 and 2013 G.C for 6 cycles of participated 12 hospital laboratories for 20 test parameters (SGOT, SGPT, ALP, BUN, Criatinine, TBI, DBI, WBC, RBC, Hgb, PLT, Morphology test, Parasitological-examination, Syphilis, HBsAg, HCV, HIV, CD4 count, AFB, Gram stain), indicated that they got their PT from one world accuracy PT provider through EPHI and for each cycle 5 challenges were identified for all test parameters except CD4 (2 challenges per each cycle).

Based on the finding, among different identified challenges which can affect PT utilization, staff resistance for participating on PT is one of the main challenge which is because of knowledge gap on miss understandings on the objectives of PT and not knowing how to interpreted the feedback, how to summarize the challenges of the PT participation of their laboratory and also, gap to develop the quality improvement plan and due to equipment down time and reagent stock out. In this study each 20 test parameters EQA failure rates were 63.8% and there were an evidence indicated that, there was no continuous improvement rather than participation, which is comparable with similar study conducted by Novak RW, which was indicated that PT feedback of group C streptococcus challenges from 1996 through 2001 the unacceptable performance was 19.6%, 16.7%, 19.5%, 18.2%, 20.2%, 19.0% in each year respectively, and there was also no continuous improvement was observed in the year 1996 through 2001 [22]. This study is incomparable with study conducted at department of Hematology, All Indian institute of medical sciences, New Delhi, which has been conducted in external proficiency testing program since 1992, which has showed that an improvement in overall percentage of laboratories with acceptable result and it increased from 38%, 40%, 40 % in 1992 to 85%, 90%, 94% in 2006 for Hgb, total leucocytes count, reticulocytes count, the difference could be due to due to low PT participation rate among distributed PT samples among the study hospitals [23].

From the total of 6984 PT samples distributed for 20 test parameters among 12 hospital laboratories within 6 cycles only 37.85% analytical value reported were acceptable (meet target value), which indicate there was 62.15% failure rate. Of which 41.23% failure rate is accounted by 4 of the 12 laboratories, which is comparable finding with similar study conducted in Western region of Amhara, Ethiopia, that quality of eight public medical laboratories were assessed for liver and kidney function tests of six analytes. 65% of 2013 values reported were failed outside of the allowable limits of errors for the chemistry test of the control specimens [17]. Too much high failure rate is identified in the current study than the study conducted in Sub-sharan Africa with 1.63% failure rate, which was 76% of failure rate was accounted in 4 of 21 laboratories this high difference in failure rate may due to the participation rate difference and difference in strong mentorship of regulatory bodies and the staff difference in the magnitude of challenges in utilization of PT [16].

In this study most laboratories where in the position of partial participation or not participate totally. Hence, the overall participation rate among 12 hospital laboratories for 20 test parameters throughout the 6 cycle was 44.79%. This finding is less than the study conducted in Amhara, Ethiopia, in which the quality of eight public medical laboratories were assessed for liver and kidney function tests of six analyte, the participation rate was 65.7%, these difference could be due to difference in the number of study test parameters and study institution [17]. The FGD and KII participant of this study agreed that the challenges for effective utilization of PT were knowledge gap on PT feedback interpretation, development of improvement plan, gap identification and root cause analysis as well as poor staff commitment on other hand maintenance problem for equipment down and reagent stock out are among identified challenges, which is comparable with study conducted by Carter JY et al. which was indicated that challenges identified for low rate of EQA participation in Contributing factors for low participation was shortage of staff and lack of time in busy rural laboratories together with difficulties in communication and lack of appreciations of the benefit of participation was identified challenges [12].

Beyond PT participation, there was no objective evidence for a continuous quality improvement for both the participation rate and the trend of analytical performance score, which is contrary to participating in PT allows laboratories to recognize analytical and interpretive error that may indicate internal problems with quality control, calibration, assay design, or test interpretation [10]. Furthermore according to Sciacovelli L, Secchiero S, Zardo L, Plebani M. ongoing monitoring of PT performance will help to reduce laboratory errors, produce accurate PT result mainly improve patient care [11]. As described on Bulletin of WHO, participating in PT alone is not enough to improve the performance of the laboratory but evaluating the gap based on the feedback from PT provider and taking corrective action will improve the performance of the laboratory sustainably and implementation of a continuous quality improvement process should be the central point of Proficiency testing/external quality assessment participation [7].

As indicated in the current study, the major challenge for PT failure is due to failure to participate. There are many factors which contribute for PT failure; these factors were related with personnel, equipment, and supplies. PT is not utilized effectively among the participated hospital laboratories as of the objective. Hence, stakeholders and collaborators should be actively participate and act accordingly in order to curve the low level utilization of PT feedback, and also participating laboratories should take appropriate root cause analysis/corrective action of their non-conformance.

Based on the finding of this study, the following points are recommended by the PIs for effective utilization of PT. The EPHI and regional laboratories together with partners should work hard on training for the laboratory personnel’s on service and pre service on utilization of PT. The higher education laboratory schools should provide competency based in-depth practically supported knowledge on PTP for their student. The laboratory manager together with the hospital administrative should minimize service interruption due to equipment down and reagent stock out. Laboratory associations should support by designing different approaches for effective utilization of PTP and also laboratories having best performance should periodically promote to increase the effective utilization of PT.

Reader should be into consideration the following limitation while inferring our result. Due to high number of participation data and administrative issues, this study focused only at public hospitals in Addis Ababa and the result would be more informative if more laboratories outside Addis Ababa and privates were included.

Ethics approval and consent to participant.

Ethical approval was obtained from Departmental Research and Ethics Review committee (DRERC) of School of clinical laboratory sciences, College of health science, Addis Ababa University and In addition ethical approval also got from Addis Ababa city regional Health bureau. Official permission to access the external quality assessment scheme feedback obtained from EPHI. Official permission was obtained from each study hospital for the questionnaire of gap identification. All results and information obtained from the study was kept confidential at all times in the lockable cabinet.

The dataset during and/or analyzed the current study, available from the corresponding author on reasonable request.

Ashebir Gurmessa and Abay Sisay conceived and designed the study and collected data, performed analysis, interpretation of data and also critically reviewed of this manuscript.

First of all we would like to thank to our almighty GOD who giving us good health throughout our life. We also thankful EPHI, Addis Ababa university and Addis Ababa Health Bureau and Respective Hospital Laboratories for giving the ethical approval visited by this study. Our last but not least gratitude goes to our study participants who gave their time and knowledge to provide all the necessary information, friends and families for all the support during this study.