To ensure the optimal potency of time and temperature sensitive pharmaceutical products (TTSPPs), careful attention is needed in handling practices at the country level. These include storage and transport of TTSPPs from the manufacturers through wholesalers down to the end-users at the service centres (pharmacies and health centres). The World Health Organization (WHO) recommends that all vaccines should be stored at between +2°C and +8°C at all immunization points in the periphery (health centres, health houses, hospitals, clinics, and pharmacies). Liquid formulations of vaccines with aluminium adjuvant containing diphtheria, pertussis, tetanus, hepatitis B, Haemophilus influenzae type b, IPV and their combinations should not be frozen [1-3]. TTSPPs also require cold storage, similar to vaccines, they are stored refrigerated at +2°C and +8°C unless stated otherwise by the manufacturer.

Practices exposing TTSPPs to both high and sub-zero temperatures are widespread in both developed and developing countries at all levels of health systems. However, major focus in published temperature monitoring studies is mainly vaccines [4-15]. One recent study on oxytocin supply chain from manufacturer in Germany to service level in Ghana found temperatures ranging from -9.9°C to +30.1°C at the country level [16]. The most recent systematic literature review of vaccine freezing highlights that accidental freezing is widespread and °C curs across all segments of the cold chain [17].

Temperature monitoring in the supply cold chain is critical in ensuring that products are kept under optimal cold chain conditions. The general practice for temperature monitoring in refrigerators at the periphery is to use a thermometer (alcohol stem thermometer or bimetal thermometer). However, we know that thermometer is NOT a “monitoring” device; because it is only good when you look at it.

Oxford dictionary defines monitor as follows: “A device used for observing, checking, or keeping a continuous record of something.” That means monitoring includes an action that is to pay continued close attention to [something] for a particular purpose. Here “continuity” is the key, and thermometer has no continuity. When checked and a temperature value of between +2°C and +8°C is found, pharmacist may erroneously conclude that the products are safe since this snapshot reading only provides a value by the time it is checked and by no means covers the rest of the day/night period. Unless a temperature excursion is seen at the time the temperature is checked with a regular thermometer, almost all temperature violations go unnoticed. A temperature monitoring study conducted in Albania showed that 110 low (below -0.5°C for 1 h) and 98 high (above 8°C for 10 h) alarms were missed by health centres using regular thermometers during three months of study period [18].

The very first specialized temperature monitoring device for refrigerators was prequalified in 2007 by the Performance, Quality and Safety (PQS)13 project in the Department of Essential Medicines and Health Products of WHO [19]. These devices have the following features:

1. Capacity to store the last 30-day temperature recordings that can be checked through a history mode and read through an LCD display (or optionally downloaded).

2. Set alarms for high (above +8°C more than 10 hours continuously) and low (-0.5°C and lower for 1 h continuously) temperature exposures.

3. LCD display of the highest and lowest temperatures reached as well as their duration compared to set alarms through the history mode- for details of the performance specifications of the device see WHO/PQS/E06/TR06.1 [20].

Fridge-tag^® was the first model prequalified by the WHO - prequalified on 30 September 2007, and Vaxtag^® the second model on 19 May 2011 [19,21].

This study is designed to establish the programmatic value of a 30-day electronic refrigerator temperature logger in pharmacies using Fridge-tag^® and Vaxtag^®, as well as assessing these two devices’ userfriendliness and their abilities in field conditions.

Retail pharmacies in Greece

Greece has 13 administrative regions (peripheries) as first level administrative entities, each compromising several secondary level units (originally prefectures and, since 2011, in total 74 regional units). This study was conducted in Magnisia and Sporades regional units of Thessaly periphery. There were 236 of pharmacies in the study region, representing 2.3% of total pharmacies (10,386) in the country. The population density of pharmacies in Greece is the highest among the European Union (EU) Member States, with ratio of 1:1000 compared to 1:3300 EU average. More than 60% of pharmacists are women, while the central tendency in the age distribution is between 55 to 65 years, very close to retirement age. In general pharmacies are quite small size stores, hardly exceeding 50 m2. Most pharmacies work five days a week, mostly from 8 am to 2 pm and 5 pm to 9 pm. Significant number of pharmacies work on Saturdays from 9 am to 2 pm. Along with other countries, such as Spain and Italy, pharmacies in Greece follow so called “Mediterranean” model in contradiction to “Northern” European model prevalent in countries of central and northern Europe; which is scattered, many in numbers, small in size, and working only with one pharmacists (the owner). Office of Public Health of the Periphery oversees the opening process of a retail pharmacy. Same office is also responsible for monitoring of the pharmacies’ activities including management of the cold chain. Having a refrigerator is an obligation for a pharmacy, however, regulations do not specify any specificities for the refrigerators.

Study design

This study was designed as an intervention study with the objective of establishing the superiority of the 30-day electronic refrigerator logger against thermometers. Two-stage cluster sampling was used to obtain a representative sample of retail pharmacies Magnisia and Sporades regional units. Self-weighting was ensured in such a sample through a choice of municipalities in which clusters are selected by using probability proportional to population size, and by selecting an equal number of sampling units in each cluster. Twenty retail pharmacies were selected as “intervention” groups and another 20 retail pharmacies were selected as “control” groups (Figure 1).

Figure 1: Study design.

Fridge-tag^® and Vaxtag^® devices were randomly assigned to intervention pharmacies and responsible pharmacists were trained accordingly either on the use of Fridge-tag^® or Vaxtag^® through a series of demonstrations by the study team followed by drill and practice by participants. No training was provided to the pharmacists from the “control” sites. All refrigerators in the intervention and control sites were equipped with downloadable electronic data loggers as a reference (Trix-8 LogTag^® recorder, WHO PQS code E06/06 prequalified on 22 January 2008) to record temperatures for three months [22].

Intervention sites were asked to monitor temperatures during the study period using either Fridge-tag^® or Vaxtag^®, and a modified temperature control record sheet, while control sites continued with their routine operation with thermometers and recording the temperature data on record sheets. At the end of the three-month data collection period, electronic data loggers and temperature monitoring record sheets were collected back from both groups. In addition, focus group sessions were conducted with the participants of the study to discuss temperature monitoring, the use of Fridge-tag^® and Vaxtag^®, and their user-friendliness (Figure 2).

Figure 2: Fridge-tag^® and Vaxtag^® devices used in the study.

Downloaded data from the electronic data loggers were crosstabulated with temperature monitoring record sheets from both groups and further analyses were conducted to document overlapping and missed information.

The following hypothesis were tested:

1. Pharmacists with Fridge-tag^® and Vaxtag^® devices are more confident in temperature monitoring compared to pharmacists with thermometers.

2. Pharmacists with regular thermometers are not aware of majority of temperature violations in the refrigerator.

3. Fridge-tag^® and Vaxtag^® devices are capable of catching all temperature violations while such go unnoticed with thermometers.

The distribution of types of refrigerators in both the intervention and control sites were similar. Both sites had 14 household and six pharmaceutical type refrigerators each. The oldest refrigerator was 27 years old in both groups. Comparison of make of year of refrigerators in both groups is given in Figure 3.

Figure 3: Comparison of make of year of refrigerators in study and control groups (Box and Whisker plots).

Volume of the refrigerators in intervention and control groups varied between 80 to 530 L and 50 to 420 L respectively. All refrigerators were front-opening. Eight of these refrigerators in both groups had a freezing compartment, remaining 12 were with no freezer. Prior to the study, temperature control was done with thermometers (14 alcohol stem and six digital thermometers in intervention group; and 15 alcohol stem and five digital thermometer in control group). Although electricity cuts are extremely rare in Magnisia and Sporades regional units, four of pharmacies in the intervention group had generator while none of the pharmacies in control group had any. Thirteen intervention and nine control sites had security alarms.

Significant discrepancies were observed between the data from thermometer readings and electronic data loggers in control group. A discrepancy is defined as °C curring where the difference between two readings is greater than the sum of accuracy of both devices, e.g., since the accuracy of thermometers is ± 1°C and of data loggers is ± 0.5°C, differences in readings of 1.6°C and higher were accepted as discrepancy in the control group. Discrepancy was set to be equal or greater than 1.1°C in the study group since both devices’ accuracy was ± 0.5°C. With thermometer readings, there were only four °C casions that the temperature was recorded below 0°C by two pharmacies (-2.0°C, -1.0°C, -1.7°C, and -1.3°C), while all other pharmacies measurements ranged from 0°C to 11.5°C. During the same period, electronic data loggers recorded temperatures ranging from -5.6°C to 30.0°C. Figure 4 displays the discrepancies through minimum and maximum temperatures recorded in the control group.

Figure 4: Comparison of thermometer readings and electronic data logger recordings by pharmacy, control group (n=20)

In the intervention group, Fridge-tag^® and Vaxtag^® readings overlapped with the electronic data logger readings without any discrepancy in 17 cases. In three cases, readings overlapped at the lower end but higher temperatures were recorded with Fridge-tag^® and Vaxtag^® at the higher end. All three cases reported that the device was forgotten outside after reading for longer than 10 min. When these three cases were ignored, Fridge-tag^® and Vaxtag^® readings were in full overlap with the electronic data logger recordings. Figure 5 displays the minimum and maximum temperatures recorded in intervention group by the centre.

Figure 5: Comparison of Fridge-tag^® and Vaxtag^® readings and electronic data logger recordings by centre, intervention group (n=20).

Minimum and maximum temperature recordings in intervention and control groups in comparison with the data logger are given in Tables 1 and 2.

Table 1: Minimum and maximum temperature recordings in intervention group in comparison with the data logger.

Table 2: Minimum and maximum temperature recordings in control group in comparison with the data logger.

The alarm status in the control group based on the data logger readings is shown in Table 3. WHO recommended alarm settings for the 30-day electronic refrigerator temperature logger were used to illustrate the missed alarms. In the control site, 11 pharmacies experienced temperatures below 2°C and above 8°C without having an alarm. Nine pharmacies had at least one alarm either high or low. Tables 3 and 4 display alarms in control (calculated) and the intervention groups (recorded and confirmed by the data loggers).

Table 3: Calculated number of alarms and exposure times according to the data logger readings in the control group (n=20).

Table 4: Recorded number of alarms confirmed by electronic data logger in the intervention group (n=20).

Figure 6 illustrates electronic data logger recordings along with marking lines of minimum and maximum thermometer readings in pharmacy #8 in control group. Based on the WHO recommended alarm settings for the 30-day electronic refrigerator temperature loggers, missed alarms are also indicated.

Figure 6: Comparison of thermometer and electronic data logger recordings, Control group (pharmacy #8).

In Figure 7, temperature records from the data logger is displayed in another pharmacy (#4) from control group. The figure illustrates the longest three continuous high temperature violations that were recorded during the study.

Figure 7: Comparison of thermometer and electronic data logger recordings, control group (pharmacy #8).

The following Figure 8 illustrates electronic data logger recordings along with marking of Fridge-tag^® and Vaxtag^® readings in two selected pharmacies from the intervention group (#4). All alarms indicated by the Fridge-tag^® and Vaxtag^® devices are confirmed by the electronic data logger.

Figure 8: Comparison of Fridge-tag^® and electronic data logger recordings, Intervention group (pharmacy #4).

In the control group, 11 pharmacies did not experience any alarms while no alarm was observed in 13 pharmacies in intervention group. Although there were no alarms, in both groups, all refrigerators had temperature violations, but shorter than duration required to trigger an alarm. In control group, a minimum of one alarm was observed in nine pharmacies based on data from the electronic data loggers. Each of two pharmacies experienced a total of 65 alarms during the three-month study period. In control group, only two health centres had confirmed four of the temperature violations at the alarm level (pharmacy #6 and #7) through the use of thermometers. All other alarms in those two pharmacies, as well as the alarms in the other pharmacies had been missed. In total, 173 low and 77 high alarms were calculated in the control group. Although no systematic alarm pattern was observed in the control group; four pharmacies had alarms every month. Alarms were found in two of the months in four pharmacies and one pharmacy had alarms only in one of the months.

Large discrepancies were discovered between the thermometer and electronic data logger device recordings in the control group. These discrepancies varied from 27°C at the high end to 7.6°C at the lower end. This is understandable to some extent since a thermometer reading is a “snapshot at a point in time” activity while electronic data logging is “continuous”. However, despite electronic data recordings being below -0.5°C for longer than 55 h, manual recording through thermometer readings were found to be only between 2.0°C and 7.5°C. Similarly, in another pharmacy from the control group, electronic data logger recordings indicated temperature excursion above 8°C for 22 consecutive days (and reaching to a maximum 19.9°C), thermometer readings indicated temperatures between 5.0°C and 9.0°C during this period. This raises questions as to whether staff really checked the thermometers before recording the reading manually.

In the intervention group, all Fridge-tag^® and Vaxtag^® readings at the lower end were confirmed by the electronic data logger. The variation of readings between the Fridge-tag^® and Vaxtag^® and electronic data logger was found to be between 0 and 0.8°C. Since the accuracy of the both devices is ± 0.5°C, such differences are considered within the accuracy limits, therefore as overlapping. In three cases, quite a big difference was observed at the higher end readings. In all three cases, in recording form, it was indicated that the device was forgotten longer than 10 min outside the refrigerator following reading (both devices have a lock function for 10 min to prevent high spikes during reading due to heating of the device). When this short-lived spike readings are ignored, full overlap could be concluded for both Fridge-tag^® and Vaxtag^® devices.

In total, throughout the study period, there were 29 low alarms and two high alarms in the intervention group. Since the pharmacists were aware of these alarms, they took corrective measures such as regulating the thermostat, and managed to reduce or eliminate the alarms in the following months. The highest temperature recorded in intervention group (28.4°C) was indicated as to overlap with the cleaning of the refrigerator. All other pharmacies also indicated explanations for temperatures higher than 8°C usually due to placing newly arrived products to the refrigerator.

Very low correlation was found between the age of the refrigerator and number of alarms (R2 =0.0972). All alarms appeared in household type refrigerators in both groups.

Semi-structured interviews were held with all participating pharmacy staff through focus group discussions. Except a couple of exceptions, both study and control groups mentioned that with thermometer use, they never felt confident about the temperature monitoring. Study group mentioned that they were happy and lucky to be chosen as the study group that they have experienced these new devices. In the study group, all participants stated that their opinion was changed and that they felt more confident and comfortable about both, the quality of the measurements and the fact that they could know what was happening in their refrigerator during weekends and official holidays.

“Though I said I was confident with the thermometer, seeing what this new device could do, especially showing what has happened during the night and weekends, now I know that thermometer is not a solution.”

Some of the participants in the study group mentioned that they would feel even more confident if there was an alert system, like an SMS alert.

All participants, regardless of their age, praised the devices that they are easy to use and read.

All participants in the study group supported the idea of these type of new devices becoming the norm and are forced by regulations by abandoning thermometers in pharmacies.

When asked about the possibility of missing some alarms, in control group majority of pharmacists agreed that this could happen during weekends “It is a simple thermometer.” Control group pharmacists also agreed that the devices study group was using is the answer to better monitoring in refrigerators that they all support such devices to become a norm in Greece.

Some pharmacists also brought up use of domestic refrigerators contributing the quality of control of pharmaceuticals, though couple of them mentioned the price could be an issue.

Temperature monitoring is critical in pharmaceutical cold chain. Responsible personnel in pharmacy handling TTSPPs at the periphery should have tools to discover problems and, based on the information, take necessary measures on time. Thermometers (alcohol stem or bimetal) do not provide necessary level of security and can only provide a spot check rather than truly “continuous” monitoring. It is very unlikely that a pharmacist could catch a cold chain violation using a thermometer, unless the problem continues throughout the day and, by chance, it is observed when the temperature spot check is made. The 30- day electronic refrigerator temperature loggers bring a highly significant improvement in temperature monitoring. They allow responsible personnel to see how temperatures changed throughout the day/night cycle, including weekends and holidays; so they can determine whether there were any unacceptable temperature exposures. With the help of such devices, pharmacists can take informed decisions as corrective measures to reduce and/or eliminate cold chain problems. This is the only way that they can ensure pharmaceutical products to their clients that are not compromised.

The authors strongly recommend abandoning the use of thermometers as temperature monitoring devices for pharmacy refrigerators and replace them with 30-day electronic refrigerator temperature loggers. Household type refrigerators are not designed to store pharmaceutical products, and should not be used in any of the health care facilities including pharmacies.

As part of the research communication plan, following these conclusions, investigators had a meeting with the President of the Greek Board of Pharmacies in Athens. Policy implications of the findings and recommendations from the research team were discussed. The following action points were agreed upon:

• Findings and recommendations from the study will be published in summary in forthcoming issue of the official journal of the Board of Pharmacies.

• Full manuscript will be submitted for publication in a peer-review journal.

• A new guideline will be prepared by the Board of Pharmacies on best practices on temperature monitoring at the last mile with pharmacies. New devices and the importance of using pharmaceutical refrigerators will be the main focus of this new guideline. This guideline will be printed and distributed to all pharmacies through the association.

• Board of Pharmacies will contact the Ministry of Health for possible policy changes to be translated into law.

• The study and these action points will be presented in forthcoming November 2018 Pharmacies Forum.

This study was conducted in association with Cooperative of Pharmacists of Thessalia (in Volos), and Confederation of Cooperatives of Pharmacists of Greece (OSFE), and was co-sponsored by Berlinger & Co. AG and LogTag Recorders Ltd. The sponsors were not involved in the design, data collection, analysis, interpretation or writing of this report, nor the decision to submit the article for publication.

None declared.