The gold standard for diagnosing osteoporosis is measuring bone mineral densitometry (BMD) by dual X-ray absorptiometry (DXA) BMD is categorized as normal, osteopenic or osteoporotic [1]. When orthopedic surgeons treat broken bones, they literally get hands-on experience of the quality of bones. During fracture reduction and instrumentation, an experienced surgeon may be able to determine whether the bone is solid, indicating high bone quality or whether the bone is very fragile, indicating very poor bone quality.

Many patients with fractures are concerned about their future fracture risk, and occasionally they ask the surgeon whether they have osteoporosis. A response from any physician often has a major impact on patients, but how valid is the surgeons’ subjective intraoperative assessment of osteoporosis? Is the orthopedic surgeon able to give an accurate response if the bone quality is not one of the aforementioned extremes? These research questions seem very important, because many orthopedic surgeons hesitate to screen patients with fragility fracture for osteoporosis, as they claim to be without knowledge about treating osteoporosis. In a survey of 2021 orthopedic surgeons, less than 10% answered that they would use BMD when evaluating patients with fragility fractures and only 17% claimed to have knowledge about managing osteoporosis according to the results of BMD analysis [2].

The purpose of this study was to investigate orthopedic surgeons’ subjective estimation of bone quality against the gold standard, BMD measured by DXA.

This study was conducted as a prospective study at Aarhus University Hospital from January 2015 to March 2015. All patients 18 years or older with an acute fracture, who received operative treatment in the orthopedic trauma unit were eligible for inclusion. The trauma unit is a specialized sector within the Department of Orthopedics, and the surgeons in this unit are responsible for treating any skeletal fracture, except spinal and facial fractures.

During a 2 month period, all orthopedic surgeons were asked immediately postoperatively to grade patients’ bone quality on a 10 cm long visual analog scale (VAS) ranging from very poor bone quality (1) to extremely high bone quality (10). The evaluation was based on surgeon’s intraoperative experience.

Orthopedic surgeons were also asked about the status of patients’ osteoporosis (i.e., osteoporotic, not osteoporotic, or unable to answer). Within 3 months postoperatively, all patients underwent DXA. BMD was measured at the lumbar spine and total hip using Hologic densitometers. Bone status was categorized as follows: Normal: T-score >-1 at both sites. Osteopenia: T-score ≤ -1 at lumbar spine or total hip. Osteoporosis: T-score ≤ -2.5 at lumbar spine or total hip. Mean T-score was the average T-score at the lumbar spine and total hip.

Statistical analysis

Receiver operating characteristic (ROC) curves were used as diagnostic tools to determine the accuracy of the VAS score against bone status: osteoporosis or abnormal bone status (osteopenia or osteoporosis) [3,4]. Nonparametric methods were used to calculate the area under the ROC curves and bone status assessment was binary. An area of 0.90-1 represents an excellent test, 0.7-0.9 represents a test with moderate accuracy, 0.5-0.7 low accuracy and 0.5 represents a fail test with same performance as by chance alone [4]. All statistical analysis was performed using STATA 12.1 software (Stata Corp., College Station, TX, USA).

Ethical approval

According to Danish law, the Regional Ethical Committee concluded that a formal approval was not required. Written consent from patients was obtained before DXA.

Fifty-three patients (40 women) were included in this study and evaluated by 13 orthopedic surgeons. Patients’ mean age was 58 years (standard deviation [SD] 16.79, range 22-88 years). The most common fracture sites were the wrist, ankle and hip. Osteopenia was found in 26 patients and 12 patients were diagnosed with osteoporosis. Eight patients out of 38 with osteopenia or osteoporosis were men. The mean T score was -1.11 (SD 1.37, range -3.5-2) and the mean Z score was -1.23 (SD 1.24, range -3.5-2). The mean VAS score produced by the surgeons was 5.5 (SD 2.48, range 0.25-10) (Table 1).

Table 1: Anatomic distribution of the fractures.

Areas under the ROC curve for measuring the accuracy of the VAS as a diagnostics tool for osteoporosis were 0.70 (confidence interval [CI]: 0.51-0.88) and 0.73 (CI: 0.56-0.89) for diagnosing abnormal bone status (osteopenia or osteoporosis). The sensitivity and specificity for different cutoff point on the VAS scale for diagnosing osteoporosis is presented in. In 15 patients (28%), the surgeons were unable to conclude whether osteoporosis was present or not, the mean VAS score in these patients was 5.1. The positive predictive value of surgeons’ conclusion of osteoporosis was 50%, and the negative predictive value was 83%. If surgeons’ conclusion of osteoporosis was used as a surrogate for abnormal bone status (osteopenia or osteoporosis), the positive predictive value increased to 86% (Figures 1 and 2 and Tables 2 and 3).

Figure 1: ROC curve for measuring the accuracy of the VAS as a diagnostics tool for osteoporosis.
ROC: Receiver Operating Characteristic; VAS: Visual Analog Scale

Figure 2: ROC curve for measuring the accuracy of the VAS as a diagnostics tool for abnormal bone status (osteopenia and osteoporosis).
ROC: Receiver Operating Characteristic; VAS: Visual Analog Scale

Table 2: Sensitivity and specificity for each cutoff point on the visual analog scale for diagnosing osteoporosis.

Table 3: Surgeons’ conclusion about patients’ bone quality.

Surgeons’ conclusion that osteoporosis was present was only correct in 50% of patients, but an additional 36% of patients had osteopenia. Overall, the orthopedic surgeons’ subjective evaluation of bone quality seems to be a fair screening tool and adding a simple instrument, such as VAS, increases the usability and relevance of their intraoperative evaluation. A diagnostic test should be accurate, simple, rapid, and inexpensive to use in the clinical setting. In this study, we described a tool that almost fulfills these criteria. The tool, which is fairly accurate, may be used to identify patients with fracture who may require further diagnostic evaluation with DXA [5].

We also compared BMD based on DXA scans against orthopedic surgeons’ subjective evaluation of bone quality. One may suggest that when orthopedic surgeons drill the bone and insert implants intraoperative, they should obtain a whole sample biopsy of solid bone, including both cortical and trabecular bone, instead of performing a histological analysis, to get a hands-on perception of the bone quality. However, skilled orthopedic surgeons assess bone quality intraoperatively and treat the patient accordingly. In patients with very fragile bone, different kinds of implant choices may be used, such as locking screws and plates, helical blades and various augmentation techniques [6,7].

In an in vivo study of 62 patients undergoing spinal operation with lumbar intervertebral fusion, a correlation between the maximum rotational force needed to insert pedicle screws and BMD was found, although this study and other studies could not find a relationship with fixation failure [8,9]. The latter is perhaps because the stability of implants is not only affected by the bone quantity and quality [10]. Today, several commercial devices are available and they can be used intraoperatively to mechanically test the patient’s bone strength by measuring the limits of rotational forces or stability of dental implants and the relevance of these devices in spine, hip, foot and dental operations has been demonstrated [10-12].

When orthopedic surgeons were asked to make a conclusion about a patient’s status of osteoporosis, it seems that they hesitated in several patients and could not draw a conclusion, but when they are certain about the presence of osteoporosis, their conclusion of an abnormal bone status (osteopenia or osteoporosis) has a high predictive value.

The VAS in this study was very easy to use in clinical practice and the results from the ROC curves have shown a fair association with abnormal BMD. The VAS is a well-known tool used to measure pain and quality of life, but to our knowledge, it has never been used as an instrument to measure bone quality. We used a 10 cm long VAS and suggest a cutoff point between 3 to 4 cm. Using 4 cm as cut-off, the VAS tool correctly classified 75% of 53 patients and using 3 cm the sensitivity raises to 95% and 80% are correctly but it is at the expense of a lower specificity. The choice of cutoff must be based on an overall assessment of the morbidity from the screened disease, the prevalence and the risk from further investigation. If orthopedic surgeons use 4 cm as cutoff point to select the patients for a DXA, then 6 of 10 patients will unnecessarily undergo DXA, however, in view of the morbidity associated with osteoporosis and future fracture risk in relation to the risk and cost of DXA, this seems reasonable [13].

One main weakness of this study was the lack of reliability testing. In an optimal setting, repeated measurements should be performed by the same orthopedic surgeon over time, and different orthopedic surgeons should evaluate the same patient. In most patients, we aim to operate on them only once, so this is not feasible. Additionally, the high prevalence of osteoporosis and osteopenia in our study population is a potential bias, and it increases the probability of a DXA showing osteoporosis or abnormal bone status. Spectrum bias may affect the performance of the test because of the case mix of patients [14]. A different case mix can affect both the sensitivity and specificity of the test. A higher number of patients may reduce the bias. Despite the lack of patients with spinal fracture, we thought our case mix was representative of general patients from an orthopedic clinic. In contrast to many other studies, we included younger patients; our patients’ mean age was only 58 years. Moreover, the orthopedic surgeons in this study were not blinded to the radiographs, as radiographs are mandatory tools in orthopedic operations; therefore, interpretation of the radiographs may have influenced the surgeons’ conclusion given post operatively. Previous studies have shown that estimation of the cortical thickness of the distal radius from standard radiographs of the wrist in the post anterior view can be correlated with the BMD [15]. In our study, 24% of patients had a wrist fracture. Although we cannot estimate the impact of evaluating the radiographs in an unblinded manner on our results, our study’s setting reflects daily clinical practice and from this experience, surgeons can better discuss osteoporosis with their patients.

Further studies with a larger population are needed to optimize the subjective evaluation of bone quality, refine the VAS, validate it for different anatomic locations, and further strengthen the specificity of the evaluation. The design of future studies also need to address the reliability by including repeated examination of the same patient.

Orthopedic surgeons’ subjective intraoperative evaluation cannot replace results of DXA scans, but when using a simple VAS tool to subjectively grade bone quality based on surgical experience, orthopedic surgeons can identify which patients will require further evaluation with DXA. Orthopedic surgeons do not have the last word about osteoporosis, but when the conclusion is that osteoporosis is present, patients need to listen.