Article Text
PDF
Abstract
Objectives The authors examined the validity, interobserver reliability and interscanner variation in detecting bone erosions with ultrasonography using a custom-made phantom.
Methods 21 bovine bones were used. Artificial erosions were made into 15 bones and six bones were left as controls. In the processed bones the numbers of erosions, their depths and widths varied between 1–7, 1–4 and 1.5–5 mm, respectively. Each bone was coated with polyvinyl alcohol cryogel to mimic overlying soft tissue and to hide the erosions. Four musculoskeletal sonography experts scanned the 21 blind-coded phantoms using one of the three sets of ultrasound equipment. Finally, quality assurance measurements of the ultrasound equipment was carried out using two additional bone samples.
Results The sonographers detected the erosions successfully with ultrasound. The mean correlation coefficient for a correct result in terms of the number of erosions detected was 0.88 (range 0.75–0.975). The overall Cohen's κ coefficient for interobserver agreement was 0.683 in terms of discrimination between healthy bones and bones with erosions. The different sets of equipment showed that their overall performance was equal.
Conclusions The sonographers had good correlations with the number of erosions and they were successful in separating healthy bones from bones with erosions. It seems that neither depth nor width is crucial but that in experimental conditions a 1.5 mm erosion width was the limit for the resolution with current ultrasound equipment. Ultrasound is a valid and reliable method of detecting cortical bone erosions in vitro, when the round erosion is at least 1 mm deep and 1.5 mm wide.
Statistics from Altmetric.com
Grey scale ultrasound imaging (B-mode) is an imaging method widely used in rheumatology to detect soft tissue lesions in synovial structures such as joints, tendons and bursae as well as structural damage.1,–,12 In ultrasound imaging bone erosions appear as interruptions in the bone profile and they must be scanned at two perpendicular planes.13 Because of the multiplanar nature of the scanning process ultrasound imaging has been reported to be more sensitive in the detection of cortical defects than conventional radiography, provided that an adequate acoustic window is available.4 14,–,21 Furthermore, studies comparing erosions detected by ultrasound, CT and MRI have confirmed the accuracy of ultrasound.4 14,–,16 On the other hand, multidetector CT has been demonstrated to be more sensitive than MRI for detecting cortical erosions in a rheumatoid hand.20 21 In other words, there is a discrepancy between magnetic resonance erosions and lesions detected with CT. Consequently, the in-vivo imaging of the erosions performed in earlier studies is always a comparison between different imaging modalities and the absolute numbers and sizes of the erosions on the bone surface are not known. In order to characterise the diagnostic potential of different imaging modalities completely for detecting bone erosions the imaging results should be comparable with the ‘absolute truth’, that is, compare with the true numbers and sizes of the erosions. However, this is impossible under clinical circumstances, as it would require surgical procedures to expose the bone surface.
The objective of this study was to develop a new bone erosion phantom using bovine bones and coating them with a polyvinyl alcohol cryogel (PVA-C) layer in order to mimic an arthritic bone with erosions and soft tissue. Artificial erosions were created by drilling round holes of different depths and widths into the bone surface. The numbers and dimensions of the man-made erosions were thus known accurately. The phantom model was subsequently tested in ultrasound erosion scanning. The validity and interobserver reliability of detecting erosions with ultrasound as well as the interscanner variation between three sets of ultrasound equipment were evaluated in vitro.
Materials and methods
Preparation of bones and creation of artificial erosions
Altogether 21 bovine lower leg bones were obtained from a local slaughterhouse (Atria Oyj, Kuopio, Finland). Soft tissues were removed in the autopsy theatre and the bones were fixed in 4% formaldehyde for 3 weeks and left to dry for a month.
The bones were first rubbed down with a machine and sandpaper to remove the articular cartilage. The cartilage tissue was removed because preliminary tests showed that a PVA-C-coated cartilage did not act in a similar way in ultrasound scanning as in vivo (unnatural double echoes occurred). After rubbing down the bones were totally smooth to the touch and the eye. Cortical round holes of different sizes were then drilled randomly around the whole bone using a Makita Driller 6270D (Makita Corporation, Anjo, Aichi, Japan) and different titanium drill bits (Target Tools Ltd, Braddan, UK) into 15 bones, and six bones were left untouched (‘healthy’ or control bones). The numbers, depths and widths of the erosions ranged between 1–7, 1–4 and 1.5–5 mm, respectively (table 1). Preliminary tests showed that a round hole of the width of 1.5 mm could be detected with ultrasound and this was therefore chosen as the smallest erosion width in 15 bones. The dimensions (width and depth) of the erosions were verified after drilling using a digital propulsion meter (Biltema Suomi Oy, Helsinki, Finland). According to the manufacturer the measurement accuracy of the propulsion meter was ±0.03 mm. A metal wire was placed on the neck of the bone to keep the PVA-C material in place during the whole time of the study (figure 1A).
A test bone with two erosions before (A) and after it was coated with polyvinyl alcohol cryogel (PVA-C) material (B, the phantom).
Number and dimensions of erosions on the bones in the study
Preparation of PVA-C material and phantoms
An amount of 43 g of PVA-C material (polyvinyl alcohol 28–29; Fluka Chemie, Steinheim, Luxembourg) was mixed with 0.5 litre of de-ionised water and left untouched for 24 h. The mixture was slowly heated up to 96ºC mixing it continuously for an hour. The aqueous PVA-C solution was then poured into a mould in which a drilled bone was also placed. This way each bone was coated with an approximately 10–20 mm thick PVA-C layer. The filled mould remained untouched at room temperature for 12–24 h. When all air bubbles had disappeared the mould with the PVA-C solution and a bone was put in a freezer for 24 h. Then it was put in a closed styrofoam box and kept at room temperature for 24 h so that the solution would thaw slowly. Five cycles of freezing and thawing were needed to complete the preparation of a phantom (figure 1B).
After preparing the phantoms, two additional bone samples were prepared for the quality assurance measurements of the ultrasound equipment. Cortical round holes in four lines were manually drilled into these two bones. The erosion depths in the lines were 1.5 and 5.0 mm. The erosion widths were 1.0, 1.5, 2.0, 3.0, 4.0 and 5.0 mm. For practical reasons it was not possible to prepare smaller (<1.0 mm) erosions on the phantoms or test bones. These additional bones were inserted into a small box filled with PVA-C solution. The ‘test box’ with bones and PVA-C solution underwent the same freezing and thawing cycles as the 21 phantom bones.
The acoustic properties (speed of sound, ultrasound attenuation, elastic modulus and density) of the PVA-C material to mimic soft tissue have been confirmed in our earlier study.22
The stability of the phantoms was controlled with 64 multislice CT scans (General Electric Light Speed VCT, Milwaukee, Wisconsin, USA) using a 0.67 mm slice thickness before and after the trial over a period of 6 months. The CT scans showed that the phantoms remained stable during the 6-month trial.
Ultrasound scanning of the phantoms
Four musculoskeletal sonography experts, two radiologists (KK, IS) and two rheumatologists (EA, JMK) scanned the 21 blind-coded phantoms with ultrasound. The investigators were told that the bones were of young oxen and the growth plates as well as the metal wire on the neck of the bone could be detected with ultrasound (figure 2). The investigators reported on a document sheet whether they had detected erosion(s) or not. They were asked to give the number of erosions in each individual bone. The ultrasound equipment used in this study was Siemens Antares with a VFX 13-5 MHz linear probe (KK and IS), Esaote Technos with a LA424 13 MHz linear probe (JMK) and General Electric Logiq 7 with a 12 MHz linear transducer (EA). Each investigator used one piece of equipment.
Ultrasound image of the phantom.
After reporting their results of the phantoms they were given the ‘test box’ and asked to scan the four lines to get as clear images as possible using the same probes and settings as they had used when scanning the 21 phantoms (figure 3).
Ultrasound images of one of the additional test bones. The depth of the erosions is 5 mm and the widths are shown in figures. See text for the equipment and probes used (Esaote, Siemens and GE).
Statistical analysis
Spearman's correlation analyses between variables were tested for two-tailed probability values. Values of p<0.05 were considered significant. The interobserver agreement was assessed by calculating a κ coefficient between the readers.23 24 Generally, interobserver agreement can be interpreted as follows: κ<0, κ=0.00–0.20, κ=0.21–0.40, κ=0.41–0.60, κ=0.61–0.80 and κ=0.81–1.00 signifying poor, slight, fair, moderate, substantial and almost perfect agreement, respectively.
Results
The observers could generally detect the erosions well with ultrasound. The mean correlation coefficient for a correct result in terms of the number of erosions was 0.88 (range 0.75–0.975) (figure 4). The small and large erosions in width and depth could be detected with a similar reliability. One observer was able to separate all bones correctly as either healthy (control) or with erosions. Two sonographers classified one healthy bone wrongly as having erosions (not the same bone) and the fourth observer made three mistakes in the group of 21 bones (one bone with erosions was mistaken for a healthy one and two healthy bones were reported as having erosions). The overall Cohen's κ coefficient for interobserver agreement was 0.683 in terms of discrimination between healthy (control) bones and bones with erosions. The Cohen's κ between two observers ranged from 0.475 to 0.877 (table 2). The agreement in percentages between two investigators in terms of discrimination between healthy (control) bones and bones with erosions varied from 81.0% to 95.2%.
Correlation between the number of erosions observed and the actual number of erosions among the four sonographers.
Cohen's κ coefficients for interobserver reliability between two investigators in terms of discriminating between healthy (control) bones and bones with erosions
Scans of the ‘test box’ bones showed that the images were similar. The ultrasound detection limit of round erosion was approximately 1.5 mm with all ultrasound equipment. An erosion of a 1.0 mm width could be depicted but not clearly using transducer frequencies available in this work (figure 3).
Discussion
This study investigated the ability of ultrasound imaging to detect bone erosions in a phantom model. The PVA-C phantom model proved to be suitable for ultrasonographic scanning of bone erosions. The observers differentiated successfully between healthy (control) bones and bones with erosions. The observers also had significant correlations with the true number of erosions, and interobserver reliability ranged from moderate to almost perfect. The results showed that 1.0 mm deep erosions could be detected as accurately as a 4.0 mm deep erosion. Similar results were also obtained for the widths of the erosions, but the detection limit seemed to be 1.5 mm in round lesions. It is notable that bovine bones were rubbed smooth and normal bony irregularities were not present. It is thus possible that the validity and reliability figures obtained may be artificially a little too high. However, the current results still indicate that ultrasound is a valid and reliable method for detecting cortical bone erosions at least in the experimental work described. As far as we know, this is the first study to use a bone phantom model for investigating bone erosions and, more significantly, to compare the clinical ultrasound imaging results with the true numbers and dimensions of erosions.
The interobserver agreement between four musculoskeletal sonography experts was substantial, although there was some divergence between the observers. This could be caused by different settings of the ultrasound equipment, because each observer used their typical clinical scanning settings. Furthermore, the criteria to identify bone erosions may vary slightly among the operators. However, the true reasons for the slight divergence of the results among the observers remained unknown.
In the present study actual bovine bone was used in the phantoms. However, preservation of bone material requires controlled conditions in order to prevent contamination. Therefore, some artificial material could also be used as a replacement for bone phantom to keep the phantoms intact during long-term storage. However, in the case of artificial material it should be verified that the acoustic properties (speed of sound, ultrasound attenuation, elastic modulus and density) correspond to real bone. This would require a comprehensive experimental characterisation of the material in the laboratory. Therefore, in this approach we decided to use real bovine bone. However, the manufacturing and testing of artificial bone erosion phantoms would be an intriguing issue for further studies.
A notable limiting factor in ultrasound erosion scanning in vivo is inaccessibility in many joint regions, for example, the third and fourth metacarpophalangeal joints, wrist joints and metatarsophalangeal joints 2–4 in which the acoustic window for exploring the intra-articular bone surfaces is narrow. However, in the fifth metatarsophalangeal and second metacarpophalangeal joints, which are the important early targets in rheumatoid arthritis, ultrasonography is a sensitive imaging method for showing erosions.7 The ultrasound scanning of these joints resembles scanning with the phantom presented. Using a phantom makes it possible to examine the accuracy of the ultrasound imaging method in detecting erosions as the exact locations, number and sizes of erosions are known. This is not possible in vivo, which makes using other imaging modalities the only alternative, for example, CT or MRI, for reference, and they always raise the question of whether they are accurate enough. In comparison, the phantom approach presented provides a direct way to investigate the true accuracy of ultrasound imaging methods. Furthermore, this kind of phantom approach is also applicable to other imaging modalities, for example CT and MRI, provided that the properties of the soft tissue mimicking material correspond to the x-ray attenuation properties or magnetic properties.
The sensitivity of ultrasound in the detection of bone erosion not only depends on the investigator's experience in this field but also on the grey scale performance of the ultrasound equipment. This can be investigated with different types of phantoms such as a general purpose CIRS model 40—phantom (CIRS Inc, Norfolk, Virginia, USA). In this study a man-made ‘test box’ was constructed and used for the quality assurance measurements. The results showed that all three sets of equipment gave equal performances in the detection of erosions. This means that differences in scanning results between the investigators originated in the operator. Scientific papers should always carry out systematic quality assurance testing in order to verify the performance of the ultrasound equipment. In particular in clinical trials using multiple ultrasound machines, quality assurance is crucial to minimising interscanner variation and to achieving an accurate interpretation of the results.
In conclusion, by using a phantom model this study showed that ultrasound is a valid and reliable method for detecting cortical erosions on the bone when the erosion is at least 1.0 mm deep and 1.5 mm wide. There were also no clear differences between the performances of the sets of ultrasound equipment used, which implies that the differences in the scanning results were caused by inconsistencies between operators. These findings are clinically significant as they confirm that physicians can make an erosion diagnosis by using ultrasound scanning provided that the operator is experienced and that quality assurance measurements are performed on the ultrasound equipment used.
References
Footnotes
-
Funding Financial support was received from the Academy of Finland (project 127198, Kuopio, Finland) and the Ministry of Education, Finland, to the University of Eastern Finland (project 5741), Kuopio University Hospital (EVO grant).
-
Competing interests None.
-
Provenance and peer review Not commissioned; externally peer reviewed.