Ex) Article Title, Author, Keywords
pISSN 1598-298X
eISSN 2384-0749
Ex) Article Title, Author, Keywords
J Vet Clin 2022; 39(3): 114-120
https://doi.org/10.17555/jvc.2022.39.3.114
Published online June 30, 2022
Young-kwon Cho1 , Youngjin Lee2
, Kichang Lee3
Correspondence to:*kclee@jbnu.ac.kr
Copyright © The Korean Society of Veterinary Clinics.
Erratum: J Vet Clin 39(4): 197-197 (2022) https://doi.org/10.17555/jvc.2022.39.4.197
The purpose of this study is to examine the status of quality control using multipurpose phantom of ultrasound equipment used in hospital of veterinary college in South Korea by using ATS-539 multipurpose phantom so as to examine quantitative and objective new image evaluation method. Specialists discussed and analyzed multipurpose phantom images acquired by using convex transducer of 10 ultrasound imaging devices, currently used in 9 veterinary colleges, at 4.0-6.0 MHz. Total 8 items that can be measured with ATS-539 multipurpose phantom including dead zone, vertical and horizontal measurement, axial/lateral resolution, sensitivity, focal zone, functional resolution and gray scale/dynamic range were evaluated. For qualitative evaluation, valid decisions were made based on dead zone, axial/lateral resolution, and gray scale/dynamic range which are resolution index, and coefficient of variation (COV) and blind referenceless image spatial quality evaluator (BRISQUE) were found to increase objectivity. As a result of experiment, all the targeted ultrasonic devices were found appropriate from qualitative evaluation items of dead zone, axial/lateral resolution, and gray scale/dynamic range. In other evaluation items, they were found to be appropriate from focal zone and vertical measurement of quantitative evaluation while inappropriate from horizontal measurement, sensitivity, and functional resolution. COV value was 0.12 ± 0.04, and BRISQUE value was 47.77 ± 2.77, both analysis results show that the noise level of all ultrasonic devices was located within tolerance range. Upon image examination using ATS- 539 multipurpose phantom, they were 100% appropriate with inspection standards of dead zone, axial/lateral resolution, and gray scale/dynamic range, and besides, focal zone and functional resolution can be used as evaluation items. In the field of veterinary medicine, 8 standard items using ATS-539 multipurpose phantom and image evaluation items using COV and BRISQUE can be used as standards for quality control of ultrasonography machine.
Keywords: quality evaluation of ultrasonographic equipment, ATS-539 multipurpose phantom, COV, BRISQUE.
Ultrasonography in veterinary medicine is safer than other diagnostic modalities, and the devices develop quickly. Therefore, the importance of checkup emerges gradually and the distribution rate of equipment constantly increases. Using improper medical imaging and diagnostic devices may lower diagnostic quality increase repeated examination and may miss the period of treatment as the disease progresses. The biggest advantage of multipurpose phantom evaluation as an image evaluation for ultrasound imaging device is that it is able to acquire data about image diagnosis device easily without any restrictions of time and space. Also, when the multipurpose phantom image has poor image quality, it is expected that the resolution and contrast of clinical image would decline (4,5,15).
The quality control of convex transducer has been standardized with ATS-539 multipurpose phantom (ATS Laboratories, Bridgeport, CT, U.S.A.) (Fig. 1) and widely used for abdominal diagnosis (5,11,13). In the field of veterinary medicine, however, the inspection items and appropriate/inappropriate criteria for ultrasound standard phantom image evaluation have not yet been prepared, and ultrasound imaging devices currently used in Korea have not yet been evaluated.
As multipurpose phantom provides the most objective data until now (5,7), If the image quality assessment of ultrasonic device is made only by subjective opinion of the observer, the quality control can be inconsistent and varied upon observer’s experience and personnel opinion. In order to supplement this subjective point of view, various quantitative evaluation methods of ultrasound images are being used. Among quantitative evaluation methods, coefficient of variation (COV) is a representative parameter that can analyze the ratio of signal to noise and is widely used in basic medical image analysis. In addition, a parameter, blind referenceless image spatial quality evaluator (BRISQUE), that can analyze image quality has recently been used in various imaging fields, and it is also necessary to analyze its applicability in ultrasound images. We studied, therefore, added image evaluation method that uses COV and BRISQUE to supplement the limit of image evaluation using multipurpose phantom image of low objectivity.
This study aimed to prepare objective, minimum regulations of ultrasound imaging device and examine quantitative and objective new image evaluation method by ATS-539 multipurpose phantom of veterinary medical ultrasound imaging device that are currently used for diagnostic purpose.
Image evaluation was performed on 10 ultrasound imaging devices used by 9 veterinary colleges of Korea. ATS-539 multipurpose phantom was used for micro convex transducer at 4-6 MHz which is most widely used in the field of veterinary medicine.
We used setup value for abdominal examination which is the most widely used with monitor depth at 14cm. We examined 8 items of ATS-539 multipurpose phantom (10) and measured COV and BRISQUE through DICOM (Digital Imaging and Communications in Medicine) image acquired by using phantom according to the standard of measurement.
There are 8 image evaluation items of ultrasound standard multipurpose phantom such as dead zone, vertical and horizontal measurement, axial/lateral resolution, sensitivity, focal zone, functional resolution and gray scale/dynamic range. Among them, we decided dead zone, axial/lateral resolution and gray scale/dynamic range for qualitative evaluation and the remained 5 items for quantitative evaluation. We obtained sequential score according to the number of targets seen for functional resolution, and the suitability of 7 items was measured and evaluated on the basis of 0 in case of fit and -1 point in case of nonconformity (6,11).
Dead zone is the distance from the front of probe to the first identifiable echo between the probe and multipurpose phantom. It consists of total 9 targets, and we evaluated it appropriate when all 9 targets are separately seen by adjusting 5th target come to the center (Fig. 2a).
Axial/lateral resolution refers to the ability of distinction between two adjacent objects, and it can be divided into axial direction and outer direction. The resolution per sound beam axis is called axial direction resolution, and in this study, we decided it appropriate when 11 targets in vertical measurement image are seen clearly separated (Fig. 2b).
Vertical measurement refers to checking if the actual distance within a medium is correctly expressed along the axis of sound beam; there are 17 targets and the distance between targets is 1.0 ± 0.1 cm. We tried to use multi-focus as much as we could and locate target in the center of the screen so as to acquire the image, and measured 10 cm up to 1-11 cm. 10 ± 1 cm of error tolerance was used in the evaluation (Fig. 3a).
Horizontal measurement is a value vertically acquired to the axis of sound beam; in this study, we measured 8 cm lateral distance in vertical measurement image and evaluated it with 8 cm ± 4 mm of error tolerance (Fig. 3b).
Sensitivity is the depth from which round structures of 8 mm, 6 mm, 4 mm, and 2 mm sizes, which are anechoic targets, can be distinguished. Targets of 8 mm and 6 mm sizes are composed of 8 pieces each, and the other targets are composed of 17 pieces respectively. Acquiring image with targets of 8 mm size arranged in a row in the center, and the targets should be visible up to 5th target and the error tolerance of measuring distance was evaluated to be appropriate at 8.0 ± 0.5 cm (Fig. 4).
When focal zone strength and outer resolution are at maximum, the most accurate diagnosis information is provided to the area around the focus. We located one focus in 7 cm depth in vertical measurement image, measured lateral distance of a target in focus depth, measured lateral distance of another target 4 cm behind this target so as to see focusing rate of the targets (Fig. 5), and decided the focal rate 75% and above as appropriate (10). The focal rate (%) is defined with the following formula (11).
Functional resolution means the ability to express the size, shape and 146 depth of anechoic target in multipurpose phantom. The target arrangement is the same with the structure of sensitivity. The decision was made in the image acquired by using multifocus for 4 mm size target to be vertically located in the center, and the number of targets appropriately observed in 8 mm and 6 mm were scored (Table 1, Fig. 6).
Table 1 Functional resolution evaluation criterion
Functional resolution | Appropriate number of targets observed at 8 mm and 6 mm | target: 0, 7 target: –1, 6 target: –2, 5 target: –3, 4 target: –4, 3 target: –5, 2 target: –6, 1 target: –7, 0 target: –8 |
Appropriate number of targets observed at 4 mm, 3 mm, and 2 mm | 17 target: 0, 16 target: –1, 15 target: –2, 14 target: –3, 13 target: –4, 12 target: –5, 11 target: –6, 10 target: –7, 9 target: –8, 8 target: –9, 7 target: –10, 6 target: –11, 5 target: –12, 4 target: –13, 3 target: –14, 2 target: –15, 1 target: –16, 0 target: –17 |
Gray scale and Dynamic range means expressing the brightness of image differently according to the magnitude of received echo, and it expresses the brightness from the lowest gray scale level to the maximum brightness level by adjusting the echo signal. 6 targets are all seen in one screen and the center of target is adjusted to be on the center of the screen. Each target should be clearly distinguished and we decided it appropriate when the boundary line maintains more than 4 circles consecutively over 180° (Fig. 7) (11).
For COV, we measured COV value in ROI area set to evaluate noise level of acquired ultrasound image (Fig. 8). COV can be calculated with the average value of signal intensity and standard deviation ratio. dynamic range measurement image that can calculate COV the most appropriately was used and average value of 4-area data was drawn for each device.
BRISQUE value is able to analyze entire image no-reference basis, and the smaller the BRISQUE value is the better quality of the image is shown (14). We calculated the BRISQUE using the code provided by the MATLAB program.
Phantom image evaluation results of dead zone, vertical and horizontal measurement, axial/lateral resolution, sensitivity, focal zone and gray scale/dynamic range (Table 2).
Table 2 Phantom image evaluation result
User* | Dead zone (No) | Vertical measurement (cm) | Horizontal measurement (cm) | Focal zone (%) | Axial/lateral resolution (No) | Sensitivity (No) | Gray scale/dynamic range (No) |
---|---|---|---|---|---|---|---|
A | 9 | 10.3 | 8.9 | 10.3 | 11 | 4 | 4 |
B | 9 | 10.4 | 9.2 | 10.4 | 11 | 4 | 4 |
C | 9 | 10.5 | 8.9 | 10.5 | 11 | 4 | 4 |
D | 9 | 10.3 | 8.9 | 10.3 | 11 | 4 | 4 |
E | 9 | 10.1 | 8.9 | 10.1 | 11 | 3 | 4 |
Fa | 9 | 10.2 | 8.7 | 10.2 | 11 | 4 | 4 |
Fb | 9 | 10 | 8.5 | 10 | 11 | 4 | 4 |
G | 9 | 8.2 | 8.7 | 8.2 | 11 | 3 | 4 |
H | 9 | 10.4 | 9.4 | 10.4 | 11 | 4 | 4 |
I | 9 | 10.4 | 9.1 | 10.4 | 11 | 4 | 4 |
*University is A-I, University F has two equipment a and b.
All the 10 ultrasound imaging devices showed 9 targets separated which are included in the appropriate standard, and so all of them were evaluated with 0 point.
9 out of 10 ultrasound imaging devices were included in the 10.00 ± 1.00 cm of error tolerance, and so 9 devices were evaluated with 0 point, and the other 1 device did not show up to 11th depth which is not included in the appropriate standard being evaluated as –1 point. The average of appropriate 9 devices was 10.30 ± 0.10 cm.
All the 10 ultrasound imaging devices were above 8.40 cm and therefore evaluated as –1 point. The average was 8.90 ± 0.25 cm.
All the 10 devices showed 11 targets clearly separated which are included in the appropriate standard, and so all of them were evaluated with 0 point.
8 out of 10 devices had 6.90 ± 0.10 cm of average size up to 4 targets and the other 2 devices showed up to 3 targets with average 5.00 cm measured.
All the 10 devices were above 75% of error tolerance and so all of them were evaluated as 0 point. The lowest focal rate was 75% and the highest focal rate was 100%. Average of the 10 devices was 84.90%.
With the number of targets observed per size, the device with –2 point showed the highest functional resolution, and another device with –16 point represented the lowest functional resolution with average –9.60 ± 3.63.
All the 10 devices were observed with 4 targets being evaluated as 0 point.
As a result of quantitative analysis on COV, average 0.12 ± 0.04 value was drawn. Among 10 devices of 9 veterinary colleges in Korea, the most excellent COV value was 0.05 while the lowest COV value was measured to be 0.17 (Fig. 9a).
As a result of quantitative analysis, average 47.76 ± 2.78 value was drawn. Among 10 devices of 9 veterinary colleges in Korea, the most excellent BRISQUE value was 43.02 while the lowest BRISQUE value was measured to be 52.34 (Fig. 9b).
In this experiment, the image analysis was performed in both qualitative evaluation and quantitative evaluation. The qualitative evaluation was performed with dead zone, axial and lateral resolutions, and gray scale/dynamic range and the quantitative evaluation was performed with the remained vertical and the horizontal measurements, focal zone, sensitivity, and functional resolution.
In this experiment, all the 10 devices were evaluated to be appropriate in the qualitative evaluation. In the quantitative evaluation of 5 items, however, horizontal measurements showed –1 for all devices which means that the lesion is expected to be measured horizontally longer upon diagnosis in horizontal measurements item.
In the experiment, showing the scannable depth, 3 devices could only see 3 circular structures and, on average, 5 cm, so there was a limit to seeing deep organs. we think that it was caused by the penetrance relatively going down as the frequency domain going up. The frequency adjusts the depth from the patient’s surface (4,11). It does not only adjust the image contrast but also ultrasonic wavelength (9).
In this experiment, the average value was 84.93% which exceeds standard 75% showing high reliability on the focus per depth. The focal zone is where the strength and lateral direction resolution are shown the highest.
In the evaluation result with 8 items by using the existing multipurpose phantom image, the evaluation result with objective minimum regulations of dead zone, axial/lateral resolution, and gray scale/dynamic range was excellent.
The result of multipurpose phantom image can be different by the evaluation standard, and so it would be quite insufficient to evaluate ultrasonic device only with the existing multipurpose phantom image evaluation for accurate and objective evaluation standard.
This study, therefore, used COV and BRISQUE from various image evaluations so as to overcome the limit of subjectivity of image evaluation using multipurpose phantom and suggest quantitative image evaluation method.
The lower the COV and BRISQUE values the better the picture quality. COV and BRISQUE can be measured and evaluated in ultrasound imaging device, and so we measured the image signals along with multipurpose phantom image evaluation consequently evaluating devices more quantitative way.
In clinical thyroid ultrasonic image, COV value was about 0.25, and it was drawn to be about 0.20 even by applying the conventional noise reduction algorithm (2,12). Consequently, the noise level of all ultrasonic devices used by veterinary colleges is thought to be within allowable range for all clinical fields. Among 10 devices of 9 veterinary colleges, the best BRISQUE value was 43.02, and the worst BRISQUE value was measured to be 52.34. In particular, it was verified that the tendencies in COV result and BRISQUE result previously measured are matched.
This study basically used COV evaluation factor which is used the most widely upon evaluating noise level, and additionally applied recently developed BRISQUE for the first time which is an objective evaluation factor for entire image without reference image. BRISQUE has been approved of its accuracy by various researches (14), and it was applied to ultrasonic image for the first time in this study, and we will perform semantic analysis on the numerical values based on the result of this study.
Evaluating 8 items of multipurpose phantom image evaluation is insufficient method for correct ultrasonic device, therefor COV and BRISQUE measurement can be a supplementary method for such insufficiency.
In the field of veterinary medicine, 2.5-3.0 MHz of convex is actually used for large dogs and larger animals but for the smaller animals than those, 4-6 MHz of micro convex is widely used (1,3,8). Therefore, the existing convex with high penetrating power due to low frequency just needs to follow the multipurpose phantom image standard as used in the human medicine, but 4-6 MHz of micro convex, which are used for small and medium sized animals, needs new standard.
This study, therefore, aims to prepare new standard for the multipurpose phantom image evaluation for 4-6 MHz of micro convex probe, which is widely used for small and medium sized animals, and it is meaningful in performing more accurate and reliable diagnosis through systematic quality control of ultrasonic device.
The authors have no conflicting interests.
J Vet Clin 2022; 39(3): 114-120
Published online June 30, 2022 https://doi.org/10.17555/jvc.2022.39.3.114
Copyright © The Korean Society of Veterinary Clinics.
Young-kwon Cho1 , Youngjin Lee2
, Kichang Lee3
1Department of Radiologic Science, Cheongju University, Cheongju 28503, Korea
2Department of Radiologic Science, Gachon University, Incheon 21936, Korea
3Department of Veterinary Medicine, Jeonbuk National University, Iksan 54596, Korea
Correspondence to:*kclee@jbnu.ac.kr
This is an open access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
Erratum: J Vet Clin 39(4): 197-197 (2022) https://doi.org/10.17555/jvc.2022.39.4.197
The purpose of this study is to examine the status of quality control using multipurpose phantom of ultrasound equipment used in hospital of veterinary college in South Korea by using ATS-539 multipurpose phantom so as to examine quantitative and objective new image evaluation method. Specialists discussed and analyzed multipurpose phantom images acquired by using convex transducer of 10 ultrasound imaging devices, currently used in 9 veterinary colleges, at 4.0-6.0 MHz. Total 8 items that can be measured with ATS-539 multipurpose phantom including dead zone, vertical and horizontal measurement, axial/lateral resolution, sensitivity, focal zone, functional resolution and gray scale/dynamic range were evaluated. For qualitative evaluation, valid decisions were made based on dead zone, axial/lateral resolution, and gray scale/dynamic range which are resolution index, and coefficient of variation (COV) and blind referenceless image spatial quality evaluator (BRISQUE) were found to increase objectivity. As a result of experiment, all the targeted ultrasonic devices were found appropriate from qualitative evaluation items of dead zone, axial/lateral resolution, and gray scale/dynamic range. In other evaluation items, they were found to be appropriate from focal zone and vertical measurement of quantitative evaluation while inappropriate from horizontal measurement, sensitivity, and functional resolution. COV value was 0.12 ± 0.04, and BRISQUE value was 47.77 ± 2.77, both analysis results show that the noise level of all ultrasonic devices was located within tolerance range. Upon image examination using ATS- 539 multipurpose phantom, they were 100% appropriate with inspection standards of dead zone, axial/lateral resolution, and gray scale/dynamic range, and besides, focal zone and functional resolution can be used as evaluation items. In the field of veterinary medicine, 8 standard items using ATS-539 multipurpose phantom and image evaluation items using COV and BRISQUE can be used as standards for quality control of ultrasonography machine.
Keywords: quality evaluation of ultrasonographic equipment, ATS-539 multipurpose phantom, COV, BRISQUE.
Ultrasonography in veterinary medicine is safer than other diagnostic modalities, and the devices develop quickly. Therefore, the importance of checkup emerges gradually and the distribution rate of equipment constantly increases. Using improper medical imaging and diagnostic devices may lower diagnostic quality increase repeated examination and may miss the period of treatment as the disease progresses. The biggest advantage of multipurpose phantom evaluation as an image evaluation for ultrasound imaging device is that it is able to acquire data about image diagnosis device easily without any restrictions of time and space. Also, when the multipurpose phantom image has poor image quality, it is expected that the resolution and contrast of clinical image would decline (4,5,15).
The quality control of convex transducer has been standardized with ATS-539 multipurpose phantom (ATS Laboratories, Bridgeport, CT, U.S.A.) (Fig. 1) and widely used for abdominal diagnosis (5,11,13). In the field of veterinary medicine, however, the inspection items and appropriate/inappropriate criteria for ultrasound standard phantom image evaluation have not yet been prepared, and ultrasound imaging devices currently used in Korea have not yet been evaluated.
As multipurpose phantom provides the most objective data until now (5,7), If the image quality assessment of ultrasonic device is made only by subjective opinion of the observer, the quality control can be inconsistent and varied upon observer’s experience and personnel opinion. In order to supplement this subjective point of view, various quantitative evaluation methods of ultrasound images are being used. Among quantitative evaluation methods, coefficient of variation (COV) is a representative parameter that can analyze the ratio of signal to noise and is widely used in basic medical image analysis. In addition, a parameter, blind referenceless image spatial quality evaluator (BRISQUE), that can analyze image quality has recently been used in various imaging fields, and it is also necessary to analyze its applicability in ultrasound images. We studied, therefore, added image evaluation method that uses COV and BRISQUE to supplement the limit of image evaluation using multipurpose phantom image of low objectivity.
This study aimed to prepare objective, minimum regulations of ultrasound imaging device and examine quantitative and objective new image evaluation method by ATS-539 multipurpose phantom of veterinary medical ultrasound imaging device that are currently used for diagnostic purpose.
Image evaluation was performed on 10 ultrasound imaging devices used by 9 veterinary colleges of Korea. ATS-539 multipurpose phantom was used for micro convex transducer at 4-6 MHz which is most widely used in the field of veterinary medicine.
We used setup value for abdominal examination which is the most widely used with monitor depth at 14cm. We examined 8 items of ATS-539 multipurpose phantom (10) and measured COV and BRISQUE through DICOM (Digital Imaging and Communications in Medicine) image acquired by using phantom according to the standard of measurement.
There are 8 image evaluation items of ultrasound standard multipurpose phantom such as dead zone, vertical and horizontal measurement, axial/lateral resolution, sensitivity, focal zone, functional resolution and gray scale/dynamic range. Among them, we decided dead zone, axial/lateral resolution and gray scale/dynamic range for qualitative evaluation and the remained 5 items for quantitative evaluation. We obtained sequential score according to the number of targets seen for functional resolution, and the suitability of 7 items was measured and evaluated on the basis of 0 in case of fit and -1 point in case of nonconformity (6,11).
Dead zone is the distance from the front of probe to the first identifiable echo between the probe and multipurpose phantom. It consists of total 9 targets, and we evaluated it appropriate when all 9 targets are separately seen by adjusting 5th target come to the center (Fig. 2a).
Axial/lateral resolution refers to the ability of distinction between two adjacent objects, and it can be divided into axial direction and outer direction. The resolution per sound beam axis is called axial direction resolution, and in this study, we decided it appropriate when 11 targets in vertical measurement image are seen clearly separated (Fig. 2b).
Vertical measurement refers to checking if the actual distance within a medium is correctly expressed along the axis of sound beam; there are 17 targets and the distance between targets is 1.0 ± 0.1 cm. We tried to use multi-focus as much as we could and locate target in the center of the screen so as to acquire the image, and measured 10 cm up to 1-11 cm. 10 ± 1 cm of error tolerance was used in the evaluation (Fig. 3a).
Horizontal measurement is a value vertically acquired to the axis of sound beam; in this study, we measured 8 cm lateral distance in vertical measurement image and evaluated it with 8 cm ± 4 mm of error tolerance (Fig. 3b).
Sensitivity is the depth from which round structures of 8 mm, 6 mm, 4 mm, and 2 mm sizes, which are anechoic targets, can be distinguished. Targets of 8 mm and 6 mm sizes are composed of 8 pieces each, and the other targets are composed of 17 pieces respectively. Acquiring image with targets of 8 mm size arranged in a row in the center, and the targets should be visible up to 5th target and the error tolerance of measuring distance was evaluated to be appropriate at 8.0 ± 0.5 cm (Fig. 4).
When focal zone strength and outer resolution are at maximum, the most accurate diagnosis information is provided to the area around the focus. We located one focus in 7 cm depth in vertical measurement image, measured lateral distance of a target in focus depth, measured lateral distance of another target 4 cm behind this target so as to see focusing rate of the targets (Fig. 5), and decided the focal rate 75% and above as appropriate (10). The focal rate (%) is defined with the following formula (11).
Functional resolution means the ability to express the size, shape and 146 depth of anechoic target in multipurpose phantom. The target arrangement is the same with the structure of sensitivity. The decision was made in the image acquired by using multifocus for 4 mm size target to be vertically located in the center, and the number of targets appropriately observed in 8 mm and 6 mm were scored (Table 1, Fig. 6).
Table 1 . Functional resolution evaluation criterion.
Functional resolution | Appropriate number of targets observed at 8 mm and 6 mm | target: 0, 7 target: –1, 6 target: –2, 5 target: –3, 4 target: –4, 3 target: –5, 2 target: –6, 1 target: –7, 0 target: –8 |
Appropriate number of targets observed at 4 mm, 3 mm, and 2 mm | 17 target: 0, 16 target: –1, 15 target: –2, 14 target: –3, 13 target: –4, 12 target: –5, 11 target: –6, 10 target: –7, 9 target: –8, 8 target: –9, 7 target: –10, 6 target: –11, 5 target: –12, 4 target: –13, 3 target: –14, 2 target: –15, 1 target: –16, 0 target: –17 |
Gray scale and Dynamic range means expressing the brightness of image differently according to the magnitude of received echo, and it expresses the brightness from the lowest gray scale level to the maximum brightness level by adjusting the echo signal. 6 targets are all seen in one screen and the center of target is adjusted to be on the center of the screen. Each target should be clearly distinguished and we decided it appropriate when the boundary line maintains more than 4 circles consecutively over 180° (Fig. 7) (11).
For COV, we measured COV value in ROI area set to evaluate noise level of acquired ultrasound image (Fig. 8). COV can be calculated with the average value of signal intensity and standard deviation ratio. dynamic range measurement image that can calculate COV the most appropriately was used and average value of 4-area data was drawn for each device.
BRISQUE value is able to analyze entire image no-reference basis, and the smaller the BRISQUE value is the better quality of the image is shown (14). We calculated the BRISQUE using the code provided by the MATLAB program.
Phantom image evaluation results of dead zone, vertical and horizontal measurement, axial/lateral resolution, sensitivity, focal zone and gray scale/dynamic range (Table 2).
Table 2 . Phantom image evaluation result.
User* | Dead zone (No) | Vertical measurement (cm) | Horizontal measurement (cm) | Focal zone (%) | Axial/lateral resolution (No) | Sensitivity (No) | Gray scale/dynamic range (No) |
---|---|---|---|---|---|---|---|
A | 9 | 10.3 | 8.9 | 10.3 | 11 | 4 | 4 |
B | 9 | 10.4 | 9.2 | 10.4 | 11 | 4 | 4 |
C | 9 | 10.5 | 8.9 | 10.5 | 11 | 4 | 4 |
D | 9 | 10.3 | 8.9 | 10.3 | 11 | 4 | 4 |
E | 9 | 10.1 | 8.9 | 10.1 | 11 | 3 | 4 |
Fa | 9 | 10.2 | 8.7 | 10.2 | 11 | 4 | 4 |
Fb | 9 | 10 | 8.5 | 10 | 11 | 4 | 4 |
G | 9 | 8.2 | 8.7 | 8.2 | 11 | 3 | 4 |
H | 9 | 10.4 | 9.4 | 10.4 | 11 | 4 | 4 |
I | 9 | 10.4 | 9.1 | 10.4 | 11 | 4 | 4 |
*University is A-I, University F has two equipment a and b..
All the 10 ultrasound imaging devices showed 9 targets separated which are included in the appropriate standard, and so all of them were evaluated with 0 point.
9 out of 10 ultrasound imaging devices were included in the 10.00 ± 1.00 cm of error tolerance, and so 9 devices were evaluated with 0 point, and the other 1 device did not show up to 11th depth which is not included in the appropriate standard being evaluated as –1 point. The average of appropriate 9 devices was 10.30 ± 0.10 cm.
All the 10 ultrasound imaging devices were above 8.40 cm and therefore evaluated as –1 point. The average was 8.90 ± 0.25 cm.
All the 10 devices showed 11 targets clearly separated which are included in the appropriate standard, and so all of them were evaluated with 0 point.
8 out of 10 devices had 6.90 ± 0.10 cm of average size up to 4 targets and the other 2 devices showed up to 3 targets with average 5.00 cm measured.
All the 10 devices were above 75% of error tolerance and so all of them were evaluated as 0 point. The lowest focal rate was 75% and the highest focal rate was 100%. Average of the 10 devices was 84.90%.
With the number of targets observed per size, the device with –2 point showed the highest functional resolution, and another device with –16 point represented the lowest functional resolution with average –9.60 ± 3.63.
All the 10 devices were observed with 4 targets being evaluated as 0 point.
As a result of quantitative analysis on COV, average 0.12 ± 0.04 value was drawn. Among 10 devices of 9 veterinary colleges in Korea, the most excellent COV value was 0.05 while the lowest COV value was measured to be 0.17 (Fig. 9a).
As a result of quantitative analysis, average 47.76 ± 2.78 value was drawn. Among 10 devices of 9 veterinary colleges in Korea, the most excellent BRISQUE value was 43.02 while the lowest BRISQUE value was measured to be 52.34 (Fig. 9b).
In this experiment, the image analysis was performed in both qualitative evaluation and quantitative evaluation. The qualitative evaluation was performed with dead zone, axial and lateral resolutions, and gray scale/dynamic range and the quantitative evaluation was performed with the remained vertical and the horizontal measurements, focal zone, sensitivity, and functional resolution.
In this experiment, all the 10 devices were evaluated to be appropriate in the qualitative evaluation. In the quantitative evaluation of 5 items, however, horizontal measurements showed –1 for all devices which means that the lesion is expected to be measured horizontally longer upon diagnosis in horizontal measurements item.
In the experiment, showing the scannable depth, 3 devices could only see 3 circular structures and, on average, 5 cm, so there was a limit to seeing deep organs. we think that it was caused by the penetrance relatively going down as the frequency domain going up. The frequency adjusts the depth from the patient’s surface (4,11). It does not only adjust the image contrast but also ultrasonic wavelength (9).
In this experiment, the average value was 84.93% which exceeds standard 75% showing high reliability on the focus per depth. The focal zone is where the strength and lateral direction resolution are shown the highest.
In the evaluation result with 8 items by using the existing multipurpose phantom image, the evaluation result with objective minimum regulations of dead zone, axial/lateral resolution, and gray scale/dynamic range was excellent.
The result of multipurpose phantom image can be different by the evaluation standard, and so it would be quite insufficient to evaluate ultrasonic device only with the existing multipurpose phantom image evaluation for accurate and objective evaluation standard.
This study, therefore, used COV and BRISQUE from various image evaluations so as to overcome the limit of subjectivity of image evaluation using multipurpose phantom and suggest quantitative image evaluation method.
The lower the COV and BRISQUE values the better the picture quality. COV and BRISQUE can be measured and evaluated in ultrasound imaging device, and so we measured the image signals along with multipurpose phantom image evaluation consequently evaluating devices more quantitative way.
In clinical thyroid ultrasonic image, COV value was about 0.25, and it was drawn to be about 0.20 even by applying the conventional noise reduction algorithm (2,12). Consequently, the noise level of all ultrasonic devices used by veterinary colleges is thought to be within allowable range for all clinical fields. Among 10 devices of 9 veterinary colleges, the best BRISQUE value was 43.02, and the worst BRISQUE value was measured to be 52.34. In particular, it was verified that the tendencies in COV result and BRISQUE result previously measured are matched.
This study basically used COV evaluation factor which is used the most widely upon evaluating noise level, and additionally applied recently developed BRISQUE for the first time which is an objective evaluation factor for entire image without reference image. BRISQUE has been approved of its accuracy by various researches (14), and it was applied to ultrasonic image for the first time in this study, and we will perform semantic analysis on the numerical values based on the result of this study.
Evaluating 8 items of multipurpose phantom image evaluation is insufficient method for correct ultrasonic device, therefor COV and BRISQUE measurement can be a supplementary method for such insufficiency.
In the field of veterinary medicine, 2.5-3.0 MHz of convex is actually used for large dogs and larger animals but for the smaller animals than those, 4-6 MHz of micro convex is widely used (1,3,8). Therefore, the existing convex with high penetrating power due to low frequency just needs to follow the multipurpose phantom image standard as used in the human medicine, but 4-6 MHz of micro convex, which are used for small and medium sized animals, needs new standard.
This study, therefore, aims to prepare new standard for the multipurpose phantom image evaluation for 4-6 MHz of micro convex probe, which is widely used for small and medium sized animals, and it is meaningful in performing more accurate and reliable diagnosis through systematic quality control of ultrasonic device.
The authors have no conflicting interests.
Table 1 Functional resolution evaluation criterion
Functional resolution | Appropriate number of targets observed at 8 mm and 6 mm | target: 0, 7 target: –1, 6 target: –2, 5 target: –3, 4 target: –4, 3 target: –5, 2 target: –6, 1 target: –7, 0 target: –8 |
Appropriate number of targets observed at 4 mm, 3 mm, and 2 mm | 17 target: 0, 16 target: –1, 15 target: –2, 14 target: –3, 13 target: –4, 12 target: –5, 11 target: –6, 10 target: –7, 9 target: –8, 8 target: –9, 7 target: –10, 6 target: –11, 5 target: –12, 4 target: –13, 3 target: –14, 2 target: –15, 1 target: –16, 0 target: –17 |
Table 2 Phantom image evaluation result
User* | Dead zone (No) | Vertical measurement (cm) | Horizontal measurement (cm) | Focal zone (%) | Axial/lateral resolution (No) | Sensitivity (No) | Gray scale/dynamic range (No) |
---|---|---|---|---|---|---|---|
A | 9 | 10.3 | 8.9 | 10.3 | 11 | 4 | 4 |
B | 9 | 10.4 | 9.2 | 10.4 | 11 | 4 | 4 |
C | 9 | 10.5 | 8.9 | 10.5 | 11 | 4 | 4 |
D | 9 | 10.3 | 8.9 | 10.3 | 11 | 4 | 4 |
E | 9 | 10.1 | 8.9 | 10.1 | 11 | 3 | 4 |
Fa | 9 | 10.2 | 8.7 | 10.2 | 11 | 4 | 4 |
Fb | 9 | 10 | 8.5 | 10 | 11 | 4 | 4 |
G | 9 | 8.2 | 8.7 | 8.2 | 11 | 3 | 4 |
H | 9 | 10.4 | 9.4 | 10.4 | 11 | 4 | 4 |
I | 9 | 10.4 | 9.1 | 10.4 | 11 | 4 | 4 |
*University is A-I, University F has two equipment a and b.