검색
검색 팝업 닫기

Ex) Article Title, Author, Keywords

Article

J Vet Clin 2023; 40(4): 243-259

https://doi.org/10.17555/jvc.2023.40.4.243

Published online August 31, 2023

Scoping Review of Machine Learning and Deep Learning Algorithm Applications in Veterinary Clinics: Situation Analysis and Suggestions for Further Studies

Kyung-Duk Min*

College of Veterinary Medicine, Chungbuk National University, Cheongju 28644, Korea

Correspondence to:*kdmin@cbnu.ac.kr

Received: July 12, 2023; Revised: August 18, 2023; Accepted: August 21, 2023

Copyright © The Korean Society of Veterinary Clinics.

Machine learning and deep learning (ML/DL) algorithms have been successfully applied in medical practice. However, their application in veterinary medicine is relatively limited, possibly due to a lack in the quantity and quality of relevant research. Because the potential demands for ML/DL applications in veterinary clinics are significant, it is important to note the current gaps in the literature and explore the possible directions for advancement in this field. Thus, a scoping review was conducted as a situation analysis. We developed a search strategy following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. PubMed and Embase databases were used in the initial search. The identified items were screened based on predefined inclusion and exclusion criteria. Information regarding model development, quality of validation, and model performance was extracted from the included studies. The current review found 55 studies that passed the criteria. In terms of target animals, the number of studies on industrial animals was similar to that on companion animals. Quantitative scarcity of prediction studies (n = 11, including duplications) was revealed in both industrial and non-industrial animal studies compared to diagnostic studies (n = 45, including duplications). Qualitative limitations were also identified, especially regarding validation methodologies. Considering these gaps in the literature, future studies examining the prediction and validation processes, which employ a prospective and multi-center approach, are highly recommended. Veterinary practitioners should acknowledge the current limitations in this field and adopt a receptive and critical attitude towards these new technologies to avoid their abuse.

Keywords: machine learning, deep learning, veterinary clinics, scoping review, validation

The application of machine learning and deep learning (ML/DL) algorithms has altered the medical landscape. With respect to medical diagnostics, such as radiology and pathology, a growing number of studies have reported the reliable performance of ML/DL-based automatic systems (61) which are equivalent to or even better than those of human experts (40). Based on accumulated research, numerous commercialized medical devices have been officially approved for use in clinical practice, especially in countries such as Europe and the USA (49). Moreover, various digital biomarkers have been developed to predict the prognosis of chronic diseases such as cancer and cardiovascular diseases (46).

However, ML/DL applications in veterinary medicine seem to be far behind that in human medicine in terms of both quantity and quality, especially in South Korea. One of the major reasons for this slow progress can be the lack of high-quality medical data. Although most veterinary clinics use electronic medical chart systems (38), the analyzable data present in them is insufficient. Most veterinarians have not been educated and motivated regarding appropriate charting, especially in South Korea, where an insurance system for animals is lacking and purchasing some medical drugs is possible without a veterinarian’s prescription. Even if medical records are accumulated appropriately, merging multi-clinic medical records remains challenging owing to the lack of standardized medical coding classification systems (80).

However, the demand for ML/DL applications has increased with the rapid growth of the veterinary industry. Applications in industrial animal husbandry, disease screening, and medical data management have been developed, representing its growth potential (26). This growth could have positive implications, such as pioneering new markets and improving the quality of veterinary medical services. However, it could also increase the possibility of misuse and abuse, considering the challenges regarding data quality. Veterinarians who are practically involved in clinics should have a proper level of ML/DL literacy to prevent misuse and abuse and guide its development in a constructive manner.

Therefore, a scoping review was conducted to clarify current ML/DL applications in veterinary medicine and explore the directions for the advancement of this field. In this review, the application scope (specific domains in which the ML/DL methodology was applied), methodological details, and medical utility (Performance of the ML/DL models) of previously published relevant studies were investigated.

A scoping review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (53). Two electronic scientific literature databases (PubMed and Embase) were used to identify published studies that used ML/DL algorithms in veterinary clinics, especially those examining diagnostics and prognosis prediction. The search terms were developed (Supplementary Tables 1-2) based on two previous studies that examined artificial intelligence in the medical field (52) and veterinary medicine (62).

An initial search using these terms was conducted on September 22, 2022, and screening processes were implemented using predetermined inclusion/exclusion criteria. The inclusion criteria were as follows: 1) ML/DL applications in veterinary medicine focusing on diagnostics and prognosis prediction; 2) applications to companion animals, industrial animals, and/or wildlife; 3) studies written in English; and 4) original articles. The exclusion criteria were as follows: 1) applications to experimental animals; 2) population-level studies whose analysis unit was the aggregate level; 3) applications to animal husbandry or management (e.g., pregnancy detection); 4) applications to drug discovery; 5) applications to biosecurity; and 6) methodological studies that focused on algorithm development, optimization, or automatic data labelling. During the first screening process, the titles and abstracts of each searched paper were checked. In the second screening process, the full text was examined, and the final list of the included studies was determined.

For each included study the following information was extracted: 1) authors, publication year, and title; 2) purpose of study; 3) target animals; 4) number of samples used for ML/DL models; 5) algorithm types (e.g. artificial neural network, recurrent neural network [RNN], or other types); 6) whether the study used cross-validation framework to assess model performance; 7) whether the study collected test dataset prospectively or retrospectively; 8) whether the study used multi-center datasets for model building or validation; 9) measurements used for the performance of models (e.g., sensitivity, specificity, or other measurements); and 10) model performances.

The screening process for this scoping review is illustrated in Fig. 1. In the initial search, 598 and 376 studies were found in the PubMed and Embase databases, respectively. After removing duplicate studies, 699 studies were identified. A total of 532 and 112 papers were excluded after the first and second screenings, respectively. The remaining 55 studies were included. Detailed information on the included papers is provided in Tables 1, 2.

Table 1 General information regarding included studies in the review

Author and yearAnimal typeTarget animalsSample sizeAlgorithm
G. Theodoropoulos et al., 2000 (71)DomesticSheep255 images of 57 individual larvae (5genera)ANN (artificial neural network; feature selection by manual, 16 features were measured)
W. B. Roush et al., 2001 (63)DomesticChickenCase 6-40, normal 33-91BP3(back propagation neural network), WardBP (Ward back propagation neural network), PNN (Probabilistic neural network), GRNN (general regression neural network)
H. Schobesberger and C. Peham, 2002 (66)DomesticHorse175 (42 control/ 133 low to medium grade lame)ANN (feature selection by manual)
K. G. Keegan et al., 2003 (32)DomesticHorse12 adult horseANN (feature selection by manual)
M. E. Pastell and M. Kujalaf, 2007 (56)DomesticDairy cow73 cows (training 37 cows, 5,074 observation, validation 36 cows, 4,868 measurements)Probabilistic Neural Network Model (feature selection by manual)
S. M. Ghotoorlar et al., 2012 (25)DomesticDairy cow105 dairy cowsANN (feature selection by manual)
T. Banzato et al., 2018 (4)CompanionCanine80 (56 meningioma, 24 glioma)Convolutional neural networks (CNN), GoogleNet
T. Banzato et al., 2018 (5)CompanionCanine48 (32 case, 16 control)Deep neural networks (DNN), especially AlexNet
T. Banzato et al., 2018 (6)CompanionCanine56 (grade 1 = 26, grade 2 = 22, grade 3 = 8)AlexNet, DNN
A. Yakubu et al., 2018 (73)DomesticChicken167ANN
Y. Yoon et al., 2018 (75)CompanionDogs3,142 for cardiomegaly (1,571 normal and 1,571 abnormal from 1,143 dogs), 2,086 for lung pattern (1,043 normal and 1,043 abnormal from 1,247 dogs), 892 for mediastinal shift (446 normal and 446 abnormal from 387 dogs), 940 for pleural effusion (470 normal and 470 abnormal from 284 dogs), and 78 for pneumothorax (39 normal and 39 abnormal from 61 dogs)Bag-of-features (BOF) and CNN
R. Bradley et al., 2019 (15)CompanionCat106,251 catsRecurrent Neural Network (RNN)
M. Ebrahimi et al., 2019 (20)DomesticCow297,004 milking samples each with eight milking featuresANN, Naïve Bayes, GLM, Decision tree, Random forest, Gradient boosted tree
J. Y. Kim et al., 2019 (35)CompanionDogs1,040 imagesCNN (GoogLe net, Resnet, and VGGnet)
M. Aubreville et al., 2020 (3)CompanionDogs32 whole slide imagesCNN, RetinaNet, ResNet-18, Unet
V. Biourge et al., 2020 (12)CompanionCats218ANN
L. E. Broughton-Neiswanger et al., 2020 (16)CompanionCats12Partial least squares discriminant analysis, Random forest
S. Burti et al., 2020 (17)CompanionDogs1,465 imagesCNN
E. Fernández-Carrión et al., 2020 (22)Etc.Wild boar8CNN
M. A. Fraiwan and S. M. Abutarbush, 2020 (24)DomesticHorse285 horsesBayes Network, Naïve Bayes, DNN, Random forest
X. Kang et al., 2020 (30)DomesticCow100 cowsRFB_NET_SSD deep learning network
N. Kil et al., 2020 (33)DomesticHorse34 horses (65 video)CNN
S. Li et al., 2020 (39)CompanionDogs792 radiographsCNN
C. Marzahl et al., 2020 (42)DomesticHorse17 completely annotated cytology whole slide images (WSI) containing 78,047 hemosiderophagesCNN (RetinaNet)
S. Mouloodi et al., 2020 (47)DomesticHorse3 third metacarpal bones from 3 racehorsesANN
S. Mouloodi et al., 2020 (48)DomesticHorse9 equine third metacarpal bones from 9 thoroughbred horsesANN
Y. Nagamori et al., 2020 (50)CompanionCat, dogs100CNN
C. Post et al., 2020 (59)DomesticCow167 cowsLogistic Regression (LR), Support Vector Machine (SVM), K-nearest neighbors (KNN), Gaussian Naïve Bayes (GNB), Extra Trees Classifier (ET), Random forest
A. R. Trachtman et al., 2020 (72)DomesticPigs5,902 imagesCNN
T. Banzato et al., 2021 (7)CompanionDogs3,839 latero-lateral radiographsCNN (ResNet-50, DenseNet-121)
T. Banzato et al., 2021 (8)CompanionCat1,062 latero-lateral radiographsCNN (ResNet 50 and Inception V3)
A. Biercher et al., 2021 (11)CompanionDogsThoracolumbar MR images from 500 dogsCNN
E. Boissady et al., 2021 (13)CompanionCat, dogs30 canine and 30 feline thoracic lateral radiographsCNN
L. Bonicelli et al., 2021 (14)DomesticPigs7,564 picturesCNN
V. Kittichai et al., 2021 (36)DomesticPoultry12,761 single cell imagesCNN (Darknet, Darknet19, Darknet19-448 and Densenet201)
Y. Nagamori et al., 2021 (51)CompanionCat, dogs460 samples for 4 parasites (80-200 per parasite)You only look once (YOLOv3) model
J. Park et al., 2021 (54)CompanionDogs90 dogsHA, DLBAS, and the readjustment of the predicted data obtained via the DLBAS of the clinical test sets (HA_DLBAS)
I. R. Porter et al., 2021 (58)DomesticCattleA total of 398 digital images from dairy cows’ uddersCNN (GoogLeNet)
M. Salvi et al., 2021 (64)CompanionDogs416 canine cutaneous round cell tumors (RCT) (117 cases)AlexNet, Inceptionv3, ResNet, Emsemble
S. Shahinfar et al., 2021 (68)DomesticCattle2,535 lameness scores (2,248 sound and 287 unsound)Naïve Bayes (NB), Random Forest (RF) and Multilayer Perceptron (MLP), to predict cases of lameness using milk production and conformation traits logistc (LR)
Y. Ye et al., 2021 (74)CompanionDogs220 imagesCNN (ResNet-50)
M. Zhang et al., 2021 (79)CompanionDogs2,670 lateral X-ray imagesCNN (HRNet)
A.N. ELKhamary et al., 2022 (21)DomesticHorse16 horse 32 limbs (16 normal tendons and 16 abnormal tendons)C4.5 algorithm (Quinlan), a decision tree classifier of Weka software package
E. A. Bauer and W. Jagusiak, 2022 (9)DomesticCattle168 cowsANN
K. Benfodil et al., 2022 (10)DomesticDromedaries115 dromedariesANN
L. Dumortier et al., 2022 (19)CompanionCat500 annotated Thoracic radiograph images(348 veterinary visit 296 cats)CNN (ResNet50V2)
P. Figueirinhas et al., 2022 (23)CompanionDogs15 working dogs (pilot study)LSTM
Y. Kokkinos et al., 2022 (37)CompanionDogs57,402 dogsRNN
A. Mao et al., 2022 (41)DomesticChicken5,336 voice calls (3,363 distress calls and 1,973 natural barn sound)CNN (light-VGG11)
A. May et al., 2022 (43)DomesticHorse2,607 imagesCNN
T. R. Müller et al., 2022 (45)CompanionDogs62 canine (41 case 21 control) 4,000 images (2,000 case 2,000 control)CNN (VGG16)
C. Parra et al., 2022 (55)Etc.Reptile3,616 images data samples and 26 videos (4,849 frames)CNN (MobileNet)
T. Rai et al., 2022 (60)CompanionDogs32 patientsCNN (DenseNet-161)
V. A. Teixeira et al., 2022 (70)DomesticCattle55 Holstein calvesRNN
M. ZareBidaki et al., 2022 (77)DomesticGoat, sheep cows200 paired sample (100 blood, 100 milk) 100 animalsANN

Table 2 Validation methodologies and model performance of the included studies in the review

Author and yearCVProspectiveMulti-center approachModel performancePurpose


Training setTest setIndexValue
G. Theodoropoulos et al., 2000 (71)YesNoNoNoSensitivity42.4-80.7%Diagnostics
W. B. Roush et al., 2001 (63)YesNoNoNoSensitivity0-100%Prediction
H. Schobesberger and C. Peham, 2002 (66)YesNoNoNoAgreement78.60%Diagnostics
K. G. Keegan et al., 2003 (32)YesNoNoNoAgreement85%Diagnostics
M. E. Pastell and M. Kujalaf, 2007 (56)YesNoNoNoAgreement and sensitivityAgreement = 96.2%
Sensitivity = 100%
Diagnostics
S. M. Ghotoorlar et al., 2012 (25)YesNoNoNoSensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), Pearson correlation coefficientSensitivity = 0.5-1Specificity = 0.91-1PPV = 0.76-1NPV = 0.92 -1Pearson correlation coefficient = 0.94Diagnostics
T. Banzato et al., 2018 (4)YesNoYesNoAgreement, Matthews correlation coefficient (MCC)Agreement = 90-94%
MCC = 0.8-0.88
Diagnostics
T. Banzato et al., 2018 (5)YesNoNoNoAUC, sensitivity, specificityAUC = 0.91
Sensitivity = 100%
Specificity = 82.8%
Diagnostics
T. Banzato et al., 2018 (6)YesNoYesNoAgreement, multi-class Matthew’s correlation coefficient (MCMCC)Agreement = 65.2-82.2%
MCMCC = 0.44-0.68
Diagnostics
A. Yakubu et al., 2018 (73)YesNoNoNor, R2, RMSEr = 0.983
R2 = 0.966
RMSE = 0.04806
Prediction
Y. Yoon et al., 2018 (75)YesNoNoNoAccuracy, sensitivityAccuracy(CNN; 92.9-96.9% and BOF; 79.6-96.9%) and sensitivity (CNN; 92.1-100% and BOF; 74.1-94.8%)Prediction
R. Bradley et al., 2019 (15)YesNoNoNoSensitivity, specificity(1 year before) sensitivity 63.0%; (2 year before) sensitivity 44.2% specificity remaining around 99%Prediction
M. Ebrahimi et al., 2019 (20)YesNoNoNoAUC0.826Prediction
J. Y. Kim et al., 2019 (35)YesNoYesNoSensitivity79.4-100%Diagnostics
M. Aubreville et al., 2020 (3)YesNoNoNoCorrelation coefficient0.868-0.979Diagnostics
V. Biourge et al., 2020 (12)YesYesNoYesAccuracy, sensitivity, specificity, PPV, NPVAccuracy = 88%
Sensitivity = 87%
Specificity = 70%
PPV = 53%
NPV = 92%
Prediction
L. E. Broughton-Neiswanger et al., 2020 (16)YesNoNoNoSensitivity, specificity, AUCAUC = 0.87-1Sensitivity = 0-100%Specificity = 50-100%Diagnostics
S. Burti et al., 2020 (17)YesNoNoNoAUC0.904-0.973Diagnostics
E. Fernández-Carrión et al., 2020 (22)YesNoNoNoAgreement95.4-97.2%Diagnostics
M. A. Fraiwan and S. M. Abutarbush, 2020 (24)YesNoNoNoPrecision, recall, F-measure, Accuracy(need for surgery)Precision = 69.5-74.1%Recall = 72.4-99.3%F-measure = 72.2-81.8%Accuracy = 69.0-76.0%(survival)Precision = 87.5-97.4%Recall = 80.5-87.8%F-measure = 87.2-89.1%Accuracy = 83.9-85.2%Prediction
X. Kang et al., 2020 (30)YesNoNoNoSensitivity, specificitySensitivity = 0.83-1Specificity = 0.95-1Diagnostics
N. Kil et al., 2020 (33)YesNoNoNoSensitivity, accuracySensitivity = 0.79-0.94Accuracy = 0.82-0.94Diagnostics
S. Li et al., 2020 (39)YesNoNoNoAccuracy, sensitivity, and specificityAccuracy = 82.71%
Sensitivity = 68.42%
Specificity = 87.09%
Diagnostics
C. Marzahl et al., 2020 (42)YesNoNoNoPrecision0.64-0.66Diagnostics
S. Mouloodi et al., 2020 (47)YesNoNoNoDetermination coefficient (R2)0.9116-0.9599Prediction
S. Mouloodi et al., 2020 (48)YesNoNoNoDetermination coefficient (R2)0.9999Prediction
Y. Nagamori et al., 2020 (50)YesNoNoNoPearson correlation coefficient, sensitivity, specificityPearson correlation coefficient = 0.89-0.99Sensitivity = 0.758-1Specificity = 0.918-1Diagnostics
C. Post et al., 2020 (59)YesNoNoNoAUC0.71-0.79Diagnostics
A. R. Trachtman et al., 2020 (72)YesNoNoNoAccuracy, sensitivity, specificityAccuracy = 62-96%Sensitivity = 84-100%Specificity = 92-96%Diagnostics
T. Banzato et al., 2021 (7)YesNoNoNoAUC0.8Diagnostics
T. Banzato et al., 2021 (8)YesNoYesNoAUC0.58-0.97Diagnostics
A. Biercher et al., 2021 (11)YesNoYesYesSensitivity, specificityIVDE sens 73.46-90.1/spec 67.6-99.0IVDP sens 67.86-100/spec 74.9-96.4FCE/ANNPE sens 62.2-90.1/spec 90.1-97.9Syringomyelia sens 0-10/spec 100Neoplasma sens 0-37.5/spec 60-94.7Diagnostics
E. Boissady et al., 2021 (13)NANoNoNoICC0.998-0.999Diagnostics
L. Bonicelli et al., 2021 (14)YesNoYesYesSensitivity, specificity, Pearson correlation coefficientSensitivity = 81.25-100 %
Specificity = 99.38 %
Pearson correlation coefficient = 0.96
Diagnostics
V. Kittichai et al., 2021 (36)YesNoNANAAccuracy99%Dignostics
Y. Nagamori et al., 2021 (51)NANAYESNASensitivity, specificitySensitivity = 75.8-100%
Specificity = 93.1-100%
Dignostics
J. Park et al., 2021 (54)YesNoNoNoDice similarity coefficient (DSC) and the Hausdorff distance (HD)DSC 0.78-0.94
HD 2.30-4.30 mm
Dignostics
I. R. Porter et al., 2021 (58)YesNoYesYesAUC0.542-0.920Dignostics
M. Salvi et al., 2021 (64)YesNoYesYesAccuracy91.66%-100%Dignostics
S. Shahinfar et al., 2021 (68)YesNoYesYesAUC, F1AUC = 0.61-0.67
F1 = 0.01-0.27
Dignostics
Y. Ye et al., 2021 (74)YesNoNANAAUC, accuracy, F1 scoreAUC = 99.37Accuracy = 97.62 F1 score = 96.7Dignostics
M. Zhang et al., 2021 (79)YesNoYesYesSensitivity86.40%Dignostics
A.N. ELKhamary et al., 2022 (21)YesNoNoNoAccuracy, PPV, sensitivity, kappaAccuracy = 93.7%
PPV = 93.80%
Sensitivity = 93.80%
Kappa = 0.88
Dignostics
E. A. Bauer and W. Jagusiak, 2022 (9)YesNoYESYESAUC0.82-0.89Dignostics
K. Benfodil et al., 2022 (10)YesNoNANAPearson correlation coefficient0.943Dignostics
L. Dumortier et al., 2022 (19)YesNoNoNoAccuracy, F1-Score, Specificity, Positive Predictive Value and SensitivityAccuracy = 82%
F1-Score = 85%
Specificity = 75%
PPV = 81%
Sensitivity = 88%
Dignostics
P. Figueirinhas et al., 2022 (23)YesNoNoNoAccuracyAccuracy = 60%Dignostics
Y. Kokkinos et al., 2022 (37)YesNoNoNoSensitivity, PPV, NPVSensitivity = 44.8-68.8% PPV = 15-23% NPV > 99%Prediction
A. Mao et al., 2022 (41)YesNoYesYesPrecision, recall, F1-score and accuracyPrecision = 94.58%
Recall = 94.89%
F1-score = 94.73%
Accuracy = 95.07%
Dignostics
A. May et al., 2022 (43)YesNoNoNoAccuracy, cross entropyAccuracy = 96.66%
Cross entropy = 0.02
Dignostics
T. R. Müller et al., 2022 (45)YesNoNoNoAccuracy, sens, spec, PPV, NPVAccuracy 88.7%
Sensitivity 90.2%
Specificity 81.8%
PPV 92.5%
NPV 81.8%
Dignostics
C. Parra et al., 2022 (55)YesNANANAAccuracy, AUCAccuracy = 94.26
AUC = 0.996
Dignostics
T. Rai et al., 2022 (60)YesNoNoNoF1-score0.708Dignostics
V. A. Teixeira et al., 2022 (70)YesNoNoNoAccuracy, sensitivity, and specificity, PPV, NPVAccuracy = 85-98,
Sensitivity = 87-96
Specificity = 78-100
PPV = 85-100
NPV = 88-96
Prediction & diagnosis
M. ZareBidaki et al., 2022 (77)YesNoNANASensitivity, specificity, AUCSensitivity = 81%
Specificity = 62%
AUC = 0.799
Dignostics

Figure 1.Flow diagram of information through the different phases of the review.

The temporal trends in ML/DL-related publications are illustrated in Fig. 2. Although most of these studies were published after 2000, a rapid growth in their quantity began in 2018. Before this surge, the applications of ML/DL were concentrated in industrial animals; however, their applications in companion animals have been expanding since 2018. Only a few studies on other animal species (wildlife and exotic animals) have been published, even after 2020.

Figure 2.Temporal trend of machine learning and deep learning application studies in veterinary clinics by types of target animals. Note: The others group includes wildlife and exotic animals.

Fig. 3 shows the proportion of the specific purposes of each study, such as target animal species and domains of application (whether ML/DL was used for predictive or diagnostic purposes). While the number of studies for both industrial and non-industrial animals was similar (31 and 30 for non-industrial and industrial animals, respectively, including duplicates), the number of diagnostic studies was higher than that of prediction studies (the number of diagnostic and prediction studies were 45 and 11, respectively, including duplicates). In terms of specific animal species, studies on dogs were generally dominant among studies on non-industrial animals (70.3% of diagnostic studies and 50.0% of prediction studies), while studies on cows (39.1% of diagnostic studies and 28.6% of prediction studies) and horses (26.1% of diagnostic studies and 42.9% of prediction studies) were dominant among studies on industrial animals.

Figure 3.Proportion of target animal species by the purpose of studies. Note: Numbers include duplicates. For example, a study on industrial animals has the purposes of both diagnostics and prediction.

Table 3 shows details regarding the identified studies, including the sample size used for model development and validation, the algorithm used, whether the authors employed prospective data collection for validation, whether they used multi-center data for model development and validation, and model performance. In terms of validation, almost every publication stated that they implemented cross-validation (splitting data into training and test sets to avoid over-evaluation), although there was an insufficiency in the relevant descriptions in some of the studies (n = 2). However, a minority of the studies employed a multi-center approach for model development (n = 13) and validation (n = 9), and only one study prospectively collected the test datasets. The majority of the identified studies used neural network-based algorithms, such as RNN and convolutional neural network, and most of the studies targeted binary problems rather than continuous outcomes. Although the numbers of data that used for model development are relatively small for several studies (16,22,33), the reported model performance of most studies tended to be within an acceptable range (e.g., Area Under the Receiver Operating Characteristic Curve (AUC) value >0.9).

Table 3 Profile of included studies

Target animal typeStudy purposesN*NNCVPros§Multi∥
Industrial animalsDiagnostics21192105
Prediction77700
Companion animalsDiagnostics22212002
Prediction44411
OthersDiagnostics22200

*Number of studies.

Number of studies that used neural network-based algorithm.

Number of studies that conducted cross-validation approach to measure performance.

§Number of studies that employed prospective approach for collecting dataset for testing.

Number of studies that used multi-center data for validation.

The others group includes wildlife and exotic animals.

Note: The numbers include duplication. For example, a study for industrial animals have both purpose, diagnostics and prediction. There is no prediction studies for the other animals.


A scoping review was conducted as a situation analysis to identify the current gaps in ML/DL application research in veterinary clinics and suggest directions for further improvement in this field. The review found that the history of ML/DL applications in veterinary medicine is relatively short compared to that in human medicine and the healthcare sector (31). Possibly due to its short history, quantitative scarcity and methodological gaps were identified, especially regarding the validation and data collection framework, although the reported model performance was generally within acceptable levels.

The first gap that must be highlighted is quantitative scarcity. Although there is a possibility that the current review will exclude published papers, it seems clear that the relevant papers are fewer than those in the human medical field (2,52,67,69). Specifically, prediction studies were scarce, possibly because of their technical difficulties. They usually include extrapolation because the prediction target is future data. Considering that extrapolation is more sensitive to overfitting and a lack of variables, the performance of the model tends to be lower than that of the models for interpolation (57). However, prediction studies are practically useful because they can be employed for optimal treatment recommendations and prognostic assessment, which are the most frequent practices in veterinary clinics. Purification has also been observed in studies on wildlife. Lack of data may explain this discrepancy. Compared with medicine for companion and industrial animals, wildlife medicine covers more animal species with less resources. Therefore, the quantity of data for each species is usually lower than that for other medical areas, even though large amounts of data regarding specific species and medical problems are required for ML/DL applications.

Qualitative gaps in model validation should be emphasized. Considering that ML/DL approaches cannot inherently employ physiological or pathological mechanisms, an innate limitation of this data-driven approach is overfitting and induction. The issues can be practically addressed by demonstrating acceptable performance in an independent dataset, which is called cross-validation. Most of the studies identified in this review employed this approach. However, the current review found that only a few of them have obtained appropriate test sets. As the selection of the test set is essential for its validation, the representativeness of the test set must be ensured (27). Therefore, prospective data collection from multiple centers is the best way to ensure this representativeness (34,78). Veterinary clinicians should be aware of the qualitative gaps in current ML/DL application studies to avoid possible misuse of these models in clinical practice.

From the veterinary clinicians’ point of view, excellent model performance alone is not sufficient to recommend its practical use. For instance, even if some ML/DL models show very high AUC, representing great performance in diagnostics, the operation of the model could require a significant amount of manpower, time, or cost, making its usage unaffordable, especially for single-veterinarian clinics. In this regard, successful future studies need to consider the practical applicability as well (29).

Despite these gaps, there are prominent opportunities to improve research on ML/DL applications in veterinary medicine. First, privacy issues are relatively minor, when compared with human medicine. In it, data merging between hospitals and clinics is challenging owing to these issues. Therefore, the major approach in human medicine is the common data model which standardizes the data structure of each institution, facilitating meta-analysis (1,76) rather than merged big data analysis. In contrast, multi-clinic data can be merged without privacy issues in veterinary sectors, and the veterinary compass (44) and Small Animal Veterinary Surveillance Network (28,65) showed these opportunities. Furthermore, the cost of data collection in veterinary medicine, especially for continuous data, may be lower than that in human medicine. Recently, the collection of continuous data and extraction of significant signals using wearable devices (18) has become a leading research topic. In these research areas, veterinary medicine has more opportunities than in human medicine, because employing animal subjects costs less than employing human participants; additionally, compliance in applying the device could be higher in animal subjects than in human participants.

Improving the application of ML/DL in veterinary clinics necessitates the fulfillment of two essential conditions. First, the establishment of a standardized encoding system is crucial. To achieve reliable prediction performance, high-quality big data is indispensable. Considering that the medical big data should be collected by multiple institutions, a unified coding system for diseases diagnosis and prescription is essential to successfully amalgamate data from various sources. However, currently, medical records predominantly rely on free text-based descriptions which is challenging to be standardized. Although automatic encoding systems that translate free text to medical codes have been developed (78), no system is customized currently. Secondly, fostering sustainable motivation among veterinarians for accurate recording is important. The absence of a national insurance system for animal medicine has led to a lack of incentives for veterinarians to ensure precise encoding. Addressing this challenge entails appropriately valuating medical records provided by veterinary clinicians. Currently, the value of such data is not accurately evaluated, and most data utilized in ML/DL models have been acquired without enough compensation to veterinarians. Offering proper remuneration for their data contributions could incentivize them to maintain accurate recording practices (Fig. 4).

Figure 4.Current gaps and suggestions for further studies in the studies using machine learning and deep learning in veterinary clinics.

This study has some limitations. First, the reviews were conducted by a single researcher. Because the standard review process generally requires at least two researchers to increase the sensitivity and specificity of the screening process, several studies, that should have been included, could have been excluded. Second, this study included only original papers and other types of publications were excluded. Because studies on state-of-the-art methodologies can be published as conference abstracts, several studies may not have been reviewed in this study. Although this preliminary review study successfully revealed current gaps especially for validation methodologies, further studies are highly recommended to address the limitation, confirm the gaps and support the suggestions in this study. The follow-up studies should employ standard review process with at least two independent researchers and include grey articles that report up-to-date technologies.

In this review, I examined studies that covered the application of ML/DL in veterinary clinics. This revealed several gaps in the methodology and validation, that could help future studies improve their quality and allow readers to better screen appropriate veterinary studies. In the era of artificial intelligence, the expanding demand for their application in veterinary clinics is unavoidable. Furthermore, demand-driven active research using proper methodologies can fundamentally improve clinical services. In this regard, researchers should keep practical feasibility in mind when tackling methodology and model performance; moreover, veterinary clinicians should adopt a receptive and critical stance towards these new changes.

This work was supported by a funding for the academic research program of Chungbuk National University in 2022. In addition, this work was carried out with the support of “Cooperative Research Program for Agriculture Science and Technology Development (Project No. RS-2023-00232301).“ Rural Development Administration, Republic of Korea.

  1. Ahmadi N, Peng Y, Wolfien M, Zoch M, Sedlmayr M. OMOP CDM can facilitate data-driven studies for cancer prediction: a systematic review. Int J Mol Sci 2022; 23: 11834.
    Pubmed KoreaMed CrossRef
  2. Ali O, Abdelbaki W, Shrestha A, Elbasi E, Alryalat MAA, Dwivedi YK. A systematic literature review of artificial intelligence in the healthcare sector: benefits, challenges, methodologies, and functionalities. J Innov Knowl 2023; 8: 100333.
    CrossRef
  3. Aubreville M, Bertram CA, Marzahl C, Gurtner C, Dettwiler M, Schmidt A, et al. Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region. Sci Rep 2020; 10: 16447.
    Pubmed KoreaMed CrossRef
  4. Banzato T, Bernardini M, Cherubini GB, Zotti A. A methodological approach for deep learning to distinguish between meningiomas and gliomas on canine MR-images. BMC Vet Res 2018; 14: 317.
    Pubmed KoreaMed CrossRef
  5. Banzato T, Bonsembiante F, Aresu L, Gelain ME, Burti S, Zotti A. Use of transfer learning to detect diffuse degenerative hepatic diseases from ultrasound images in dogs: a methodological study. Vet J 2018; 233: 35-40.
    Pubmed CrossRef
  6. Banzato T, Cherubini GB, Atzori M, Zotti A. Development of a deep convolutional neural network to predict grading of canine meningiomas from magnetic resonance images. Vet J 2018; 235: 90-92.
    Pubmed CrossRef
  7. Banzato T, Wodzinski M, Burti S, Osti VL, Rossoni V, Atzori M, et al. Automatic classification of canine thoracic radiographs using deep learning. Sci Rep 2021; 11: 3964.
    Pubmed KoreaMed CrossRef
  8. Banzato T, Wodzinski M, Tauceri F, Donà C, Scavazza F, Müller H, et al. An AI-based algorithm for the automatic classification of thoracic radiographs in cats. Front Vet Sci 2021; 8: 731936.
    Pubmed KoreaMed CrossRef
  9. Bauer EA, Jagusiak W. The use of multilayer perceptron artificial neural networks to detect dairy cows at risk of ketosis. Animals (Basel) 2022; 12: 332.
    Pubmed KoreaMed CrossRef
  10. Benfodil K, Benbouras MA, Ansel S, Mohamed-Cherif A, Ait-Oudhia K. Prediction of Trypanosoma evansi infection in dromedaries using artificial neural network (ANN). Vet Parasitol 2022; 306: 109716.
    Pubmed CrossRef
  11. Biercher A, Meller S, Wendt J, Caspari N, Schmidt-Mosig J, De Decker S, et al. Using deep learning to detect spinal cord diseases on thoracolumbar magnetic resonance images of dogs. Front Vet Sci 2021; 8: 721167.
    Pubmed KoreaMed CrossRef
  12. Biourge V, Delmotte S, Feugier A, Bradley R, McAllister M, Elliott J. An artificial neural network-based model to predict chronic kidney disease in aged cats. J Vet Intern Med 2020; 34: 1920-1931.
    Pubmed KoreaMed CrossRef
  13. Boissady E, De La Comble A, Zhu X, Abbott J, Adrien-Maxence H. Comparison of a deep learning algorithm vs. humans for vertebral heart scale measurements in cats and dogs shows a high degree of agreement among readers. Front Vet Sci 2021; 8: 764570.
    Pubmed KoreaMed CrossRef
  14. Bonicelli L, Trachtman AR, Rosamilia A, Liuzzo G, Hattab J, Mira Alcaraz E, et al. Training convolutional neural networks to score pneumonia in slaughtered pigs. Animals (Basel) 2021; 11: 3290.
    Pubmed KoreaMed CrossRef
  15. Bradley R, Tagkopoulos I, Kim M, Kokkinos Y, Panagiotakos T, Kennedy J, et al. Predicting early risk of chronic kidney disease in cats using routine clinical laboratory tests and machine learning. J Vet Intern Med 2019; 33: 2644-2656.
    Pubmed KoreaMed CrossRef
  16. Broughton-Neiswanger LE, Rivera-Velez SM, Suarez MA, Slovak JE, Piñeyro PE, Hwang JK, et al. Urinary chemical fingerprint left behind by repeated NSAID administration: discovery of putative biomarkers using artificial intelligence. PLoS One 2020; 15: e0228989.
    Pubmed KoreaMed CrossRef
  17. Burti S, Longhin Osti V, Zotti A, Banzato T. Use of deep learning to detect cardiomegaly on thoracic radiographs in dogs. Vet J 2020; 262: 105505.
    Pubmed CrossRef
  18. Dinh-Le C, Chuang R, Chokshi S, Mann D. Wearable health technology and electronic health record integration: scoping review and future directions. JMIR Mhealth Uhealth 2019; 7: e12861.
    Pubmed KoreaMed CrossRef
  19. Dumortier L, Guépin F, Delignette-Muller ML, Boulocher C, Grenier T. Deep learning in veterinary medicine, an approach based on CNN to detect pulmonary abnormalities from lateral thoracic radiographs in cats. Sci Rep 2022; 12: 11418.
    Pubmed KoreaMed CrossRef
  20. Ebrahimi M, Mohammadi-Dehcheshmeh M, Ebrahimie E, Petrovski KR. Comprehensive analysis of machine learning models for prediction of sub-clinical mastitis: deep learning and gradient-boosted trees outperform other models. Comput Biol Med 2019; 114: 103456.
    Pubmed CrossRef
  21. ELKhamary AN, Keenihan EK, Schnabel LV, Redding WR, Schumacher J. Leveraging MRI characterization of longitudinal tears of the deep digital flexor tendon in horses using machine learning. Vet Radiol Ultrasound 2022; 63: 580-592.
    Pubmed CrossRef
  22. Fernández-Carrión E, Barasona JÁ, Sánchez Á, Jurado C, Cadenas-Fernández E, Sánchez-Vizcaíno JM. Computer vision applied to detect lethargy through animal motion monitoring: a trial on African swine fever in wild boar. Animals (Basel) 2020; 10: 2241.
    Pubmed KoreaMed CrossRef
  23. Figueirinhas P, Sanchez A, Rodríguez O, Vilar JM, Rodríguez-Altónaga J, Gonzalo-Orden JM, et al. Development of an artificial neural network for the detection of supporting hindlimb lameness: a pilot study in working dogs. Animals (Basel) 2022; 12: 1755.
    Pubmed KoreaMed CrossRef
  24. Fraiwan MA, Abutarbush SM. Using artificial intelligence to predict survivability likelihood and need for surgery in horses presented with acute abdomen (Colic). J Equine Vet Sci 2020; 90: 102973.
    Pubmed CrossRef
  25. Ghotoorlar SM, Ghamsari SM, Nowrouzian I, Ghotoorlar SM, Ghidary SS. Lameness scoring system for dairy cows using force plates and artificial intelligence. Vet Rec 2012; 170: 126.
    Pubmed CrossRef
  26. Hennessey E, DiFazio M, Hennessey R, Cassel N. Artificial intelligence in veterinary diagnostic imaging: a literature review. Vet Radiol Ultrasound 2022; 63 Suppl 1: 851-870.
    Pubmed CrossRef
  27. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2019; 2: e191095. Erratum in: JAMA Netw Open 2019; 2: e193260.
    Pubmed KoreaMed CrossRef
  28. Jones PH, Dawson S, Gaskell RM, Coyne KP, Tierney A, Setzkorn C, et al. Surveillance of diarrhoea in small animal practice through the Small Animal Veterinary Surveillance Network (SAVSNET). Vet J 2014; 201: 412-418.
    Pubmed CrossRef
  29. Joslyn S, Alexander K. Evaluating artificial intelligence algorithms for use in veterinary radiology. Vet Radiol Ultrasound 2022; 63 Suppl 1: 871-879.
    Pubmed CrossRef
  30. Kang X, Zhang XD, Liu G. Accurate detection of lameness in dairy cattle with computer vision: a new and individualized detection strategy based on the analysis of the supporting phase. J Dairy Sci 2020; 103: 10628-10638.
    Pubmed CrossRef
  31. Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc 2020; 92: 807-812.
    Pubmed CrossRef
  32. Keegan KG, Arafat S, Skubic M, Wilson DA, Kramer J. Detection of lameness and determination of the affected forelimb in horses by use of continuous wavelet transformation and neural network classification of kinematic data. Am J Vet Res 2003; 64: 1376-1381.
    Pubmed CrossRef
  33. Kil N, Ertelt K, Auer U. Development and validation of an automated video tracking model for stabled horses. Animals (Basel) 2020; 10: 2258.
    Pubmed KoreaMed CrossRef
  34. Kim DW, Jang HY, Kim KW, Shin Y, Park SH. Design characteristics of studies reporting the performance of artificial intelligence algorithms for diagnostic analysis of medical images: results from recently published papers. Korean J Radiol 2019; 20: 405-410.
    Pubmed KoreaMed CrossRef
  35. Kim JY, Lee HE, Choi YH, Lee SJ, Jeon JS. CNN-based diagnosis models for canine ulcerative keratitis. Sci Rep 2019; 9: 14209.
    Pubmed KoreaMed CrossRef
  36. Kittichai V, Kaewthamasorn M, Thanee S, Jomtarak R, Klanboot K, Naing KM, et al. Classification for avian malaria parasite Plasmodium gallinaceum blood stages by using deep convolutional neural networks. Sci Rep 2021; 11: 16919.
    Pubmed KoreaMed CrossRef
  37. Kokkinos Y, Morrison J, Bradley R, Panagiotakos T, Ogeer J, Chew D, et al. An early prediction model for canine chronic kidney disease based on routine clinical laboratory tests. Sci Rep 2022; 12: 14489.
    Pubmed KoreaMed CrossRef
  38. Krone LM, Brown CM, Lindenmayer JM. Survey of electronic veterinary medical record adoption and use by independent small animal veterinary medical practices in Massachusetts. J Am Vet Med Assoc 2014; 245: 324-332.
    Pubmed KoreaMed CrossRef
  39. Li S, Wang Z, Visser LC, Wisner ER, Cheng H. Pilot study: application of artificial intelligence for detecting left atrial enlargement on canine thoracic radiographs. Vet Radiol Ultrasound 2020; 61: 611-618.
    Pubmed KoreaMed CrossRef
  40. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019; 1: e271-e297. Erratum in: Lancet Digit Health 2019; 1: e334.
    Pubmed CrossRef
  41. Mao A, Giraudet CSE, Liu K, De Almeida Nolasco I, Xie Z, Xie Z, et al. Automated identification of chicken distress vocalizations using deep learning models. J R Soc Interface 2022; 19: 20210921.
    Pubmed KoreaMed CrossRef
  42. Marzahl C, Aubreville M, Bertram CA, Stayt J, Jasensky AK, Bartenschlager F, et al. Deep learning-based quantification of pulmonary hemosiderophages in cytology slides. Sci Rep 2020; 10: 9795.
    Pubmed KoreaMed CrossRef
  43. May A, Gesell-May S, Müller T, Ertel W. Artificial intelligence as a tool to aid in the differentiation of equine ophthalmic diseases with an emphasis on equine uveitis. Equine Vet J 2022; 54: 847-855.
    Pubmed CrossRef
  44. McGreevy P, Thomson P, Dhand NK, Raubenheimer D, Masters S, Mansfield CS, et al. VetCompass Australia: a national big data collection system for veterinary science. Animals (Basel) 2017; 7: 74.
    Pubmed KoreaMed CrossRef
  45. Meller S, Zamansky A, Sinitca A, Kaplun D, Meyerhoff N, Stein V, et al. Sounds of seizures-acoustic information enables immediate recognition and detection of generalized tonic-clonic seizures in dogs. J Vet Intern Med 2022; 36: 305.
  46. Motahari-Nezhad H, Fgaier M, Mahdi Abid M, Péntek M, Gulácsi L, Zrubka Z. Digital biomarker-based studies: scoping review of systematic reviews. JMIR Mhealth Uhealth 2022; 10: e35722.
    Pubmed KoreaMed CrossRef
  47. Mouloodi S, Rahmanpanah H, Burvill C, Davies HMS. Prediction of displacement in the equine third metacarpal bone using a neural network prediction algorithm. Biocybern Biomed Eng 2020; 40: 849-863.
    CrossRef
  48. Mouloodi S, Rahmanpanah H, Burvill C, Davies HMS. Prediction of load in a long bone using an artificial neural network prediction algorithm. J Mech Behav Biomed Mater 2020; 102: 103527.
    Pubmed CrossRef
  49. Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health 2021; 3: e195-e203.
    Pubmed CrossRef
  50. Nagamori Y, Hall Sedlak R, DeRosa A, Pullins A, Cree T, Loenser M, et al. Evaluation of the VETSCAN IMAGYST: an in-clinic canine and feline fecal parasite detection system integrated with a deep learning algorithm. Parasit Vectors 2020; 13: 346.
    Pubmed KoreaMed CrossRef
  51. Nagamori Y, Sedlak RH, DeRosa A, Pullins A, Cree T, Loenser M, et al. Further evaluation and validation of the VETSCAN IMAGYST: in-clinic feline and canine fecal parasite detection system integrated with a deep learning algorithm. Parasit Vectors 2021; 14: 89.
    Pubmed KoreaMed CrossRef
  52. Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, et al. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 2020; 368: m689.
    Pubmed KoreaMed CrossRef
  53. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.
    Pubmed KoreaMed CrossRef
  54. Park J, Choi B, Ko J, Chun J, Park I, Lee J, et al. Deep-learning-based automatic segmentation of head and neck organs for radiation therapy in dogs. Front Vet Sci 2021; 8: 721612.
    Pubmed KoreaMed CrossRef
  55. Parra C, Grijalva F, Núñez B, Núñez A, Pérez N, Benítez D. Automatic identification of intestinal parasites in reptiles using microscopic stool images and convolutional neural networks. PLoS One 2022; 17: e0271529.
    Pubmed KoreaMed CrossRef
  56. Pastell ME, Kujala M. A probabilistic neural network model for lameness detection. J Dairy Sci 2007; 90: 2283-2292.
    Pubmed CrossRef
  57. Pichler M, Hartig F. Machine learning and deep learning—a review for ecologists. Methods Ecol Evol 2023; 14: 994-1016.
    CrossRef
  58. Porter IR, Wieland M, Basran PS. Feasibility of the use of deep learning classification of teat-end condition in Holstein cattle. J Dairy Sci 2021; 104: 4529-4536.
    Pubmed CrossRef
  59. Post C, Rietz C, Büscher W, Müller U. Using sensor data to detect lameness and mastitis treatment events in dairy cows: a comparison of classification models. Sensors (Basel) 2020; 20: 3863.
    Pubmed KoreaMed CrossRef
  60. Rai T, Morisi A, Bacci B, Bacon NJ, Dark MJ, Aboellail T, et al. Deep learning for necrosis detection using canine Perivascular Wall Tumour whole slide images. Sci Rep 2022; 12: 10634.
    Pubmed KoreaMed CrossRef
  61. Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med 2022; 28: 31-38.
    Pubmed CrossRef
  62. Rose N, Toews L, Pang DS. A systematic review of clinical audit in companion animal veterinary medicine. BMC Vet Res 2016; 12: 40.
    Pubmed KoreaMed CrossRef
  63. Roush WB, Wideman RF Jr, Cahaner A, Deeb N, Cravener TL. Minimal number of chicken daily growth velocities for artificial neural network detection of pulmonary hypertension syndrome (PHS). Poult Sci 2001; 80: 254-259.
    Pubmed CrossRef
  64. Salvi M, Molinari F, Iussich S, Muscatello LV, Pazzini L, Benali S, et al. Histopathological classification of canine cutaneous round cell tumors using deep learning: a multi-center study. Front Vet Sci 2021; 8: 640944.
    Pubmed KoreaMed CrossRef
  65. Sánchez-Vizcaíno F, Jones PH, Menacere T, Heayns B, Wardeh M, Newman J, et al. Small animal disease surveillance. Vet Rec 2015; 177: 591-594.
    Pubmed CrossRef
  66. Schobesberger H, Peham C. Computerized detection of supporting forelimb lameness in the horse using an artificial neural network. Vet J 2002; 163: 77-84.
    Pubmed CrossRef
  67. Secinaro S, Calandra D, Secinaro A, Muthurangu V, Biancone P. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak 2021; 21: 125.
    Pubmed KoreaMed CrossRef
  68. Shahinfar S, Khansefid M, Haile-Mariam M, Pryce JE. Machine learning approaches for the prediction of lameness in dairy cows. Animal 2021; 15: 100391.
    Pubmed CrossRef
  69. Song KD, Kim M, Do S. The latest trends in the use of deep learning in radiology illustrated through the stages of deep learning algorithm development. J Korean Soc Radiol 2019; 80: 202-212.
    CrossRef
  70. Teixeira VA, Lana AMQ, Bresolin T, Tomich TR, Souza GM, Furlong J, et al. Using rumination and activity data for early detection of anaplasmosis disease in dairy heifer calves. J Dairy Sci 2022; 105: 4421-4433.
    Pubmed CrossRef
  71. Theodoropoulos G, Loumos V, Anagnostopoulos C, Kayafas E, Martinez-Gonzales B. A digital image analysis and neural network based system for identification of third-stage parasitic strongyle larvae from domestic animals. Comput Methods Programs Biomed 2000; 62: 69-76.
    Pubmed CrossRef
  72. Trachtman AR, Bergamini L, Palazzi A, Porrello A, Capobianco Dondona A, Del Negro E, et al. Scoring pleurisy in slaughtered pigs using convolutional neural networks. Vet Res 2020; 51: 51.
    Pubmed KoreaMed CrossRef
  73. Yakubu A, Oluremi OIA, Ekpo EI. Predicting heat stress index in Sasso hens using automatic linear modeling and artificial neural network. Int J Biometeorol 2018; 62: 1181-1186.
    Pubmed CrossRef
  74. Ye Y, Sun WW, Xu RX, Selmic LE, Sun M. Intraoperative assessment of canine soft tissue sarcoma by deep learning enhanced optical coherence tomography. Vet Comp Oncol 2021; 19: 624-631.
    Pubmed CrossRef
  75. Yoon Y, Hwang T, Lee H. Prediction of radiographic abnormalities by the use of bag-of-features and convolutional neural networks. Vet J 2018; 237: 43-48.
    Pubmed CrossRef
  76. You SC, Lee S, Choi B, Park RW. Establishment of an international evidence sharing network through common data model for cardiovascular research. Korean Circ J 2022; 52: 853-864.
    Pubmed KoreaMed CrossRef
  77. ZareBidaki M, Allahyari E, Zeinali T, Asgharzadeh M. Occurrence and risk factors of brucellosis among domestic animals: an artificial neural network approach. Trop Anim Health Prod 2022; 54: 62.
    Pubmed CrossRef
  78. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med 2018; 15: e1002683.
    Pubmed KoreaMed CrossRef
  79. Zhang M, Zhang K, Yu D, Xie Q, Liu B, Chen D, et al. Computerized assisted evaluation system for canine cardiomegaly via key points detection with deep learning. Prev Vet Med 2021; 193: 105399.
    Pubmed CrossRef
  80. Zhang Y, Nie A, Zehnder A, Page RL, Zou J. VetTag: improving automated veterinary diagnosis coding via large-scale language modeling. NPJ Digit Med 2019; 2: 35.
    Pubmed KoreaMed CrossRef

Article

Review Article

J Vet Clin 2023; 40(4): 243-259

Published online August 31, 2023 https://doi.org/10.17555/jvc.2023.40.4.243

Copyright © The Korean Society of Veterinary Clinics.

Scoping Review of Machine Learning and Deep Learning Algorithm Applications in Veterinary Clinics: Situation Analysis and Suggestions for Further Studies

Kyung-Duk Min*

College of Veterinary Medicine, Chungbuk National University, Cheongju 28644, Korea

Correspondence to:*kdmin@cbnu.ac.kr

Received: July 12, 2023; Revised: August 18, 2023; Accepted: August 21, 2023

This is an open access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Machine learning and deep learning (ML/DL) algorithms have been successfully applied in medical practice. However, their application in veterinary medicine is relatively limited, possibly due to a lack in the quantity and quality of relevant research. Because the potential demands for ML/DL applications in veterinary clinics are significant, it is important to note the current gaps in the literature and explore the possible directions for advancement in this field. Thus, a scoping review was conducted as a situation analysis. We developed a search strategy following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. PubMed and Embase databases were used in the initial search. The identified items were screened based on predefined inclusion and exclusion criteria. Information regarding model development, quality of validation, and model performance was extracted from the included studies. The current review found 55 studies that passed the criteria. In terms of target animals, the number of studies on industrial animals was similar to that on companion animals. Quantitative scarcity of prediction studies (n = 11, including duplications) was revealed in both industrial and non-industrial animal studies compared to diagnostic studies (n = 45, including duplications). Qualitative limitations were also identified, especially regarding validation methodologies. Considering these gaps in the literature, future studies examining the prediction and validation processes, which employ a prospective and multi-center approach, are highly recommended. Veterinary practitioners should acknowledge the current limitations in this field and adopt a receptive and critical attitude towards these new technologies to avoid their abuse.

Keywords: machine learning, deep learning, veterinary clinics, scoping review, validation

Introduction

The application of machine learning and deep learning (ML/DL) algorithms has altered the medical landscape. With respect to medical diagnostics, such as radiology and pathology, a growing number of studies have reported the reliable performance of ML/DL-based automatic systems (61) which are equivalent to or even better than those of human experts (40). Based on accumulated research, numerous commercialized medical devices have been officially approved for use in clinical practice, especially in countries such as Europe and the USA (49). Moreover, various digital biomarkers have been developed to predict the prognosis of chronic diseases such as cancer and cardiovascular diseases (46).

However, ML/DL applications in veterinary medicine seem to be far behind that in human medicine in terms of both quantity and quality, especially in South Korea. One of the major reasons for this slow progress can be the lack of high-quality medical data. Although most veterinary clinics use electronic medical chart systems (38), the analyzable data present in them is insufficient. Most veterinarians have not been educated and motivated regarding appropriate charting, especially in South Korea, where an insurance system for animals is lacking and purchasing some medical drugs is possible without a veterinarian’s prescription. Even if medical records are accumulated appropriately, merging multi-clinic medical records remains challenging owing to the lack of standardized medical coding classification systems (80).

However, the demand for ML/DL applications has increased with the rapid growth of the veterinary industry. Applications in industrial animal husbandry, disease screening, and medical data management have been developed, representing its growth potential (26). This growth could have positive implications, such as pioneering new markets and improving the quality of veterinary medical services. However, it could also increase the possibility of misuse and abuse, considering the challenges regarding data quality. Veterinarians who are practically involved in clinics should have a proper level of ML/DL literacy to prevent misuse and abuse and guide its development in a constructive manner.

Therefore, a scoping review was conducted to clarify current ML/DL applications in veterinary medicine and explore the directions for the advancement of this field. In this review, the application scope (specific domains in which the ML/DL methodology was applied), methodological details, and medical utility (Performance of the ML/DL models) of previously published relevant studies were investigated.

Materials and Methods

A scoping review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines (53). Two electronic scientific literature databases (PubMed and Embase) were used to identify published studies that used ML/DL algorithms in veterinary clinics, especially those examining diagnostics and prognosis prediction. The search terms were developed (Supplementary Tables 1-2) based on two previous studies that examined artificial intelligence in the medical field (52) and veterinary medicine (62).

An initial search using these terms was conducted on September 22, 2022, and screening processes were implemented using predetermined inclusion/exclusion criteria. The inclusion criteria were as follows: 1) ML/DL applications in veterinary medicine focusing on diagnostics and prognosis prediction; 2) applications to companion animals, industrial animals, and/or wildlife; 3) studies written in English; and 4) original articles. The exclusion criteria were as follows: 1) applications to experimental animals; 2) population-level studies whose analysis unit was the aggregate level; 3) applications to animal husbandry or management (e.g., pregnancy detection); 4) applications to drug discovery; 5) applications to biosecurity; and 6) methodological studies that focused on algorithm development, optimization, or automatic data labelling. During the first screening process, the titles and abstracts of each searched paper were checked. In the second screening process, the full text was examined, and the final list of the included studies was determined.

For each included study the following information was extracted: 1) authors, publication year, and title; 2) purpose of study; 3) target animals; 4) number of samples used for ML/DL models; 5) algorithm types (e.g. artificial neural network, recurrent neural network [RNN], or other types); 6) whether the study used cross-validation framework to assess model performance; 7) whether the study collected test dataset prospectively or retrospectively; 8) whether the study used multi-center datasets for model building or validation; 9) measurements used for the performance of models (e.g., sensitivity, specificity, or other measurements); and 10) model performances.

Results

The screening process for this scoping review is illustrated in Fig. 1. In the initial search, 598 and 376 studies were found in the PubMed and Embase databases, respectively. After removing duplicate studies, 699 studies were identified. A total of 532 and 112 papers were excluded after the first and second screenings, respectively. The remaining 55 studies were included. Detailed information on the included papers is provided in Tables 1, 2.

Table 1 . General information regarding included studies in the review.

Author and yearAnimal typeTarget animalsSample sizeAlgorithm
G. Theodoropoulos et al., 2000 (71)DomesticSheep255 images of 57 individual larvae (5genera)ANN (artificial neural network; feature selection by manual, 16 features were measured)
W. B. Roush et al., 2001 (63)DomesticChickenCase 6-40, normal 33-91BP3(back propagation neural network), WardBP (Ward back propagation neural network), PNN (Probabilistic neural network), GRNN (general regression neural network)
H. Schobesberger and C. Peham, 2002 (66)DomesticHorse175 (42 control/ 133 low to medium grade lame)ANN (feature selection by manual)
K. G. Keegan et al., 2003 (32)DomesticHorse12 adult horseANN (feature selection by manual)
M. E. Pastell and M. Kujalaf, 2007 (56)DomesticDairy cow73 cows (training 37 cows, 5,074 observation, validation 36 cows, 4,868 measurements)Probabilistic Neural Network Model (feature selection by manual)
S. M. Ghotoorlar et al., 2012 (25)DomesticDairy cow105 dairy cowsANN (feature selection by manual)
T. Banzato et al., 2018 (4)CompanionCanine80 (56 meningioma, 24 glioma)Convolutional neural networks (CNN), GoogleNet
T. Banzato et al., 2018 (5)CompanionCanine48 (32 case, 16 control)Deep neural networks (DNN), especially AlexNet
T. Banzato et al., 2018 (6)CompanionCanine56 (grade 1 = 26, grade 2 = 22, grade 3 = 8)AlexNet, DNN
A. Yakubu et al., 2018 (73)DomesticChicken167ANN
Y. Yoon et al., 2018 (75)CompanionDogs3,142 for cardiomegaly (1,571 normal and 1,571 abnormal from 1,143 dogs), 2,086 for lung pattern (1,043 normal and 1,043 abnormal from 1,247 dogs), 892 for mediastinal shift (446 normal and 446 abnormal from 387 dogs), 940 for pleural effusion (470 normal and 470 abnormal from 284 dogs), and 78 for pneumothorax (39 normal and 39 abnormal from 61 dogs)Bag-of-features (BOF) and CNN
R. Bradley et al., 2019 (15)CompanionCat106,251 catsRecurrent Neural Network (RNN)
M. Ebrahimi et al., 2019 (20)DomesticCow297,004 milking samples each with eight milking featuresANN, Naïve Bayes, GLM, Decision tree, Random forest, Gradient boosted tree
J. Y. Kim et al., 2019 (35)CompanionDogs1,040 imagesCNN (GoogLe net, Resnet, and VGGnet)
M. Aubreville et al., 2020 (3)CompanionDogs32 whole slide imagesCNN, RetinaNet, ResNet-18, Unet
V. Biourge et al., 2020 (12)CompanionCats218ANN
L. E. Broughton-Neiswanger et al., 2020 (16)CompanionCats12Partial least squares discriminant analysis, Random forest
S. Burti et al., 2020 (17)CompanionDogs1,465 imagesCNN
E. Fernández-Carrión et al., 2020 (22)Etc.Wild boar8CNN
M. A. Fraiwan and S. M. Abutarbush, 2020 (24)DomesticHorse285 horsesBayes Network, Naïve Bayes, DNN, Random forest
X. Kang et al., 2020 (30)DomesticCow100 cowsRFB_NET_SSD deep learning network
N. Kil et al., 2020 (33)DomesticHorse34 horses (65 video)CNN
S. Li et al., 2020 (39)CompanionDogs792 radiographsCNN
C. Marzahl et al., 2020 (42)DomesticHorse17 completely annotated cytology whole slide images (WSI) containing 78,047 hemosiderophagesCNN (RetinaNet)
S. Mouloodi et al., 2020 (47)DomesticHorse3 third metacarpal bones from 3 racehorsesANN
S. Mouloodi et al., 2020 (48)DomesticHorse9 equine third metacarpal bones from 9 thoroughbred horsesANN
Y. Nagamori et al., 2020 (50)CompanionCat, dogs100CNN
C. Post et al., 2020 (59)DomesticCow167 cowsLogistic Regression (LR), Support Vector Machine (SVM), K-nearest neighbors (KNN), Gaussian Naïve Bayes (GNB), Extra Trees Classifier (ET), Random forest
A. R. Trachtman et al., 2020 (72)DomesticPigs5,902 imagesCNN
T. Banzato et al., 2021 (7)CompanionDogs3,839 latero-lateral radiographsCNN (ResNet-50, DenseNet-121)
T. Banzato et al., 2021 (8)CompanionCat1,062 latero-lateral radiographsCNN (ResNet 50 and Inception V3)
A. Biercher et al., 2021 (11)CompanionDogsThoracolumbar MR images from 500 dogsCNN
E. Boissady et al., 2021 (13)CompanionCat, dogs30 canine and 30 feline thoracic lateral radiographsCNN
L. Bonicelli et al., 2021 (14)DomesticPigs7,564 picturesCNN
V. Kittichai et al., 2021 (36)DomesticPoultry12,761 single cell imagesCNN (Darknet, Darknet19, Darknet19-448 and Densenet201)
Y. Nagamori et al., 2021 (51)CompanionCat, dogs460 samples for 4 parasites (80-200 per parasite)You only look once (YOLOv3) model
J. Park et al., 2021 (54)CompanionDogs90 dogsHA, DLBAS, and the readjustment of the predicted data obtained via the DLBAS of the clinical test sets (HA_DLBAS)
I. R. Porter et al., 2021 (58)DomesticCattleA total of 398 digital images from dairy cows’ uddersCNN (GoogLeNet)
M. Salvi et al., 2021 (64)CompanionDogs416 canine cutaneous round cell tumors (RCT) (117 cases)AlexNet, Inceptionv3, ResNet, Emsemble
S. Shahinfar et al., 2021 (68)DomesticCattle2,535 lameness scores (2,248 sound and 287 unsound)Naïve Bayes (NB), Random Forest (RF) and Multilayer Perceptron (MLP), to predict cases of lameness using milk production and conformation traits logistc (LR)
Y. Ye et al., 2021 (74)CompanionDogs220 imagesCNN (ResNet-50)
M. Zhang et al., 2021 (79)CompanionDogs2,670 lateral X-ray imagesCNN (HRNet)
A.N. ELKhamary et al., 2022 (21)DomesticHorse16 horse 32 limbs (16 normal tendons and 16 abnormal tendons)C4.5 algorithm (Quinlan), a decision tree classifier of Weka software package
E. A. Bauer and W. Jagusiak, 2022 (9)DomesticCattle168 cowsANN
K. Benfodil et al., 2022 (10)DomesticDromedaries115 dromedariesANN
L. Dumortier et al., 2022 (19)CompanionCat500 annotated Thoracic radiograph images(348 veterinary visit 296 cats)CNN (ResNet50V2)
P. Figueirinhas et al., 2022 (23)CompanionDogs15 working dogs (pilot study)LSTM
Y. Kokkinos et al., 2022 (37)CompanionDogs57,402 dogsRNN
A. Mao et al., 2022 (41)DomesticChicken5,336 voice calls (3,363 distress calls and 1,973 natural barn sound)CNN (light-VGG11)
A. May et al., 2022 (43)DomesticHorse2,607 imagesCNN
T. R. Müller et al., 2022 (45)CompanionDogs62 canine (41 case 21 control) 4,000 images (2,000 case 2,000 control)CNN (VGG16)
C. Parra et al., 2022 (55)Etc.Reptile3,616 images data samples and 26 videos (4,849 frames)CNN (MobileNet)
T. Rai et al., 2022 (60)CompanionDogs32 patientsCNN (DenseNet-161)
V. A. Teixeira et al., 2022 (70)DomesticCattle55 Holstein calvesRNN
M. ZareBidaki et al., 2022 (77)DomesticGoat, sheep cows200 paired sample (100 blood, 100 milk) 100 animalsANN

Table 2 . Validation methodologies and model performance of the included studies in the review.

Author and yearCVProspectiveMulti-center approachModel performancePurpose


Training setTest setIndexValue
G. Theodoropoulos et al., 2000 (71)YesNoNoNoSensitivity42.4-80.7%Diagnostics
W. B. Roush et al., 2001 (63)YesNoNoNoSensitivity0-100%Prediction
H. Schobesberger and C. Peham, 2002 (66)YesNoNoNoAgreement78.60%Diagnostics
K. G. Keegan et al., 2003 (32)YesNoNoNoAgreement85%Diagnostics
M. E. Pastell and M. Kujalaf, 2007 (56)YesNoNoNoAgreement and sensitivityAgreement = 96.2%
Sensitivity = 100%
Diagnostics
S. M. Ghotoorlar et al., 2012 (25)YesNoNoNoSensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), Pearson correlation coefficientSensitivity = 0.5-1Specificity = 0.91-1PPV = 0.76-1NPV = 0.92 -1Pearson correlation coefficient = 0.94Diagnostics
T. Banzato et al., 2018 (4)YesNoYesNoAgreement, Matthews correlation coefficient (MCC)Agreement = 90-94%
MCC = 0.8-0.88
Diagnostics
T. Banzato et al., 2018 (5)YesNoNoNoAUC, sensitivity, specificityAUC = 0.91
Sensitivity = 100%
Specificity = 82.8%
Diagnostics
T. Banzato et al., 2018 (6)YesNoYesNoAgreement, multi-class Matthew’s correlation coefficient (MCMCC)Agreement = 65.2-82.2%
MCMCC = 0.44-0.68
Diagnostics
A. Yakubu et al., 2018 (73)YesNoNoNor, R2, RMSEr = 0.983
R2 = 0.966
RMSE = 0.04806
Prediction
Y. Yoon et al., 2018 (75)YesNoNoNoAccuracy, sensitivityAccuracy(CNN; 92.9-96.9% and BOF; 79.6-96.9%) and sensitivity (CNN; 92.1-100% and BOF; 74.1-94.8%)Prediction
R. Bradley et al., 2019 (15)YesNoNoNoSensitivity, specificity(1 year before) sensitivity 63.0%; (2 year before) sensitivity 44.2% specificity remaining around 99%Prediction
M. Ebrahimi et al., 2019 (20)YesNoNoNoAUC0.826Prediction
J. Y. Kim et al., 2019 (35)YesNoYesNoSensitivity79.4-100%Diagnostics
M. Aubreville et al., 2020 (3)YesNoNoNoCorrelation coefficient0.868-0.979Diagnostics
V. Biourge et al., 2020 (12)YesYesNoYesAccuracy, sensitivity, specificity, PPV, NPVAccuracy = 88%
Sensitivity = 87%
Specificity = 70%
PPV = 53%
NPV = 92%
Prediction
L. E. Broughton-Neiswanger et al., 2020 (16)YesNoNoNoSensitivity, specificity, AUCAUC = 0.87-1Sensitivity = 0-100%Specificity = 50-100%Diagnostics
S. Burti et al., 2020 (17)YesNoNoNoAUC0.904-0.973Diagnostics
E. Fernández-Carrión et al., 2020 (22)YesNoNoNoAgreement95.4-97.2%Diagnostics
M. A. Fraiwan and S. M. Abutarbush, 2020 (24)YesNoNoNoPrecision, recall, F-measure, Accuracy(need for surgery)Precision = 69.5-74.1%Recall = 72.4-99.3%F-measure = 72.2-81.8%Accuracy = 69.0-76.0%(survival)Precision = 87.5-97.4%Recall = 80.5-87.8%F-measure = 87.2-89.1%Accuracy = 83.9-85.2%Prediction
X. Kang et al., 2020 (30)YesNoNoNoSensitivity, specificitySensitivity = 0.83-1Specificity = 0.95-1Diagnostics
N. Kil et al., 2020 (33)YesNoNoNoSensitivity, accuracySensitivity = 0.79-0.94Accuracy = 0.82-0.94Diagnostics
S. Li et al., 2020 (39)YesNoNoNoAccuracy, sensitivity, and specificityAccuracy = 82.71%
Sensitivity = 68.42%
Specificity = 87.09%
Diagnostics
C. Marzahl et al., 2020 (42)YesNoNoNoPrecision0.64-0.66Diagnostics
S. Mouloodi et al., 2020 (47)YesNoNoNoDetermination coefficient (R2)0.9116-0.9599Prediction
S. Mouloodi et al., 2020 (48)YesNoNoNoDetermination coefficient (R2)0.9999Prediction
Y. Nagamori et al., 2020 (50)YesNoNoNoPearson correlation coefficient, sensitivity, specificityPearson correlation coefficient = 0.89-0.99Sensitivity = 0.758-1Specificity = 0.918-1Diagnostics
C. Post et al., 2020 (59)YesNoNoNoAUC0.71-0.79Diagnostics
A. R. Trachtman et al., 2020 (72)YesNoNoNoAccuracy, sensitivity, specificityAccuracy = 62-96%Sensitivity = 84-100%Specificity = 92-96%Diagnostics
T. Banzato et al., 2021 (7)YesNoNoNoAUC0.8Diagnostics
T. Banzato et al., 2021 (8)YesNoYesNoAUC0.58-0.97Diagnostics
A. Biercher et al., 2021 (11)YesNoYesYesSensitivity, specificityIVDE sens 73.46-90.1/spec 67.6-99.0IVDP sens 67.86-100/spec 74.9-96.4FCE/ANNPE sens 62.2-90.1/spec 90.1-97.9Syringomyelia sens 0-10/spec 100Neoplasma sens 0-37.5/spec 60-94.7Diagnostics
E. Boissady et al., 2021 (13)NANoNoNoICC0.998-0.999Diagnostics
L. Bonicelli et al., 2021 (14)YesNoYesYesSensitivity, specificity, Pearson correlation coefficientSensitivity = 81.25-100 %
Specificity = 99.38 %
Pearson correlation coefficient = 0.96
Diagnostics
V. Kittichai et al., 2021 (36)YesNoNANAAccuracy99%Dignostics
Y. Nagamori et al., 2021 (51)NANAYESNASensitivity, specificitySensitivity = 75.8-100%
Specificity = 93.1-100%
Dignostics
J. Park et al., 2021 (54)YesNoNoNoDice similarity coefficient (DSC) and the Hausdorff distance (HD)DSC 0.78-0.94
HD 2.30-4.30 mm
Dignostics
I. R. Porter et al., 2021 (58)YesNoYesYesAUC0.542-0.920Dignostics
M. Salvi et al., 2021 (64)YesNoYesYesAccuracy91.66%-100%Dignostics
S. Shahinfar et al., 2021 (68)YesNoYesYesAUC, F1AUC = 0.61-0.67
F1 = 0.01-0.27
Dignostics
Y. Ye et al., 2021 (74)YesNoNANAAUC, accuracy, F1 scoreAUC = 99.37Accuracy = 97.62 F1 score = 96.7Dignostics
M. Zhang et al., 2021 (79)YesNoYesYesSensitivity86.40%Dignostics
A.N. ELKhamary et al., 2022 (21)YesNoNoNoAccuracy, PPV, sensitivity, kappaAccuracy = 93.7%
PPV = 93.80%
Sensitivity = 93.80%
Kappa = 0.88
Dignostics
E. A. Bauer and W. Jagusiak, 2022 (9)YesNoYESYESAUC0.82-0.89Dignostics
K. Benfodil et al., 2022 (10)YesNoNANAPearson correlation coefficient0.943Dignostics
L. Dumortier et al., 2022 (19)YesNoNoNoAccuracy, F1-Score, Specificity, Positive Predictive Value and SensitivityAccuracy = 82%
F1-Score = 85%
Specificity = 75%
PPV = 81%
Sensitivity = 88%
Dignostics
P. Figueirinhas et al., 2022 (23)YesNoNoNoAccuracyAccuracy = 60%Dignostics
Y. Kokkinos et al., 2022 (37)YesNoNoNoSensitivity, PPV, NPVSensitivity = 44.8-68.8% PPV = 15-23% NPV > 99%Prediction
A. Mao et al., 2022 (41)YesNoYesYesPrecision, recall, F1-score and accuracyPrecision = 94.58%
Recall = 94.89%
F1-score = 94.73%
Accuracy = 95.07%
Dignostics
A. May et al., 2022 (43)YesNoNoNoAccuracy, cross entropyAccuracy = 96.66%
Cross entropy = 0.02
Dignostics
T. R. Müller et al., 2022 (45)YesNoNoNoAccuracy, sens, spec, PPV, NPVAccuracy 88.7%
Sensitivity 90.2%
Specificity 81.8%
PPV 92.5%
NPV 81.8%
Dignostics
C. Parra et al., 2022 (55)YesNANANAAccuracy, AUCAccuracy = 94.26
AUC = 0.996
Dignostics
T. Rai et al., 2022 (60)YesNoNoNoF1-score0.708Dignostics
V. A. Teixeira et al., 2022 (70)YesNoNoNoAccuracy, sensitivity, and specificity, PPV, NPVAccuracy = 85-98,
Sensitivity = 87-96
Specificity = 78-100
PPV = 85-100
NPV = 88-96
Prediction & diagnosis
M. ZareBidaki et al., 2022 (77)YesNoNANASensitivity, specificity, AUCSensitivity = 81%
Specificity = 62%
AUC = 0.799
Dignostics

Figure 1. Flow diagram of information through the different phases of the review.

The temporal trends in ML/DL-related publications are illustrated in Fig. 2. Although most of these studies were published after 2000, a rapid growth in their quantity began in 2018. Before this surge, the applications of ML/DL were concentrated in industrial animals; however, their applications in companion animals have been expanding since 2018. Only a few studies on other animal species (wildlife and exotic animals) have been published, even after 2020.

Figure 2. Temporal trend of machine learning and deep learning application studies in veterinary clinics by types of target animals. Note: The others group includes wildlife and exotic animals.

Fig. 3 shows the proportion of the specific purposes of each study, such as target animal species and domains of application (whether ML/DL was used for predictive or diagnostic purposes). While the number of studies for both industrial and non-industrial animals was similar (31 and 30 for non-industrial and industrial animals, respectively, including duplicates), the number of diagnostic studies was higher than that of prediction studies (the number of diagnostic and prediction studies were 45 and 11, respectively, including duplicates). In terms of specific animal species, studies on dogs were generally dominant among studies on non-industrial animals (70.3% of diagnostic studies and 50.0% of prediction studies), while studies on cows (39.1% of diagnostic studies and 28.6% of prediction studies) and horses (26.1% of diagnostic studies and 42.9% of prediction studies) were dominant among studies on industrial animals.

Figure 3. Proportion of target animal species by the purpose of studies. Note: Numbers include duplicates. For example, a study on industrial animals has the purposes of both diagnostics and prediction.

Table 3 shows details regarding the identified studies, including the sample size used for model development and validation, the algorithm used, whether the authors employed prospective data collection for validation, whether they used multi-center data for model development and validation, and model performance. In terms of validation, almost every publication stated that they implemented cross-validation (splitting data into training and test sets to avoid over-evaluation), although there was an insufficiency in the relevant descriptions in some of the studies (n = 2). However, a minority of the studies employed a multi-center approach for model development (n = 13) and validation (n = 9), and only one study prospectively collected the test datasets. The majority of the identified studies used neural network-based algorithms, such as RNN and convolutional neural network, and most of the studies targeted binary problems rather than continuous outcomes. Although the numbers of data that used for model development are relatively small for several studies (16,22,33), the reported model performance of most studies tended to be within an acceptable range (e.g., Area Under the Receiver Operating Characteristic Curve (AUC) value >0.9).

Table 3 . Profile of included studies.

Target animal typeStudy purposesN*NNCVPros§Multi∥
Industrial animalsDiagnostics21192105
Prediction77700
Companion animalsDiagnostics22212002
Prediction44411
OthersDiagnostics22200

*Number of studies..

Number of studies that used neural network-based algorithm..

Number of studies that conducted cross-validation approach to measure performance..

§Number of studies that employed prospective approach for collecting dataset for testing..

Number of studies that used multi-center data for validation..

The others group includes wildlife and exotic animals..

Note: The numbers include duplication. For example, a study for industrial animals have both purpose, diagnostics and prediction. There is no prediction studies for the other animals..


Discussion

A scoping review was conducted as a situation analysis to identify the current gaps in ML/DL application research in veterinary clinics and suggest directions for further improvement in this field. The review found that the history of ML/DL applications in veterinary medicine is relatively short compared to that in human medicine and the healthcare sector (31). Possibly due to its short history, quantitative scarcity and methodological gaps were identified, especially regarding the validation and data collection framework, although the reported model performance was generally within acceptable levels.

The first gap that must be highlighted is quantitative scarcity. Although there is a possibility that the current review will exclude published papers, it seems clear that the relevant papers are fewer than those in the human medical field (2,52,67,69). Specifically, prediction studies were scarce, possibly because of their technical difficulties. They usually include extrapolation because the prediction target is future data. Considering that extrapolation is more sensitive to overfitting and a lack of variables, the performance of the model tends to be lower than that of the models for interpolation (57). However, prediction studies are practically useful because they can be employed for optimal treatment recommendations and prognostic assessment, which are the most frequent practices in veterinary clinics. Purification has also been observed in studies on wildlife. Lack of data may explain this discrepancy. Compared with medicine for companion and industrial animals, wildlife medicine covers more animal species with less resources. Therefore, the quantity of data for each species is usually lower than that for other medical areas, even though large amounts of data regarding specific species and medical problems are required for ML/DL applications.

Qualitative gaps in model validation should be emphasized. Considering that ML/DL approaches cannot inherently employ physiological or pathological mechanisms, an innate limitation of this data-driven approach is overfitting and induction. The issues can be practically addressed by demonstrating acceptable performance in an independent dataset, which is called cross-validation. Most of the studies identified in this review employed this approach. However, the current review found that only a few of them have obtained appropriate test sets. As the selection of the test set is essential for its validation, the representativeness of the test set must be ensured (27). Therefore, prospective data collection from multiple centers is the best way to ensure this representativeness (34,78). Veterinary clinicians should be aware of the qualitative gaps in current ML/DL application studies to avoid possible misuse of these models in clinical practice.

From the veterinary clinicians’ point of view, excellent model performance alone is not sufficient to recommend its practical use. For instance, even if some ML/DL models show very high AUC, representing great performance in diagnostics, the operation of the model could require a significant amount of manpower, time, or cost, making its usage unaffordable, especially for single-veterinarian clinics. In this regard, successful future studies need to consider the practical applicability as well (29).

Despite these gaps, there are prominent opportunities to improve research on ML/DL applications in veterinary medicine. First, privacy issues are relatively minor, when compared with human medicine. In it, data merging between hospitals and clinics is challenging owing to these issues. Therefore, the major approach in human medicine is the common data model which standardizes the data structure of each institution, facilitating meta-analysis (1,76) rather than merged big data analysis. In contrast, multi-clinic data can be merged without privacy issues in veterinary sectors, and the veterinary compass (44) and Small Animal Veterinary Surveillance Network (28,65) showed these opportunities. Furthermore, the cost of data collection in veterinary medicine, especially for continuous data, may be lower than that in human medicine. Recently, the collection of continuous data and extraction of significant signals using wearable devices (18) has become a leading research topic. In these research areas, veterinary medicine has more opportunities than in human medicine, because employing animal subjects costs less than employing human participants; additionally, compliance in applying the device could be higher in animal subjects than in human participants.

Improving the application of ML/DL in veterinary clinics necessitates the fulfillment of two essential conditions. First, the establishment of a standardized encoding system is crucial. To achieve reliable prediction performance, high-quality big data is indispensable. Considering that the medical big data should be collected by multiple institutions, a unified coding system for diseases diagnosis and prescription is essential to successfully amalgamate data from various sources. However, currently, medical records predominantly rely on free text-based descriptions which is challenging to be standardized. Although automatic encoding systems that translate free text to medical codes have been developed (78), no system is customized currently. Secondly, fostering sustainable motivation among veterinarians for accurate recording is important. The absence of a national insurance system for animal medicine has led to a lack of incentives for veterinarians to ensure precise encoding. Addressing this challenge entails appropriately valuating medical records provided by veterinary clinicians. Currently, the value of such data is not accurately evaluated, and most data utilized in ML/DL models have been acquired without enough compensation to veterinarians. Offering proper remuneration for their data contributions could incentivize them to maintain accurate recording practices (Fig. 4).

Figure 4. Current gaps and suggestions for further studies in the studies using machine learning and deep learning in veterinary clinics.

This study has some limitations. First, the reviews were conducted by a single researcher. Because the standard review process generally requires at least two researchers to increase the sensitivity and specificity of the screening process, several studies, that should have been included, could have been excluded. Second, this study included only original papers and other types of publications were excluded. Because studies on state-of-the-art methodologies can be published as conference abstracts, several studies may not have been reviewed in this study. Although this preliminary review study successfully revealed current gaps especially for validation methodologies, further studies are highly recommended to address the limitation, confirm the gaps and support the suggestions in this study. The follow-up studies should employ standard review process with at least two independent researchers and include grey articles that report up-to-date technologies.

In this review, I examined studies that covered the application of ML/DL in veterinary clinics. This revealed several gaps in the methodology and validation, that could help future studies improve their quality and allow readers to better screen appropriate veterinary studies. In the era of artificial intelligence, the expanding demand for their application in veterinary clinics is unavoidable. Furthermore, demand-driven active research using proper methodologies can fundamentally improve clinical services. In this regard, researchers should keep practical feasibility in mind when tackling methodology and model performance; moreover, veterinary clinicians should adopt a receptive and critical stance towards these new changes.

Supplemental Material

Acknowledgements

This work was supported by a funding for the academic research program of Chungbuk National University in 2022. In addition, this work was carried out with the support of “Cooperative Research Program for Agriculture Science and Technology Development (Project No. RS-2023-00232301).“ Rural Development Administration, Republic of Korea.

Conflicts of Interest

The author has no conflicting interests.

Fig 1.

Figure 1.Flow diagram of information through the different phases of the review.
Journal of Veterinary Clinics 2023; 40: 243-259https://doi.org/10.17555/jvc.2023.40.4.243

Fig 2.

Figure 2.Temporal trend of machine learning and deep learning application studies in veterinary clinics by types of target animals. Note: The others group includes wildlife and exotic animals.
Journal of Veterinary Clinics 2023; 40: 243-259https://doi.org/10.17555/jvc.2023.40.4.243

Fig 3.

Figure 3.Proportion of target animal species by the purpose of studies. Note: Numbers include duplicates. For example, a study on industrial animals has the purposes of both diagnostics and prediction.
Journal of Veterinary Clinics 2023; 40: 243-259https://doi.org/10.17555/jvc.2023.40.4.243

Fig 4.

Figure 4.Current gaps and suggestions for further studies in the studies using machine learning and deep learning in veterinary clinics.
Journal of Veterinary Clinics 2023; 40: 243-259https://doi.org/10.17555/jvc.2023.40.4.243

Table 1 General information regarding included studies in the review

Author and yearAnimal typeTarget animalsSample sizeAlgorithm
G. Theodoropoulos et al., 2000 (71)DomesticSheep255 images of 57 individual larvae (5genera)ANN (artificial neural network; feature selection by manual, 16 features were measured)
W. B. Roush et al., 2001 (63)DomesticChickenCase 6-40, normal 33-91BP3(back propagation neural network), WardBP (Ward back propagation neural network), PNN (Probabilistic neural network), GRNN (general regression neural network)
H. Schobesberger and C. Peham, 2002 (66)DomesticHorse175 (42 control/ 133 low to medium grade lame)ANN (feature selection by manual)
K. G. Keegan et al., 2003 (32)DomesticHorse12 adult horseANN (feature selection by manual)
M. E. Pastell and M. Kujalaf, 2007 (56)DomesticDairy cow73 cows (training 37 cows, 5,074 observation, validation 36 cows, 4,868 measurements)Probabilistic Neural Network Model (feature selection by manual)
S. M. Ghotoorlar et al., 2012 (25)DomesticDairy cow105 dairy cowsANN (feature selection by manual)
T. Banzato et al., 2018 (4)CompanionCanine80 (56 meningioma, 24 glioma)Convolutional neural networks (CNN), GoogleNet
T. Banzato et al., 2018 (5)CompanionCanine48 (32 case, 16 control)Deep neural networks (DNN), especially AlexNet
T. Banzato et al., 2018 (6)CompanionCanine56 (grade 1 = 26, grade 2 = 22, grade 3 = 8)AlexNet, DNN
A. Yakubu et al., 2018 (73)DomesticChicken167ANN
Y. Yoon et al., 2018 (75)CompanionDogs3,142 for cardiomegaly (1,571 normal and 1,571 abnormal from 1,143 dogs), 2,086 for lung pattern (1,043 normal and 1,043 abnormal from 1,247 dogs), 892 for mediastinal shift (446 normal and 446 abnormal from 387 dogs), 940 for pleural effusion (470 normal and 470 abnormal from 284 dogs), and 78 for pneumothorax (39 normal and 39 abnormal from 61 dogs)Bag-of-features (BOF) and CNN
R. Bradley et al., 2019 (15)CompanionCat106,251 catsRecurrent Neural Network (RNN)
M. Ebrahimi et al., 2019 (20)DomesticCow297,004 milking samples each with eight milking featuresANN, Naïve Bayes, GLM, Decision tree, Random forest, Gradient boosted tree
J. Y. Kim et al., 2019 (35)CompanionDogs1,040 imagesCNN (GoogLe net, Resnet, and VGGnet)
M. Aubreville et al., 2020 (3)CompanionDogs32 whole slide imagesCNN, RetinaNet, ResNet-18, Unet
V. Biourge et al., 2020 (12)CompanionCats218ANN
L. E. Broughton-Neiswanger et al., 2020 (16)CompanionCats12Partial least squares discriminant analysis, Random forest
S. Burti et al., 2020 (17)CompanionDogs1,465 imagesCNN
E. Fernández-Carrión et al., 2020 (22)Etc.Wild boar8CNN
M. A. Fraiwan and S. M. Abutarbush, 2020 (24)DomesticHorse285 horsesBayes Network, Naïve Bayes, DNN, Random forest
X. Kang et al., 2020 (30)DomesticCow100 cowsRFB_NET_SSD deep learning network
N. Kil et al., 2020 (33)DomesticHorse34 horses (65 video)CNN
S. Li et al., 2020 (39)CompanionDogs792 radiographsCNN
C. Marzahl et al., 2020 (42)DomesticHorse17 completely annotated cytology whole slide images (WSI) containing 78,047 hemosiderophagesCNN (RetinaNet)
S. Mouloodi et al., 2020 (47)DomesticHorse3 third metacarpal bones from 3 racehorsesANN
S. Mouloodi et al., 2020 (48)DomesticHorse9 equine third metacarpal bones from 9 thoroughbred horsesANN
Y. Nagamori et al., 2020 (50)CompanionCat, dogs100CNN
C. Post et al., 2020 (59)DomesticCow167 cowsLogistic Regression (LR), Support Vector Machine (SVM), K-nearest neighbors (KNN), Gaussian Naïve Bayes (GNB), Extra Trees Classifier (ET), Random forest
A. R. Trachtman et al., 2020 (72)DomesticPigs5,902 imagesCNN
T. Banzato et al., 2021 (7)CompanionDogs3,839 latero-lateral radiographsCNN (ResNet-50, DenseNet-121)
T. Banzato et al., 2021 (8)CompanionCat1,062 latero-lateral radiographsCNN (ResNet 50 and Inception V3)
A. Biercher et al., 2021 (11)CompanionDogsThoracolumbar MR images from 500 dogsCNN
E. Boissady et al., 2021 (13)CompanionCat, dogs30 canine and 30 feline thoracic lateral radiographsCNN
L. Bonicelli et al., 2021 (14)DomesticPigs7,564 picturesCNN
V. Kittichai et al., 2021 (36)DomesticPoultry12,761 single cell imagesCNN (Darknet, Darknet19, Darknet19-448 and Densenet201)
Y. Nagamori et al., 2021 (51)CompanionCat, dogs460 samples for 4 parasites (80-200 per parasite)You only look once (YOLOv3) model
J. Park et al., 2021 (54)CompanionDogs90 dogsHA, DLBAS, and the readjustment of the predicted data obtained via the DLBAS of the clinical test sets (HA_DLBAS)
I. R. Porter et al., 2021 (58)DomesticCattleA total of 398 digital images from dairy cows’ uddersCNN (GoogLeNet)
M. Salvi et al., 2021 (64)CompanionDogs416 canine cutaneous round cell tumors (RCT) (117 cases)AlexNet, Inceptionv3, ResNet, Emsemble
S. Shahinfar et al., 2021 (68)DomesticCattle2,535 lameness scores (2,248 sound and 287 unsound)Naïve Bayes (NB), Random Forest (RF) and Multilayer Perceptron (MLP), to predict cases of lameness using milk production and conformation traits logistc (LR)
Y. Ye et al., 2021 (74)CompanionDogs220 imagesCNN (ResNet-50)
M. Zhang et al., 2021 (79)CompanionDogs2,670 lateral X-ray imagesCNN (HRNet)
A.N. ELKhamary et al., 2022 (21)DomesticHorse16 horse 32 limbs (16 normal tendons and 16 abnormal tendons)C4.5 algorithm (Quinlan), a decision tree classifier of Weka software package
E. A. Bauer and W. Jagusiak, 2022 (9)DomesticCattle168 cowsANN
K. Benfodil et al., 2022 (10)DomesticDromedaries115 dromedariesANN
L. Dumortier et al., 2022 (19)CompanionCat500 annotated Thoracic radiograph images(348 veterinary visit 296 cats)CNN (ResNet50V2)
P. Figueirinhas et al., 2022 (23)CompanionDogs15 working dogs (pilot study)LSTM
Y. Kokkinos et al., 2022 (37)CompanionDogs57,402 dogsRNN
A. Mao et al., 2022 (41)DomesticChicken5,336 voice calls (3,363 distress calls and 1,973 natural barn sound)CNN (light-VGG11)
A. May et al., 2022 (43)DomesticHorse2,607 imagesCNN
T. R. Müller et al., 2022 (45)CompanionDogs62 canine (41 case 21 control) 4,000 images (2,000 case 2,000 control)CNN (VGG16)
C. Parra et al., 2022 (55)Etc.Reptile3,616 images data samples and 26 videos (4,849 frames)CNN (MobileNet)
T. Rai et al., 2022 (60)CompanionDogs32 patientsCNN (DenseNet-161)
V. A. Teixeira et al., 2022 (70)DomesticCattle55 Holstein calvesRNN
M. ZareBidaki et al., 2022 (77)DomesticGoat, sheep cows200 paired sample (100 blood, 100 milk) 100 animalsANN

Table 2 Validation methodologies and model performance of the included studies in the review

Author and yearCVProspectiveMulti-center approachModel performancePurpose


Training setTest setIndexValue
G. Theodoropoulos et al., 2000 (71)YesNoNoNoSensitivity42.4-80.7%Diagnostics
W. B. Roush et al., 2001 (63)YesNoNoNoSensitivity0-100%Prediction
H. Schobesberger and C. Peham, 2002 (66)YesNoNoNoAgreement78.60%Diagnostics
K. G. Keegan et al., 2003 (32)YesNoNoNoAgreement85%Diagnostics
M. E. Pastell and M. Kujalaf, 2007 (56)YesNoNoNoAgreement and sensitivityAgreement = 96.2%
Sensitivity = 100%
Diagnostics
S. M. Ghotoorlar et al., 2012 (25)YesNoNoNoSensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), Pearson correlation coefficientSensitivity = 0.5-1Specificity = 0.91-1PPV = 0.76-1NPV = 0.92 -1Pearson correlation coefficient = 0.94Diagnostics
T. Banzato et al., 2018 (4)YesNoYesNoAgreement, Matthews correlation coefficient (MCC)Agreement = 90-94%
MCC = 0.8-0.88
Diagnostics
T. Banzato et al., 2018 (5)YesNoNoNoAUC, sensitivity, specificityAUC = 0.91
Sensitivity = 100%
Specificity = 82.8%
Diagnostics
T. Banzato et al., 2018 (6)YesNoYesNoAgreement, multi-class Matthew’s correlation coefficient (MCMCC)Agreement = 65.2-82.2%
MCMCC = 0.44-0.68
Diagnostics
A. Yakubu et al., 2018 (73)YesNoNoNor, R2, RMSEr = 0.983
R2 = 0.966
RMSE = 0.04806
Prediction
Y. Yoon et al., 2018 (75)YesNoNoNoAccuracy, sensitivityAccuracy(CNN; 92.9-96.9% and BOF; 79.6-96.9%) and sensitivity (CNN; 92.1-100% and BOF; 74.1-94.8%)Prediction
R. Bradley et al., 2019 (15)YesNoNoNoSensitivity, specificity(1 year before) sensitivity 63.0%; (2 year before) sensitivity 44.2% specificity remaining around 99%Prediction
M. Ebrahimi et al., 2019 (20)YesNoNoNoAUC0.826Prediction
J. Y. Kim et al., 2019 (35)YesNoYesNoSensitivity79.4-100%Diagnostics
M. Aubreville et al., 2020 (3)YesNoNoNoCorrelation coefficient0.868-0.979Diagnostics
V. Biourge et al., 2020 (12)YesYesNoYesAccuracy, sensitivity, specificity, PPV, NPVAccuracy = 88%
Sensitivity = 87%
Specificity = 70%
PPV = 53%
NPV = 92%
Prediction
L. E. Broughton-Neiswanger et al., 2020 (16)YesNoNoNoSensitivity, specificity, AUCAUC = 0.87-1Sensitivity = 0-100%Specificity = 50-100%Diagnostics
S. Burti et al., 2020 (17)YesNoNoNoAUC0.904-0.973Diagnostics
E. Fernández-Carrión et al., 2020 (22)YesNoNoNoAgreement95.4-97.2%Diagnostics
M. A. Fraiwan and S. M. Abutarbush, 2020 (24)YesNoNoNoPrecision, recall, F-measure, Accuracy(need for surgery)Precision = 69.5-74.1%Recall = 72.4-99.3%F-measure = 72.2-81.8%Accuracy = 69.0-76.0%(survival)Precision = 87.5-97.4%Recall = 80.5-87.8%F-measure = 87.2-89.1%Accuracy = 83.9-85.2%Prediction
X. Kang et al., 2020 (30)YesNoNoNoSensitivity, specificitySensitivity = 0.83-1Specificity = 0.95-1Diagnostics
N. Kil et al., 2020 (33)YesNoNoNoSensitivity, accuracySensitivity = 0.79-0.94Accuracy = 0.82-0.94Diagnostics
S. Li et al., 2020 (39)YesNoNoNoAccuracy, sensitivity, and specificityAccuracy = 82.71%
Sensitivity = 68.42%
Specificity = 87.09%
Diagnostics
C. Marzahl et al., 2020 (42)YesNoNoNoPrecision0.64-0.66Diagnostics
S. Mouloodi et al., 2020 (47)YesNoNoNoDetermination coefficient (R2)0.9116-0.9599Prediction
S. Mouloodi et al., 2020 (48)YesNoNoNoDetermination coefficient (R2)0.9999Prediction
Y. Nagamori et al., 2020 (50)YesNoNoNoPearson correlation coefficient, sensitivity, specificityPearson correlation coefficient = 0.89-0.99Sensitivity = 0.758-1Specificity = 0.918-1Diagnostics
C. Post et al., 2020 (59)YesNoNoNoAUC0.71-0.79Diagnostics
A. R. Trachtman et al., 2020 (72)YesNoNoNoAccuracy, sensitivity, specificityAccuracy = 62-96%Sensitivity = 84-100%Specificity = 92-96%Diagnostics
T. Banzato et al., 2021 (7)YesNoNoNoAUC0.8Diagnostics
T. Banzato et al., 2021 (8)YesNoYesNoAUC0.58-0.97Diagnostics
A. Biercher et al., 2021 (11)YesNoYesYesSensitivity, specificityIVDE sens 73.46-90.1/spec 67.6-99.0IVDP sens 67.86-100/spec 74.9-96.4FCE/ANNPE sens 62.2-90.1/spec 90.1-97.9Syringomyelia sens 0-10/spec 100Neoplasma sens 0-37.5/spec 60-94.7Diagnostics
E. Boissady et al., 2021 (13)NANoNoNoICC0.998-0.999Diagnostics
L. Bonicelli et al., 2021 (14)YesNoYesYesSensitivity, specificity, Pearson correlation coefficientSensitivity = 81.25-100 %
Specificity = 99.38 %
Pearson correlation coefficient = 0.96
Diagnostics
V. Kittichai et al., 2021 (36)YesNoNANAAccuracy99%Dignostics
Y. Nagamori et al., 2021 (51)NANAYESNASensitivity, specificitySensitivity = 75.8-100%
Specificity = 93.1-100%
Dignostics
J. Park et al., 2021 (54)YesNoNoNoDice similarity coefficient (DSC) and the Hausdorff distance (HD)DSC 0.78-0.94
HD 2.30-4.30 mm
Dignostics
I. R. Porter et al., 2021 (58)YesNoYesYesAUC0.542-0.920Dignostics
M. Salvi et al., 2021 (64)YesNoYesYesAccuracy91.66%-100%Dignostics
S. Shahinfar et al., 2021 (68)YesNoYesYesAUC, F1AUC = 0.61-0.67
F1 = 0.01-0.27
Dignostics
Y. Ye et al., 2021 (74)YesNoNANAAUC, accuracy, F1 scoreAUC = 99.37Accuracy = 97.62 F1 score = 96.7Dignostics
M. Zhang et al., 2021 (79)YesNoYesYesSensitivity86.40%Dignostics
A.N. ELKhamary et al., 2022 (21)YesNoNoNoAccuracy, PPV, sensitivity, kappaAccuracy = 93.7%
PPV = 93.80%
Sensitivity = 93.80%
Kappa = 0.88
Dignostics
E. A. Bauer and W. Jagusiak, 2022 (9)YesNoYESYESAUC0.82-0.89Dignostics
K. Benfodil et al., 2022 (10)YesNoNANAPearson correlation coefficient0.943Dignostics
L. Dumortier et al., 2022 (19)YesNoNoNoAccuracy, F1-Score, Specificity, Positive Predictive Value and SensitivityAccuracy = 82%
F1-Score = 85%
Specificity = 75%
PPV = 81%
Sensitivity = 88%
Dignostics
P. Figueirinhas et al., 2022 (23)YesNoNoNoAccuracyAccuracy = 60%Dignostics
Y. Kokkinos et al., 2022 (37)YesNoNoNoSensitivity, PPV, NPVSensitivity = 44.8-68.8% PPV = 15-23% NPV > 99%Prediction
A. Mao et al., 2022 (41)YesNoYesYesPrecision, recall, F1-score and accuracyPrecision = 94.58%
Recall = 94.89%
F1-score = 94.73%
Accuracy = 95.07%
Dignostics
A. May et al., 2022 (43)YesNoNoNoAccuracy, cross entropyAccuracy = 96.66%
Cross entropy = 0.02
Dignostics
T. R. Müller et al., 2022 (45)YesNoNoNoAccuracy, sens, spec, PPV, NPVAccuracy 88.7%
Sensitivity 90.2%
Specificity 81.8%
PPV 92.5%
NPV 81.8%
Dignostics
C. Parra et al., 2022 (55)YesNANANAAccuracy, AUCAccuracy = 94.26
AUC = 0.996
Dignostics
T. Rai et al., 2022 (60)YesNoNoNoF1-score0.708Dignostics
V. A. Teixeira et al., 2022 (70)YesNoNoNoAccuracy, sensitivity, and specificity, PPV, NPVAccuracy = 85-98,
Sensitivity = 87-96
Specificity = 78-100
PPV = 85-100
NPV = 88-96
Prediction & diagnosis
M. ZareBidaki et al., 2022 (77)YesNoNANASensitivity, specificity, AUCSensitivity = 81%
Specificity = 62%
AUC = 0.799
Dignostics

Table 3 Profile of included studies

Target animal typeStudy purposesN*NNCVPros§Multi∥
Industrial animalsDiagnostics21192105
Prediction77700
Companion animalsDiagnostics22212002
Prediction44411
OthersDiagnostics22200

*Number of studies.

Number of studies that used neural network-based algorithm.

Number of studies that conducted cross-validation approach to measure performance.

§Number of studies that employed prospective approach for collecting dataset for testing.

Number of studies that used multi-center data for validation.

The others group includes wildlife and exotic animals.

Note: The numbers include duplication. For example, a study for industrial animals have both purpose, diagnostics and prediction. There is no prediction studies for the other animals.


References

  1. Ahmadi N, Peng Y, Wolfien M, Zoch M, Sedlmayr M. OMOP CDM can facilitate data-driven studies for cancer prediction: a systematic review. Int J Mol Sci 2022; 23: 11834.
    Pubmed KoreaMed CrossRef
  2. Ali O, Abdelbaki W, Shrestha A, Elbasi E, Alryalat MAA, Dwivedi YK. A systematic literature review of artificial intelligence in the healthcare sector: benefits, challenges, methodologies, and functionalities. J Innov Knowl 2023; 8: 100333.
    CrossRef
  3. Aubreville M, Bertram CA, Marzahl C, Gurtner C, Dettwiler M, Schmidt A, et al. Deep learning algorithms out-perform veterinary pathologists in detecting the mitotically most active tumor region. Sci Rep 2020; 10: 16447.
    Pubmed KoreaMed CrossRef
  4. Banzato T, Bernardini M, Cherubini GB, Zotti A. A methodological approach for deep learning to distinguish between meningiomas and gliomas on canine MR-images. BMC Vet Res 2018; 14: 317.
    Pubmed KoreaMed CrossRef
  5. Banzato T, Bonsembiante F, Aresu L, Gelain ME, Burti S, Zotti A. Use of transfer learning to detect diffuse degenerative hepatic diseases from ultrasound images in dogs: a methodological study. Vet J 2018; 233: 35-40.
    Pubmed CrossRef
  6. Banzato T, Cherubini GB, Atzori M, Zotti A. Development of a deep convolutional neural network to predict grading of canine meningiomas from magnetic resonance images. Vet J 2018; 235: 90-92.
    Pubmed CrossRef
  7. Banzato T, Wodzinski M, Burti S, Osti VL, Rossoni V, Atzori M, et al. Automatic classification of canine thoracic radiographs using deep learning. Sci Rep 2021; 11: 3964.
    Pubmed KoreaMed CrossRef
  8. Banzato T, Wodzinski M, Tauceri F, Donà C, Scavazza F, Müller H, et al. An AI-based algorithm for the automatic classification of thoracic radiographs in cats. Front Vet Sci 2021; 8: 731936.
    Pubmed KoreaMed CrossRef
  9. Bauer EA, Jagusiak W. The use of multilayer perceptron artificial neural networks to detect dairy cows at risk of ketosis. Animals (Basel) 2022; 12: 332.
    Pubmed KoreaMed CrossRef
  10. Benfodil K, Benbouras MA, Ansel S, Mohamed-Cherif A, Ait-Oudhia K. Prediction of Trypanosoma evansi infection in dromedaries using artificial neural network (ANN). Vet Parasitol 2022; 306: 109716.
    Pubmed CrossRef
  11. Biercher A, Meller S, Wendt J, Caspari N, Schmidt-Mosig J, De Decker S, et al. Using deep learning to detect spinal cord diseases on thoracolumbar magnetic resonance images of dogs. Front Vet Sci 2021; 8: 721167.
    Pubmed KoreaMed CrossRef
  12. Biourge V, Delmotte S, Feugier A, Bradley R, McAllister M, Elliott J. An artificial neural network-based model to predict chronic kidney disease in aged cats. J Vet Intern Med 2020; 34: 1920-1931.
    Pubmed KoreaMed CrossRef
  13. Boissady E, De La Comble A, Zhu X, Abbott J, Adrien-Maxence H. Comparison of a deep learning algorithm vs. humans for vertebral heart scale measurements in cats and dogs shows a high degree of agreement among readers. Front Vet Sci 2021; 8: 764570.
    Pubmed KoreaMed CrossRef
  14. Bonicelli L, Trachtman AR, Rosamilia A, Liuzzo G, Hattab J, Mira Alcaraz E, et al. Training convolutional neural networks to score pneumonia in slaughtered pigs. Animals (Basel) 2021; 11: 3290.
    Pubmed KoreaMed CrossRef
  15. Bradley R, Tagkopoulos I, Kim M, Kokkinos Y, Panagiotakos T, Kennedy J, et al. Predicting early risk of chronic kidney disease in cats using routine clinical laboratory tests and machine learning. J Vet Intern Med 2019; 33: 2644-2656.
    Pubmed KoreaMed CrossRef
  16. Broughton-Neiswanger LE, Rivera-Velez SM, Suarez MA, Slovak JE, Piñeyro PE, Hwang JK, et al. Urinary chemical fingerprint left behind by repeated NSAID administration: discovery of putative biomarkers using artificial intelligence. PLoS One 2020; 15: e0228989.
    Pubmed KoreaMed CrossRef
  17. Burti S, Longhin Osti V, Zotti A, Banzato T. Use of deep learning to detect cardiomegaly on thoracic radiographs in dogs. Vet J 2020; 262: 105505.
    Pubmed CrossRef
  18. Dinh-Le C, Chuang R, Chokshi S, Mann D. Wearable health technology and electronic health record integration: scoping review and future directions. JMIR Mhealth Uhealth 2019; 7: e12861.
    Pubmed KoreaMed CrossRef
  19. Dumortier L, Guépin F, Delignette-Muller ML, Boulocher C, Grenier T. Deep learning in veterinary medicine, an approach based on CNN to detect pulmonary abnormalities from lateral thoracic radiographs in cats. Sci Rep 2022; 12: 11418.
    Pubmed KoreaMed CrossRef
  20. Ebrahimi M, Mohammadi-Dehcheshmeh M, Ebrahimie E, Petrovski KR. Comprehensive analysis of machine learning models for prediction of sub-clinical mastitis: deep learning and gradient-boosted trees outperform other models. Comput Biol Med 2019; 114: 103456.
    Pubmed CrossRef
  21. ELKhamary AN, Keenihan EK, Schnabel LV, Redding WR, Schumacher J. Leveraging MRI characterization of longitudinal tears of the deep digital flexor tendon in horses using machine learning. Vet Radiol Ultrasound 2022; 63: 580-592.
    Pubmed CrossRef
  22. Fernández-Carrión E, Barasona JÁ, Sánchez Á, Jurado C, Cadenas-Fernández E, Sánchez-Vizcaíno JM. Computer vision applied to detect lethargy through animal motion monitoring: a trial on African swine fever in wild boar. Animals (Basel) 2020; 10: 2241.
    Pubmed KoreaMed CrossRef
  23. Figueirinhas P, Sanchez A, Rodríguez O, Vilar JM, Rodríguez-Altónaga J, Gonzalo-Orden JM, et al. Development of an artificial neural network for the detection of supporting hindlimb lameness: a pilot study in working dogs. Animals (Basel) 2022; 12: 1755.
    Pubmed KoreaMed CrossRef
  24. Fraiwan MA, Abutarbush SM. Using artificial intelligence to predict survivability likelihood and need for surgery in horses presented with acute abdomen (Colic). J Equine Vet Sci 2020; 90: 102973.
    Pubmed CrossRef
  25. Ghotoorlar SM, Ghamsari SM, Nowrouzian I, Ghotoorlar SM, Ghidary SS. Lameness scoring system for dairy cows using force plates and artificial intelligence. Vet Rec 2012; 170: 126.
    Pubmed CrossRef
  26. Hennessey E, DiFazio M, Hennessey R, Cassel N. Artificial intelligence in veterinary diagnostic imaging: a literature review. Vet Radiol Ultrasound 2022; 63 Suppl 1: 851-870.
    Pubmed CrossRef
  27. Hwang EJ, Park S, Jin KN, Kim JI, Choi SY, Lee JH, et al. Development and validation of a deep learning-based automated detection algorithm for major thoracic diseases on chest radiographs. JAMA Netw Open 2019; 2: e191095. Erratum in: JAMA Netw Open 2019; 2: e193260.
    Pubmed KoreaMed CrossRef
  28. Jones PH, Dawson S, Gaskell RM, Coyne KP, Tierney A, Setzkorn C, et al. Surveillance of diarrhoea in small animal practice through the Small Animal Veterinary Surveillance Network (SAVSNET). Vet J 2014; 201: 412-418.
    Pubmed CrossRef
  29. Joslyn S, Alexander K. Evaluating artificial intelligence algorithms for use in veterinary radiology. Vet Radiol Ultrasound 2022; 63 Suppl 1: 871-879.
    Pubmed CrossRef
  30. Kang X, Zhang XD, Liu G. Accurate detection of lameness in dairy cattle with computer vision: a new and individualized detection strategy based on the analysis of the supporting phase. J Dairy Sci 2020; 103: 10628-10638.
    Pubmed CrossRef
  31. Kaul V, Enslin S, Gross SA. History of artificial intelligence in medicine. Gastrointest Endosc 2020; 92: 807-812.
    Pubmed CrossRef
  32. Keegan KG, Arafat S, Skubic M, Wilson DA, Kramer J. Detection of lameness and determination of the affected forelimb in horses by use of continuous wavelet transformation and neural network classification of kinematic data. Am J Vet Res 2003; 64: 1376-1381.
    Pubmed CrossRef
  33. Kil N, Ertelt K, Auer U. Development and validation of an automated video tracking model for stabled horses. Animals (Basel) 2020; 10: 2258.
    Pubmed KoreaMed CrossRef
  34. Kim DW, Jang HY, Kim KW, Shin Y, Park SH. Design characteristics of studies reporting the performance of artificial intelligence algorithms for diagnostic analysis of medical images: results from recently published papers. Korean J Radiol 2019; 20: 405-410.
    Pubmed KoreaMed CrossRef
  35. Kim JY, Lee HE, Choi YH, Lee SJ, Jeon JS. CNN-based diagnosis models for canine ulcerative keratitis. Sci Rep 2019; 9: 14209.
    Pubmed KoreaMed CrossRef
  36. Kittichai V, Kaewthamasorn M, Thanee S, Jomtarak R, Klanboot K, Naing KM, et al. Classification for avian malaria parasite Plasmodium gallinaceum blood stages by using deep convolutional neural networks. Sci Rep 2021; 11: 16919.
    Pubmed KoreaMed CrossRef
  37. Kokkinos Y, Morrison J, Bradley R, Panagiotakos T, Ogeer J, Chew D, et al. An early prediction model for canine chronic kidney disease based on routine clinical laboratory tests. Sci Rep 2022; 12: 14489.
    Pubmed KoreaMed CrossRef
  38. Krone LM, Brown CM, Lindenmayer JM. Survey of electronic veterinary medical record adoption and use by independent small animal veterinary medical practices in Massachusetts. J Am Vet Med Assoc 2014; 245: 324-332.
    Pubmed KoreaMed CrossRef
  39. Li S, Wang Z, Visser LC, Wisner ER, Cheng H. Pilot study: application of artificial intelligence for detecting left atrial enlargement on canine thoracic radiographs. Vet Radiol Ultrasound 2020; 61: 611-618.
    Pubmed KoreaMed CrossRef
  40. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019; 1: e271-e297. Erratum in: Lancet Digit Health 2019; 1: e334.
    Pubmed CrossRef
  41. Mao A, Giraudet CSE, Liu K, De Almeida Nolasco I, Xie Z, Xie Z, et al. Automated identification of chicken distress vocalizations using deep learning models. J R Soc Interface 2022; 19: 20210921.
    Pubmed KoreaMed CrossRef
  42. Marzahl C, Aubreville M, Bertram CA, Stayt J, Jasensky AK, Bartenschlager F, et al. Deep learning-based quantification of pulmonary hemosiderophages in cytology slides. Sci Rep 2020; 10: 9795.
    Pubmed KoreaMed CrossRef
  43. May A, Gesell-May S, Müller T, Ertel W. Artificial intelligence as a tool to aid in the differentiation of equine ophthalmic diseases with an emphasis on equine uveitis. Equine Vet J 2022; 54: 847-855.
    Pubmed CrossRef
  44. McGreevy P, Thomson P, Dhand NK, Raubenheimer D, Masters S, Mansfield CS, et al. VetCompass Australia: a national big data collection system for veterinary science. Animals (Basel) 2017; 7: 74.
    Pubmed KoreaMed CrossRef
  45. Meller S, Zamansky A, Sinitca A, Kaplun D, Meyerhoff N, Stein V, et al. Sounds of seizures-acoustic information enables immediate recognition and detection of generalized tonic-clonic seizures in dogs. J Vet Intern Med 2022; 36: 305.
  46. Motahari-Nezhad H, Fgaier M, Mahdi Abid M, Péntek M, Gulácsi L, Zrubka Z. Digital biomarker-based studies: scoping review of systematic reviews. JMIR Mhealth Uhealth 2022; 10: e35722.
    Pubmed KoreaMed CrossRef
  47. Mouloodi S, Rahmanpanah H, Burvill C, Davies HMS. Prediction of displacement in the equine third metacarpal bone using a neural network prediction algorithm. Biocybern Biomed Eng 2020; 40: 849-863.
    CrossRef
  48. Mouloodi S, Rahmanpanah H, Burvill C, Davies HMS. Prediction of load in a long bone using an artificial neural network prediction algorithm. J Mech Behav Biomed Mater 2020; 102: 103527.
    Pubmed CrossRef
  49. Muehlematter UJ, Daniore P, Vokinger KN. Approval of artificial intelligence and machine learning-based medical devices in the USA and Europe (2015-20): a comparative analysis. Lancet Digit Health 2021; 3: e195-e203.
    Pubmed CrossRef
  50. Nagamori Y, Hall Sedlak R, DeRosa A, Pullins A, Cree T, Loenser M, et al. Evaluation of the VETSCAN IMAGYST: an in-clinic canine and feline fecal parasite detection system integrated with a deep learning algorithm. Parasit Vectors 2020; 13: 346.
    Pubmed KoreaMed CrossRef
  51. Nagamori Y, Sedlak RH, DeRosa A, Pullins A, Cree T, Loenser M, et al. Further evaluation and validation of the VETSCAN IMAGYST: in-clinic feline and canine fecal parasite detection system integrated with a deep learning algorithm. Parasit Vectors 2021; 14: 89.
    Pubmed KoreaMed CrossRef
  52. Nagendran M, Chen Y, Lovejoy CA, Gordon AC, Komorowski M, Harvey H, et al. Artificial intelligence versus clinicians: systematic review of design, reporting standards, and claims of deep learning studies. BMJ 2020; 368: m689.
    Pubmed KoreaMed CrossRef
  53. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021; 372: n71.
    Pubmed KoreaMed CrossRef
  54. Park J, Choi B, Ko J, Chun J, Park I, Lee J, et al. Deep-learning-based automatic segmentation of head and neck organs for radiation therapy in dogs. Front Vet Sci 2021; 8: 721612.
    Pubmed KoreaMed CrossRef
  55. Parra C, Grijalva F, Núñez B, Núñez A, Pérez N, Benítez D. Automatic identification of intestinal parasites in reptiles using microscopic stool images and convolutional neural networks. PLoS One 2022; 17: e0271529.
    Pubmed KoreaMed CrossRef
  56. Pastell ME, Kujala M. A probabilistic neural network model for lameness detection. J Dairy Sci 2007; 90: 2283-2292.
    Pubmed CrossRef
  57. Pichler M, Hartig F. Machine learning and deep learning—a review for ecologists. Methods Ecol Evol 2023; 14: 994-1016.
    CrossRef
  58. Porter IR, Wieland M, Basran PS. Feasibility of the use of deep learning classification of teat-end condition in Holstein cattle. J Dairy Sci 2021; 104: 4529-4536.
    Pubmed CrossRef
  59. Post C, Rietz C, Büscher W, Müller U. Using sensor data to detect lameness and mastitis treatment events in dairy cows: a comparison of classification models. Sensors (Basel) 2020; 20: 3863.
    Pubmed KoreaMed CrossRef
  60. Rai T, Morisi A, Bacci B, Bacon NJ, Dark MJ, Aboellail T, et al. Deep learning for necrosis detection using canine Perivascular Wall Tumour whole slide images. Sci Rep 2022; 12: 10634.
    Pubmed KoreaMed CrossRef
  61. Rajpurkar P, Chen E, Banerjee O, Topol EJ. AI in health and medicine. Nat Med 2022; 28: 31-38.
    Pubmed CrossRef
  62. Rose N, Toews L, Pang DS. A systematic review of clinical audit in companion animal veterinary medicine. BMC Vet Res 2016; 12: 40.
    Pubmed KoreaMed CrossRef
  63. Roush WB, Wideman RF Jr, Cahaner A, Deeb N, Cravener TL. Minimal number of chicken daily growth velocities for artificial neural network detection of pulmonary hypertension syndrome (PHS). Poult Sci 2001; 80: 254-259.
    Pubmed CrossRef
  64. Salvi M, Molinari F, Iussich S, Muscatello LV, Pazzini L, Benali S, et al. Histopathological classification of canine cutaneous round cell tumors using deep learning: a multi-center study. Front Vet Sci 2021; 8: 640944.
    Pubmed KoreaMed CrossRef
  65. Sánchez-Vizcaíno F, Jones PH, Menacere T, Heayns B, Wardeh M, Newman J, et al. Small animal disease surveillance. Vet Rec 2015; 177: 591-594.
    Pubmed CrossRef
  66. Schobesberger H, Peham C. Computerized detection of supporting forelimb lameness in the horse using an artificial neural network. Vet J 2002; 163: 77-84.
    Pubmed CrossRef
  67. Secinaro S, Calandra D, Secinaro A, Muthurangu V, Biancone P. The role of artificial intelligence in healthcare: a structured literature review. BMC Med Inform Decis Mak 2021; 21: 125.
    Pubmed KoreaMed CrossRef
  68. Shahinfar S, Khansefid M, Haile-Mariam M, Pryce JE. Machine learning approaches for the prediction of lameness in dairy cows. Animal 2021; 15: 100391.
    Pubmed CrossRef
  69. Song KD, Kim M, Do S. The latest trends in the use of deep learning in radiology illustrated through the stages of deep learning algorithm development. J Korean Soc Radiol 2019; 80: 202-212.
    CrossRef
  70. Teixeira VA, Lana AMQ, Bresolin T, Tomich TR, Souza GM, Furlong J, et al. Using rumination and activity data for early detection of anaplasmosis disease in dairy heifer calves. J Dairy Sci 2022; 105: 4421-4433.
    Pubmed CrossRef
  71. Theodoropoulos G, Loumos V, Anagnostopoulos C, Kayafas E, Martinez-Gonzales B. A digital image analysis and neural network based system for identification of third-stage parasitic strongyle larvae from domestic animals. Comput Methods Programs Biomed 2000; 62: 69-76.
    Pubmed CrossRef
  72. Trachtman AR, Bergamini L, Palazzi A, Porrello A, Capobianco Dondona A, Del Negro E, et al. Scoring pleurisy in slaughtered pigs using convolutional neural networks. Vet Res 2020; 51: 51.
    Pubmed KoreaMed CrossRef
  73. Yakubu A, Oluremi OIA, Ekpo EI. Predicting heat stress index in Sasso hens using automatic linear modeling and artificial neural network. Int J Biometeorol 2018; 62: 1181-1186.
    Pubmed CrossRef
  74. Ye Y, Sun WW, Xu RX, Selmic LE, Sun M. Intraoperative assessment of canine soft tissue sarcoma by deep learning enhanced optical coherence tomography. Vet Comp Oncol 2021; 19: 624-631.
    Pubmed CrossRef
  75. Yoon Y, Hwang T, Lee H. Prediction of radiographic abnormalities by the use of bag-of-features and convolutional neural networks. Vet J 2018; 237: 43-48.
    Pubmed CrossRef
  76. You SC, Lee S, Choi B, Park RW. Establishment of an international evidence sharing network through common data model for cardiovascular research. Korean Circ J 2022; 52: 853-864.
    Pubmed KoreaMed CrossRef
  77. ZareBidaki M, Allahyari E, Zeinali T, Asgharzadeh M. Occurrence and risk factors of brucellosis among domestic animals: an artificial neural network approach. Trop Anim Health Prod 2022; 54: 62.
    Pubmed CrossRef
  78. Zech JR, Badgeley MA, Liu M, Costa AB, Titano JJ, Oermann EK. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med 2018; 15: e1002683.
    Pubmed KoreaMed CrossRef
  79. Zhang M, Zhang K, Yu D, Xie Q, Liu B, Chen D, et al. Computerized assisted evaluation system for canine cardiomegaly via key points detection with deep learning. Prev Vet Med 2021; 193: 105399.
    Pubmed CrossRef
  80. Zhang Y, Nie A, Zehnder A, Page RL, Zou J. VetTag: improving automated veterinary diagnosis coding via large-scale language modeling. NPJ Digit Med 2019; 2: 35.
    Pubmed KoreaMed CrossRef

Vol.41 No.5 October 2024

qrcode
qrcode
The Korean Society of Veterinary Clinics

pISSN 1598-298X
eISSN 2384-0749

Supplementary

Stats or Metrics

Share this article on :

  • line