Performance comparison among 3 BERT variants (with their optimal parameters) on the validation data set
| Model | No of Epochs | Batch Size | Learning Rate | Macro-Sensitivity (%) (95% CI) | Macro-Specificity (%) (95% CI) | Macro-AUC (95% CI) |
|---|---|---|---|---|---|---|
| BERT-base | 4 | 24 | 0.00003 | 79.46 (68.02–87.82) | 93.65 (89.26–96.46) | 0.928 (0.883–0.973) |
| BioBERT | 6 | 32 | 0.00003 | 79.08 (67.13–87.78) | 93.13 (88.58–96.13) | 0.931 (0.886–0.975) |
| ClinicalBERT | 5 | 32 | 0.00005 | 78.52 (66.91–87.07) | 93.19 (88.57–96.25) | 0.925 (0.878–0.971) |
Note:—Macro indicates the average on the macro level across different categories.