Cataract is the leading cause of blindness globally. The burden is significantly higher in low- and middle-income countries (LMICs). Surgical outcomes in these countries are often poor due to limited sources, inadequate training, insufficient infrastructure, and challenges in managing surgical complications. Small incision cataract surgery (SICS) is the most cost-effective method in LMICs. But there aren’t enough instruments available to assess its quality. While video-based evaluation enhances outcomes, it remains costly. Automated video annotation using deep learning (DL) using the MS-TCN + + architecture has been explored for the phacoemulsification process using the well-established cataract-101 public dataset. However, no such system exists for SICS.
Researchers introduce the first public SICS dataset, “SICS-105,” annotated by four Indian ophthalmologists. This enables artificial intelligence (AI)-based phase segmentation and surgical quality assessment. The current study, published in Scientific Reports, aimed at improving training and surgical outcomes through accessible and automated analysis.
In a prospective cross-sectional study, the cataract-101 dataset contained 101 surgeries performed by four Klinikum Klagenfurt, Austria surgeons. A total of 105 patients were included in the novel SICS-105 from Shankara Eye Hospital, India. The performance of MS-TCN++ was evaluated using various parameters such as precision-recall area under curve (PR-AUC), receiver operator AUC (ROC-AUC), phase-level edit score, frame-wise accuracy, F1-score, and sensitivity, along with specificity.
The MS-TCN + + architecture on the novel SICS-105 dataset showed a lambda value of 0.35, a high accuracy of 89.83%, and a better ROC-AUC of 98.82%. After hyperparameter optimization, results from both video sets were compared using held-out test videos, and 15 videos from cataract-101 and 17 from SICS-105 were observed that were not used during the training.
MS-TCN + + performance was slightly better on the cataract-101 compared to the SICS-105 with an edit score of 84.33 vs 84.52%, ROC-AUC score of 99.10 vs 98.26%, and accuracy of 89.97% vs 85.56%. Both the specificity and sensitivity were comparable between the two datasets at 97.49 vs. 97.56% and 89.97 vs. 85.56%. The 95% confidence intervals were narrow and showed a considerable overlap across all measurements. Across all phases, the PR-AUC scores varied from 45.20% to 93.18%. Cataract-101 videos regularly showed higher and more consistent scores of PR from 68.12% to 97.36%.
Furthermore, a significant negative association was observed between prediction performance and the video length in the SICS-105 dataset with a Spearman coefficient of –0.396 and p < 0.01. A similar significant correlation was observed in the cataract-101 dataset with a coefficient of –0.681 and p < 0.01. The feature extraction took 20 hours for cataract-101 and 26 hours for SICS-105 on a TITAN X GPU. MS-TCN + + training lasted 1 hour 57 minutes for Cataract-101 and 2 hours 2 minutes, respectively.
This study’s limitations include the absence of external SICS datasets, reducing generalizability, variations in surgical procedures, and reliance on a single hospital’s protocol that restricts the broader applicability. Lack of smartphone-based data, slow feature extraction, and compressed video quality also affect the performance of the MS-TCN + + architecture.
In conclusion, this study highlights that larger datasets and more advanced DL models could reduce the performance difference. This study marks a first step toward automatic assessment of SICS, aiming to improve training, enhance safety, and lower costs in LMICs. The novel SICS-105 dataset is expected to be made publicly available to support comparison and reproducibility for future research.
Reference: Mueller S, Sachdeva B, Prasad SN, et al. Phase recognition in manual small-incision cataract surgery with MS-TCN++ on the novel SICS-105 dataset. Sci Rep. 2025;15:16886. doi:10.1038/s41598-025-00303-z
