LapBot is a game aimed at looking at how AI annotations of safe and not safe dissection zones in laparoscopic cholosystechtomy (i.e. gallbladder removal) can impact training.
Laparoscopic cholecystectomy is one of the most commonly performed procedures in the United States and across the world. Within this context, we focus on AI annotations of safe (Go) and dangerous (No-Go) zones of dissection at a given point in surgery. Madani et al. (2022) developed deep learning models to identify Go and No-Go zones of dissection, liver, gallbladder, and hepatocystic triangle during laparoscopic cholecystectomy. In their work, deep learning was used to develop an AI model to accurately identify Go and No-Go zones. The model was trained on annotations of 2627 random frames from 290 LC videos, procured from 37 countries, 136 institutions, and 153 surgeons. The mean IOU (intersection over union), F1 score (uses the harmonic mean of a classifier to combine precision and recall into a single metric, accuracy, sensitivity, and specificity for the AI to identify Go zones was 0.53 (±0.24), 0.70 (±0.28), 0.94 (±0.05), 0.69 (±0.20). and 0.94 (±0.03), respectively. For No-Go zones, these metrics were 0.71 (±0.29), 0.83 (±0.31), 0.95 (±0.06), 0.80 (±0.21), and 0.98 (±0.05), respectively. The Go and No-Go zones can be visualized on surgical scenes using augmented reality (AR) for training or surgical guidance. With augmented reality virtual elements, in this case the visual representation of the AI model results of where it is safe and unsafe to dissect, are overlaid on the real surgical scenes.
In the current version of the LapBot game, the task of the player is given a still image taken from laparoscopic videos during LC surgery, to determine where they would dissect next. The player is able watch a few seconds of the annotated video (if they wish to) to get a better understanding of the surgical context. A score is then calculated based on the distance of their chosen surgical target point and the Go/No Go Zone determined by an AI model (Madani et al. 2022). Given their score the player then receives feedback based on the AI annotations of the Go and No-Go zones for that surgical image in order to understand why they received a given score. The game currently has 6 levels with increasing difficulty. The game was developed to investigate if feedback from the AI annotations of Go and No-Go zones improve learning curves or surgical trainees.
Related Publications:
A. Madani et al., “Artificial intelligence for intraoperative guidance: using semantic segmentation to identify surgical anatomy during laparoscopic cholecystectomy,” Annals of surgery, 2022.