Abstract

For avoiding collision while operating with articulate laparoscopic manipulator system, the position of the instruments should be provided in real-time. Our proposed method provides a reasonable solution for instrument tracking in vivo.

What, How, Results

What

  • A CNN (moded U-Net) based for detecting medical instruments in vivo
  • using binary distribution and gaussian distribution as the label (Fig. 1)
    Labeling
    Fig. 1 Labeling of the joins

How

  • Transformed the model to FP16-format with TensorRT, and loaded by C++ script
  • Accelerated the support-algorithms

Results

  • Speed optimization: from 8.7 fps to 44 fps (Fig. 2)
  • Accuracy improvement: 25% less pixel error and about 10% higher recall and precision (Fig. 3)
  • Experimented sensor-fusion for a 3D-reconstruction
    Runtime
    Fig. 2 Runtime before optimization (blue) and after (green). Left: CNN prediction phase; Right: labeling center extraction
Comparison
Fig. 3 Recall, Precision, and Pixel error compared to Du et al. 2018