Medical Multimedia

An exciting and challenging research topic I am working on is content analysis of videos from medical endoscopy (so called endoscopic video analysis) and interaction with content from these endoscopic videos. The background is that nowadays many surgeons record and store videos of their endoscopic procedures and surgeries for later use, such as explanations to patients, quality inspections, and training and teaching of new surgical methods. While this valuable data enables completely new use scenarios it also bears new challenges, such as managing the storage of an endoscopic video archive Рthat increases on a daily basis Рas well as automatic content analysis of this special kind of data in order to make it accessible.

Over the last few years we at ITEC have put much effort into research on this challenging new topic. Among other problems we focused on:

  • compression optimization of endoscopic videos
  • content segmentation of endoscopic videos
  • relevance filtering of content from medical endoscopy
  • keyframe detection in endoscopic videos
  • content classification of surgical actions and anatomic structures
  • content similarity search for endoscopic video segments
  • visualization of endoscopic video archives
  • content annotation of endoscopic videos
  • deep learning surgical actions
  • automatic smoke detection
  • retrieval of video segments with specific¬†descriptors

Here are a few papers that give more details.

Domain Specific Video CompressionHierarchical Visualization of Endoscopic KeyframesPhase Segmentation Through Instrument Detectiongynlearning
endotabletannotationendotmultibrowserGradient-Based SignaturesSupporting Surgical Quality Assessment

Additional information can be found at