8th VBS with Great Success

The 8th Video Browser Showdown (VBS) took place a week ago (on January 8th and 9th) at MMM2019 in Thessaloniki, and it was a great success. For the first time we used the V3C1 dataset (Part 1 of the Vimeo Creative Commons Collection), which consists of 7475 video files that amount for about 1000 hours of content. The six participating teams could solve all visual and textual Known-Item Search (KIS) tasks, as well as all Ad-Hoc Video Search (AVS) tasks within a short amount of time! The teams have clearly demonstrated that their sophisticated video retrieval systems are very powerful and allow fast and effective content-based search in videos. We look forward to the next VBS in January 2020 in Daejeon, Korea at MMM2020! More information here: www.VideoBrowserShowdown.org

FWF Research Project SQUASH Approved

I am very happy that a new research project on Surgical Quality Assessment in Gynecologic Laparoscopy (SQUASH) – in collaboration with the Medical University of Vienna – has been approved by the FWF Austrian Science Fund. In this research project we will investigate if we can improve the efficiency and feasibility of technical skill assessment by automatic video content analysis, i.e, by content-based video retrieval and current methods of machine learning/deep learning.

We consider this research project as a pioneering work in the interdisciplinary overlap of computer- and medical science, which will investigate fundamental research questions that should provide the basis for future computer-aided surgical quality assessment (SQA). We expect that our research results will help to significantly facilitate the currently cumbersome and error-prone SQA process, and hence enable more surgeons to actually perform error ratings.

More information is available here.

Detecting Semantics in Endoscopic Videos with Deep Neural Networks

Here are the slides of my talk given at the 4th European Congress on Endometriosis (EEC2018) in Vienna, Austria, on November 23, 2018.

Interactive Video Search: Where is the User in the Age of Deep Learning? @ ACMMM18

Here are the slides of our tutorial presented on Monday, October 23, 2018 at ACM Multimedia 2018 in Seoul.

The Importance of Medical Multimedia @ACMMM18

Here are the slides of our tutorial presented on Monday, October 23, 2018 at ACM Multimedia 2018 in Seoul.

Residual Motion Improves Deep Learning Performance for Surgical Actions in Gynecologic Laparoscopy

In a recent work, presented at the 31st IEEE International Symposium on Computer-Based Medical Systems (CBMS2018), we could show that the inclusion of Residual Motion improves classification performance of surgical actions in videos from gynecologic laparoscopy significantly (resulting in a boost of Recall and Precision of 5% and 9% with the GoogLeNet CNN architecture). This performance can be improved even further (to a boost of 13% and 25% in terms of Recall and Precision) by using a late fusion approach for frame classification in the videos. The corresponding paper can be found here.

diveXplore Interactive Video Retrieval System

We have created a video of the diveXplore system, which we used for the Video Browser Showdown in 2017 and 2018 (as well as for the Lifelog Search Challenge 2018 in slightly modified form) quite successfully (2nd place in all three competitions). The video shows the different features of the system when applied to the IACC.3 dataset that consists of 600 hours of video content (around 300000 shots):

Cataract-101 Video Dataset

The ITEC Cataract-101 Dataset is available under here:

It consists of videos from 101 cataract surgeries, annotated with different operation phases that were performed by four different surgeons over a period of 9 months. These surgeons are grouped into moderately experienced and highly experienced surgeons (assistant vs. senior physicians), providing the basis for experience-based video analytics, as described in detail in the corresponding paper presented at MMSYS 2018.

LapGyn4 Gynecologic Laparoscopy Dataset

The ITEC LapGyn4 Gynecologic Laparoscopy Image Dataset is available under here:

It comprises four individual datasets (surgical actions, anatomical structures, actions on anatomy, and instrument count) taken from 500+ gynecologic laparoscopic surgeries for the task of automatic content analysis, as described in detail in the corresponding paper presented at MMSYS 2018.

ITEC @ Lange Nacht der Forschung 2018

On April 13, 2018 the ‘Lange Nacht der Forschung 2018‘ (long research night) took place again in Austria and the medical multimedia research group of ITEC did also participate in this exciting event that communicates research works to the public. Our booth at Klagenfurt University showed ‘How neural networks can help surgeons to improve surgical quality‘ and it was a great success: several thousands of people came to see our research efforts and they were stunned by our interesting demos and hands-on experiences: real-time instruments detection, real-time smoke detection (with dry ice), and several laparoscopic operation simulator boxes where visitors could test their surgical skills (we also had an original KARL STORZ Endoscope with corresponding laparoscopic instruments).

First six photos by photo riccio.