Interactive Multimedia

Interactive video search (also known as video browsing or exploratory video search) is a method of interactive information retrieval in video content that is not limited to the typical query-and-browse results approach employed by traditional retrieval tools. Instead, exploratory video search tools are highly interactive and try to integrate the user during several stages of the search process (through iterative human-computer interaction). This way they provide both visual browsing facilities (e.g., smart navigation features, content overviews and structuring means) as well as query features that allow users to translate their imaginations to automatic content retrieval (e.g., query by filtering, query by sketch, similarity search, etc.). Exploratory video search therefore supports content search situations where users cannot formulate a concrete query or simply want to inspect the video in order to see its content.

This kind of modern video browsing supports two content search scenarios: directed and undirected search. In the first scenario, users have a clear information need and want to find a specific target segment in the video (e.g., the weather forecast in a news show); such a search is also known as known-item search or target search. In the second scenario, users have no concrete search goal but want to explore the content in order to learn or find something interesting (e.g., a violent scene in surveillance videos); such a search scenario is known as exploratory search.

Over the years many tools for interactive video search and exploration have been proposed in the literature (see [1],[2] for a survey), and it has been shown that these tools can effectively help users find desired content in videos (e.g., [3],[4]). Some of these tools combine sophisticated content analysis methods controlled by the user for their personal needs. Some others provide rather simple content navigation features but give the users more interactivity to allow them to effectively take advantage of their knowledge about the content and the content structure. Interestingly, it has been shown that tools of the latter kind can even outperform tools of the first kind for some search tasks [5].

Slides of a tutorial on Interactive Video Search, presented at the ACM International Conference on Multimedia 2015 (ACM MM’15) in Brisbane, Australia, can be found here:


The Video Browser Showdown (aka Video Search Showcase) is an annual interactive video retrieval competition, held at the International MultiMedia Modeling (MMM) conference, where participating teams try to solve ad-hoc known-item search tasks in a shared data set of a few hundreds hours of video content as quickly as possible. The purpose of this search competition, which I am co-organizing, is to advance interactive video search tools to the next level. All the details about the VBS competition can be found this website: