A Study on Salient Event Detection in Videos

title = "A Study on Salient Event Detection in Videos",

school = "Information Media and Environment Sciences, Graduate School of Environment and Information Sciences, Yokohama National University",

year = "2017",

address = "79-7 Tokiwadai, Hodogaya-ku, Yokohama 240-8501, Japan",

keywords = "genetic algorithms, genetic programming",

URL = "

https://dl.ndl.go.jp/info:ndljp/pid/10993906",

URL = "

https://ynu.repo.nii.ac.jp/?action=repository_uri&item_id=8752&file_id=20&file_no=1",

DOI = "

doi:10.18880/00011427",

size = "76 pages",

abstract = "Surveillance cameras are installed in many public places such as airports, stations, and apartments for security. In addition, surgery videos are also recorded in operating rooms for postoperative evaluation. However, it takes a lot of effort to use these videos because there is a large number of videos and human have to visually check these videos. For example, it is required to have constant human monitoring of videos arriving at the control center for security. For postoperative evaluation, extracting specific medical practices from surgery videos is needed to analyze the videos, but it takes a lot of labor to extract specific medical actions from many videos. In light of this situation, event detection methods from videos are highly beneficial. Detecting events, such as important events and events which occur less frequently, are very useful for analyzing the videos. We refer to these events as salient events in this paper and propose methods to detect these salient events.

There are several conceivable conditions regarding training data when we construct event detection models. The first condition is to train models using label data of salient events and normal events. In the case of this situation, we construct models based on supervised learning. The second condition is that only normal event data are available for training models. In the case of this situation, we construct normal models, and detect salient events using these models. The last condition is that we cannot use both label data of salient and normal events. In the case of this situation, it is needed to define events as salient or normal while observing videos, and we construct models based on these observations. In this paper, we propose event detection methods in each condition above, and demonstrate the effectiveness of our methods in comparative experiments with other methods.

In Section 3, we detect salient events in surgery videos using training data including salient and normal label data. Specifically, we detect electrical stimulation in videos of cotrical mapping in awake surgery.The proposed method consists of two phases: detection of a probe tip position and detection of electrical stimulation timings.

In Section 4, we detect salient events in surveillance videos using training data including only normal-label data. In this section, pedestrians are defined as normal and other objects, such as cars and bike, are defined as salient events. Our method detects these salient events using reconstruction error of convolutional autoencoder. To evaluate the proposed method, we test our method on the UCSD Pedestrian dataset.

Finally, in Section 5, we deal with intrusion detection tasks in surveillance videos in the case using training data including neither salient nor normal label data. In this section, we define pedestrians and cars as salient events for evaluation. We show that our method, self-organising network, can detect salient events in videos.",

notes = "Written in Japanese

Supervisor: Tomoharu Nagao",