Elsevier

Applied Soft Computing

Volume 56, July 2017, Pages 368-378
Applied Soft Computing

Automatic model construction for the behavior of human crowds

https://doi.org/10.1016/j.asoc.2017.03.020Get rights and content

Highlights

  • We propose an automatic methodology to learn generic crowd behavior rules from video data.

  • We formulate the problem of finding behavioral rule from video data as a symbolic regression problem..

  • The behavior rules learned by the proposed method are applicable to different scenarios that have similar behavior patterns.

  • We apply the proposed method to real world datasets and the results have demonstrated the effectiveness of the proposed method.

Abstract

Designing suitable behavioral rules of agents so as to generate realistic behaviors is a fundamental and challenging task in many forms of computational modeling. This paper proposes a novel methodology to automatically generate a descriptive model, in the form of behavioral rules, from video data of human crowds. In the proposed methodology, the problem of modeling crowd behaviors is formulated as a symbolic regression problem and the self-learning gene expression programming is utilized to solve the problem and automatically obtain behavioral rules that match data. To evaluate its effectiveness, we apply the proposed method to generate a model from a video dataset in Switzerland and then test the generality of the model by validating against video data from the United States. The results demonstrate that, based on the observed movement of people in one scenario, the proposed methodology can automatically construct a general model capable of describing the crowd dynamics of another scenario in a different context (e.g., Switzerland vs. U.S.) as long as that the crowd behavior patterns are similar.

Introduction

Agent-based modeling is a popular and powerful crowd modeling paradigm that has a range of applications in digital game design, military training, crowd evacuation planning, etc. [1], [2], [3], [4], [5], [6]. In the agent-based paradigm, individuals are modeled as autonomous agents that can perceive information and make decisions based on individual behavioral rules. A fundamental issue in all agent-based modeling is how to design suitable behavioral rules so that the simulation can match the desired crowd behaviors (e.g., those observed in videos). Once the correct behavioral rules are obtained, the crowd simulation model can be used for prediction and performing “what-if” analysis. However, designing correct behavioral rules to generate a desired crowd behavior remains a challenging task because an agent's behavior is affected by its environment as well as the surrounding agents. Typically, this is done in a fairly ad-hoc manner, and relies on expert intuition. However, intuition can often be misleading as crowds are complex systems and as such the relationships between the individual behavioral rules and the collective crowd dynamics are often unintuitive.

There are a number of existing models that have taken a scientifically Popperian approach to understanding crowd dynamics. A theory is developed which can then be falsified against real data. Helbing et al. [7], [8] suggested using virtual force to guide pedestrians’ motions. They designed mathematical equations to formulate the attractive and repulsive forces from goals, other pedestrians and obstacles. Yamaguchi et al. [9] designed behavioral rules using an energy function that considers multiple personal and social factors such as damping and grouping. Similarly, Pellegrini et al. [10], Scovanner and Tappen [11] also designed energy functions by considering different personal and social factors to determine movements of pedestrians. These manual methods usually require a model calibration process to tune coefficients of the initialized formulas, so that the model dynamics can match the desired crowd dynamics [9], [12], [10], [11], [13], [14]. However, if the initialized formulas are not well-defined, the calibration will not be able to generate the desired simulation results. In addition, the model calibration is also a time consuming and tedious process.

Recently, more Baconian, or data-driven approaches, have been proposed to automate the generation of crowd behaviors. The key idea is to learn examples (mostly in terms of state-action pairs) from video data. These examples are then used as behavioral rules in the simulation to generate agents’ movement [15], [16], [17], [18], [19], [20]. In the computer vision community, several approaches have been proposed that learn global motion patterns (e.g., certain movement curves) of crowds from video data [21], [22], [23], [24], [25]. The global motion patterns can be integrated with a crowd simulation model to generate realistic crowd behaviors. However, the knowledge learned by these methods (e.g., examples and global motion patterns) are typically highly scenario specific where training data is available. Once the application scenario changes, the learned knowledge may become incapable of generating the desired crowd dynamics.

In this paper and in previous work [26] we aim to develop a way of merging the Baconian and Popperian philosophies, by developing ways to automate model (or rule) generation. Effectively we allow the computer to find sets of plausible theories (models) that we can read and understand and perhaps manipulate later. In summary the core question of this paper is: “Can we design an automatic method to learn generic behavior rules from video data?” Here the term “generic” means that the behavioral rules are applicable not only to the scenario where training data are obtained from but also to other new scenarios. Also the notion of rule is critical, whereas a pure data-mining approach may be able to derive a model (e.g., trained neural network), the notion of a rule implies that humans can understand and perhaps modify the model.

Specifically, this paper focuses on simulating crowd behaviors in an area with one or multiple source regions (SRs) and destination regions (DRs). Each pedestrian in the given area keeps moving toward its own DR and leaves the area via the DR. Such a scenario is quite common in public places, e.g., train stations, airports, etc. We assume that the environment contains obstacles and that pedestrians move toward their DRs casually, following certain curved paths (e.g., “S” and “C” shape curves). The movement curves are determined by specific environment factors such as the positions of obstacles and the destinations. Our goal is to automatically learn a generic behavioral rule (labeled as Γ) from video data which describes the way pedestrians move toward their DRs while avoiding obstacles.

In the proposed methodology, an abstract dual-layer architecture is used to model individual behaviors. The bottom layer adopts the commonly used social force model (SFM) [8] to model the collision avoidance behaviors of pedestrians. Meanwhile, in the top layer, a behavioral rule that defines velocity fields is used to determine the strategic or goal-directed movement of pedestrians. A velocity field is a set of position-velocity pairs that describe the velocities of pedestrians in the simulation regions [27], [28], [20]. Based on the dual-layer modeling architecture, the problem of finding a suitable behavioral rule to generate the desired crowd behaviors is formulated as a symbolic regression problem. By using a genetic programming (GP) [29] based method to solve the symbolic regression problem, a behavioral rule that leads to the desired simulation results can be obtained automatically. To validate its effectiveness, the proposed methodology is applied to learn Γ from video data of a scenario in Switzerland. The learned Γ is then used to generate crowd dynamics in a new scenario in United Stated. The results have demonstrated that the proposed methodology is capable of distilling suitable behavioral rule that can generate realistic crowd dynamics in the new scenario.

This paper extends two of our previous papers on data-driven crowd modeling [26], [20]. In the first paper [26], we have conducted a preliminary research on learning goal selection rules of agents from simulation data. This paper extends [26] by addressing a complementary problem – to distill generic behavioral rules to define the velocity fields that can be used to drive agents toward their goals. In the second paper [20], we have proposed a data-driven method to learn behavioral patterns (e.g., the goal selection patterns and the velocity fields) from videos so as to generate realistic crowd dynamics. However, the behavioral patterns learned in [20] are only applicable to the specific scenario where training data is obtained from. In this paper, we aims to distill generic behavior rules that can be applied directly to different scenarios that have similar crowd behavior patterns.

In general, the major contributions of the paper are as follows. First, we have formulated the problem of distilling suitable crowd behavior rule from crowd data as a symbolic regression problem. Second, we have developed a genetic programming based framework to solve the formulated problem. The proposed framework is generic such that the solution obtained from one scenario can be applicable to other scenarios that have similar crowd behavior patterns. In addition, two real world datasets have been used to validate the effectiveness and generality of the proposed framework.

The rest of the paper is organized as follows: Section 2 describes the related work. Section 3 presents the outline of the proposed framework. Section 4 describes the procedure of applying Self-Learning Gene Expression Programming (SL-GEP) to distill behavioral rules from video data. Section 5 presents the simulation studies. Finally, Section 6 presents the conclusions and discussions.

Section snippets

Related work

Over the past decades, various methods have been proposed to help automate the process of modeling human crowds. These works can generally be classified into two groups. The first group focuses on automatic parameter calibration. The objective of this class of methods is to use optimization algorithms such as genetic algorithms (GA) to tune parameters of a model so that the simulations match the desired crowd behaviors (e.g., those observed in the video). For example, Johansson et al. [12]

Data-driven agent-based crowd modeling framework

This section proposes a generic data-driven agent-based framework to generate realistic crowd behaviors. First, the general structure of the proposed framework is described. Then, the mechanism of automatically learning behavior rule from video data is described.

Distil behavioral rules by using SL-GEP

The SL-GEP [29] is a new GP variant for automatic function generation. Compared with the traditional GP and GEP, the SL-GEP is easy to implement and contains fewer control parameters. The SL-GEP has been shown to be more efficient than the GP and the GEP on a range of problems, including the symbolic regression problems. Hence, in this paper, the SL-GEP is adopted to distill a suitable mapping function Γ to fit the generated input–output pairs. In this section, the chromosome representation of

Application of our method

In this paper, we focus on modeling the walking behavior of human in normal cases where people walk casually. More specifically, we aim to construct a human behavior model to generate crowd movements in an area with one or multiple SRs and DRs. The pedestrians move casually from SRs to DRs by following certain paths of motion, while at the same time avoiding collisions with nearby obstacles and other pedestrians.

Our objective is to automatically learn generic Γ from data (e.g., video feeds)

Conclusions and discussions

This paper has proposed an automatic methodology to extract generic behavioral rules from video for agent-based crowd modeling. Most importantly we have shown that this automatic approach to modeling can extract rules from a dataset in Switzerland and then be applied to generate realistic behavior for a crowd in the United States. We believe that this approach may be an effective method for systematic and automated creation of agent-based models in general.

The proposed methodology is capable of

Acknowledgments

The research reported in this paper is supported by National Natural Science Foundation of China (Grant Nos. 61602181 and 61502370), Tier 1 Academic Research Fund (AcRF) under project Number RG23/14, Natural Science Basic Research Plan in Shannxi Province of China (Program No. 2016JQ6003) and China 111 Project (Program No. B16037). M. Lees would like to acknowledge the Russian Science Foundation project #14-21-00137 “Supercomputer modeling of critical phenomena in complex social systems”.

References (35)

  • J. Zhong et al.

    Density-based evolutionary framework for crowd model calibration

    J. Comput. Sci.

    (2015)
  • J. Zhong et al.

    Differential evolution with sensitivity analysis and the Powell's method for crowd model calibration

    J. Comput. Sci.

    (2015)
  • W. Shao et al.

    Autonomous pedestrians

  • D. Thalmann et al.

    Crowd Simulation

    (2007)
  • N. Gaud et al.

    Holonic multiagent multilevel simulation: application to real-time pedestrian simulation in urban environment

  • L. Luo et al.

    Agent-based human behavior modeling for crowd simulation

    Comput. Anim. Virtual Worlds

    (2008)
  • J. Tsai et al.

    Escapes: evacuation simulation with children, authorities, parents, emotions, and social comparison

  • S. Zhou et al.

    Crowd modeling and simulation technologies

    ACM Trans. Model. Comput. Simul.

    (2010)
  • D. Helbing et al.

    Social force model for pedestrian dynamics

    Phys. Rev. E

    (1995)
  • D. Helbing et al.

    Simulating dynamical features of escape panic

    Nature

    (2000)
  • K. Yamaguchi et al.

    Who are you with and where are you going?

  • S. Pellegrini et al.

    You’ll never walk alone: modeling social behavior for multi-target tracking

  • P. Scovanner et al.

    Learning pedestrian dynamics from the real world

  • A. Johansson et al.

    Specification of the social force pedestrian model by evolutionary adjustment to video tracking data

    Adv. Complex Syst.

    (2007)
  • D. Wolinski et al.

    Parameter estimation and comparative evaluation of crowd simulations

  • K.H. Lee et al.

    Group behavior from video: a data-driven approach to crowd simulation

  • S.R. Musse et al.

    Using computer vision to simulate the motion of virtual agents

    Comput. Anim. Virtual Worlds

    (2007)
  • Cited by (0)

    View full text