- author = "Andrew Joseph Alexander Gildfind",
- title = "Evolving Performance Control Systems for Digital Puppetry",
- school = "Computer Science, RMIT University",
- year = "2001",
- address = "Melbourne, Australia",
- keywords = "genetic algorithms, genetic programming, Performance animation, motion capture, performance control systems, puppetry, interactive evolution, adaptive user interfaces, usability, human-computer interaction, HCI",
-
URL = "
https://collections.pomona.edu/labs/record/?pdb=3234",
-
abstract = "Digital puppetry is an emerging form of computer
animation in which performers control the movements of
an animated model in real-time, using specialised input
devices such as instrumented gloves and motion
trackers. A performance control system is the software
which converts puppeteer inputs to the movements of a
digital puppet. This thesis introduces a new approach
for creating performance control systems.
Developing a performance control system involves three tasks: 1) exploring the capabilities of the puppet; 2) designing the interface through which a puppeteer will control the puppet; and, 3) implementing the low-level functionality of the performance control system itself. In addition to understanding the techniques required to implement the performance control system, these tasks can require a detailed knowledge of both the parameterisation of the model and the structure of the input devices. Current performance animation systems adopt this low-level view. Typically the underlying structure of the input devices and models is exposed, and the designer is required to specify how each input should be associated with each output. Such systems provide a range of facilities for transforming this input data. While this provides a general and powerful infrastructure for implementing control systems, such systems provide little support for the design process itself. This has two consequences. The first is that control system developers are forced to work at a very low-level of abstraction — many potential control system designs can be very difficult to conceptualise and implement in an explicit fashion such as this. The second is that the end-users of these control systems, the puppeteers, are typically dependent on others to understand their requirements and then to design and implement the control systems they perform with.
This thesis describes a new approach that unifies and automates much of the process of developing performance control systems. Where, traditionally, the puppeteer communicates their requirements to designers and programmers who implement the control system, this work explores the possibility of replacing these human developers with an interactive, search-based optimiser. Performance control systems are evolved using a variation of the genetic programming algorithm. The fitness metric, that drives the search process, is provided interactively by the puppeteer. From an initial, randomly generated, population of control systems, user ratings for each potential solution determine those control system traits that are successful and should be explored further, and those that can be discarded. Subsequent generations are created by applying the standard genetic operations: reproduction, crossover and mutation. Each intermediate solution is a complete, functioning control system. The search process completely automates and hides the details of control system implementation and provides a powerful mechanism through which design possibilities can be explored and refined. It provides a natural mechanism through which puppeteers can tackle each of the tasks required to create a performance control system directly, without help from designers and programmers. The continuous dialog with the user allows a search process to arrive at solutions that are potentially better customised to their preferences, their dexterity and the requirements of specific performances than control systems created by others.
A significant contribution of this research is the design and development of APECS — the Adaptive Performance Control System. APECS is a prototype performance animation system which has served as the testbed for the ideas of this work. Within APECS control systems are created using the iterative user-guided search process. Two requirements have strongly influenced the design of APECS. The first is the need for flexibility and generalisability. The architecture of APECS has been designed to exploit the inherent domain independence and generalisability of the genetic programming algorithm. It makes no assumptions about the input devices and models that might be used to create a performance control system. Models and devices are represented internally in a generic form. To the search algorithm they appear the same — with the only distinguishing factor the number of data inputs or outputs. The second requirement is the need for usability. Usability in this context means that puppeteers can understand the search process and learn to control it successfully; and that they can create effective performance control systems for real performance tasks in a reasonable length of time.
The search process hides the details of device and model parameterisation and takes care of the underlying implementation details. However, a large part of the design task remains with the user. Early exploratory evaluations of the APECS system suggested that the burden on the user could become very high. As a result, considerable work has been done exploring improvements to APECS both at the architectural and user interface levels. This has resulted in a range of techniques which make the process of experimenting with and evaluating control systems more efficient. These include an expansion to the underlying genetic programming algorithm, the development of interaction techniques for selecting and exploring the functionality of unfamiliar control systems, and the development of a novel structuring for the search process that uses recorded input sequences, or gestures, to manage the complexity of the evaluation task. Usability, however, has not been achieved at the expense of the flexibility of the approach. All of the interaction techniques and algorithmic extensions are applicable to any device or model that might be integrated into the system.
The effectiveness of the search-based approach, and the success of the APECS design have been assessed via a comprehensive usability study. Ten experienced puppeteers participated in this evaluation. Each was required to use the APECS system to evolve control systems for a realistic animation task. Comprehensive pre- and post-test questionnaires were used to gauge the success of the approach. The results of this evaluation showed, among other things, that: puppeteers were extremely enthusiastic about the approach and appreciated the ability to tackle the control system design problem themselves; puppeteers were able to create control systems using APECS that allowed them to successfully complete the animation tasks; and that, rated in terms of comfort, flexibility, and a range of other quality metrics, participants found the evolved control systems to be successful.
The detailed evaluation process provided the opportunity to experiment with a wide range of different potential solutions and discuss the results with puppeteers. The last part of this thesis explores some of these insights. What are the features of usable performance control systems? This question is explored and a new form of real-time interaction, that has a BibTeX entry too long. Truncated
