Multi-modal emotive computing in a smart house environment

Abstract

We determine hazards within a smart house environment using an emotive computing framework. Representing a hazardous situation as an abnormal activity, we model normality using the concept of anxiety, using an agent based probabilistic approach. Interactions between a user and the environment are determined using multi-modal sensor data. The anxiety framework is a scalable, real-time approach that is able to incorporate data from a number of sources, or agents, and able to accommodate interleaving event sequences. In addition to using simple sensors, we introduce a method for using audio as a pervasive sensor indicating the presence of an activity. The audio data enabled the detection of activity when interactions between a user and a monitored device didn’t occur, successfully preventing false hazardous situations from being detected. We present results for a number of activity sequences, both normal and abnormal.