Modellisation of synchrony
Synchrony is a complex phenomenon that requires the perception and understanding of social and communicative signals (speech, linguistic cues, prosody, gesture, emotions…) and also a continuous adaptation. Implementation of interactive algorithm in human machine interfaces within complex tasks requires a better understanding of human strategies to regulate interaction, in particular synchrony. This new domain of research is named Social Signal Processing. In human behavior, early development and infant/care giver interactions are paradigmatic interactive situation in which synchrony is a key process. Behavior matching means that the infant and the caregiver have simultaneous behaviors. Synchrony means also that the infant and the caregiver move fluidly from one state to the next. In sum, synchronic maternal behaviors are related to efficient mother infant interactions whereas dyssynchronic ones qualify improper mother infant interactions. It makes synchrony a mirror of social contacts; these contacts can be in some cases disturbed as a consequence of a child’s difficulty (e.g. ASD) or a caregiver’s dysfunction (e.g. parents showing negligent behavior). One of the key issue in evaluating synchrony is the fact that (i) it requires expertise in early infant interaction, (ii) the phenomenon includes two (and sometimes more) partners, (iii) modeling is complex and may require computational methods, (iv) no automatic tools using engineering methods are available to extract synchronic behavior in video recording of early interactions.
In the field of psychology, several manual methods have been proposed to evaluate interaction synchrony ranging from behavior micro-analysis to global perception of synchrony. These methods are time consuming and based on rater’s subjectivity. To overcome the caveats of simple coding, several teams have also used new engineering techniques of interaction analysis to assess interaction synchrony (Delaherche et al., 2012). Two different approaches are usually followed for the characterization of interaction synchrony: (1) High-level features: Exploitation of micro-annotations then extraction of latent features (Hidden Markov Models, Factorization models). This method requires the annotation of high-level information (gesture, eye-contact…) for windows frames of few seconds. Recent research (Saint-Georges et al., 2012) applied machine-learning methods to explore TD infant and mother behavior during interaction. They showed that developmental changes were most evident when the probability of specific behaviors was examined in specific interactive contexts (Saint-Georges et al., 2012). (2) Low-level features: Automatic extraction of low-level features such as global movements, prosody, speech turn taking… These signals are then automatically combined to estimate statistical high-level models. The current work package will focus on such algorithms applied to early caregiver-infant interaction and to be used on the clinical conditions or risks details in work packages 2, 3 and 4.