Dynamic facial expressions in American Sign Language: Behavioral, neuroimaging, and facial-coding analyses for deaf and hearing subjects

The aim of this study is to determine how Deaf signers (DS) and hearing non-signers (HNS) categorize emotional, American Sign Language (ASL) grammar and non-emotional non-grammatical (NENG) facial expressions, based on video clips showing only the face. The clips include six manifestations of 20 sentences (neutral, angry, surprise, quizzical, yes/no questions and wh-question (questions using who, what, etc.] and were chosen because angry, quizzical, and wh-question look similar, as do yes/no question and surprise expressions.; Results of the initial behavioral study (Chapter Two) show that HNS accurately categorize emotional and NENG expressions, but not ASL question faces. Subjects don't confuse ASL expressions with superficially similar emotional or NENG expressions.; Chapter Three presents results of DS and HNS for a redesigned stimulus set. Both groups accurately categorize emotional and neutral expressions bit frequently mislabel quizzical expressions “neutral.” Subjects categorize most ASL question faces correctly but make errors according to feature similarities, calling wh-questions “quizzical,” and “angry,” and categorizing yes/no questions as “surprise.” DS me more confident than HNS across all expression types. Differences in HNS performance between the two stimulus sets likely result from their ability to match the more subtle second set of ASL question expressions to familiar templates of question faces in English.; Chapter Four presents a new methodology for coding facial expressions. Data for movement of every facial feature per expression type averaged over 20 samples each show that emotional expressions have faster onsets of movement in features shared with grammatical or NENG expressions. Certain head movements only occur in specific expression types. These data show that dynamic differences distinguish expression types, despite static feature similarities.; Chapter Five presents an fMRI study showing activation in the right superior temporal gyrus and right interior frontal lobule, supporting previously untested hypotheses that these regions process facial expressions in ASL. Both groups show language activation in left superior temporal gyrus (STG). DS have more activation m STG than HNS, because of increased ASL language competence. Language activation in HNS may be related to task indicating that the stimuli represent sentence No differential activation patterns for different stimulus types were detected.