As we have studied the different sensory systems here in Paris, I personally have been the most intrigued by how those systems interact with each other. In everyday life, information from each sense frequently combines with and affects information from other sensory systems. Among other examples, studies have shown that the smell of food can affect its taste (Burdach et al., 1984), someone’s mouth movements can affect what we hear them saying (McGurk and MacDonald, 1976), and the form of a nonsense shape can predict what we will name that shape (Köhler, 1929).
The kiki-bouba task is one particularly famous version of this shape-symbolism example where participants are asked to match names with nonsense shapes. This version of the shape-symbolism task comes from a 2001 scientific study. In this paper, two scientists found that, when asked to match the names bouba and kiki to the shapes shown below, 95% of people called the jagged shape “kiki” and the rounded shape “bouba” (Ramachandran and Hubbard, 2001).
The “kiki” and “bouba” shapes
From Hamburg et al., 2017
In the years since this initial paper has published, many subsequent studies have been done to try to understand why so many people make the same sound-shape connections (e.g. Cuskley et al., 2015). Scientists believe that one reason many people label the nonsense shapes similarly is because the actual soundwaves mimic the rounded and jagged shapes of the letters in the names. (Ramachandran & Hubbard, 2001).
To match the sound of a word to what it looks like, our brains need to be able to integrate and compare auditory and visual information (Król and Ferenc, 2019). This ability is one example of what is known as multisensory integration. In general, multisensory integration is when information from more than one sensory systems is combined to create one unified representation (Stein and Stanford, 2008).
Multisensory integration begins when we are infants and continues to develop throughout childhood (Flom and Bahrick, 2007; Barutchu et al., 2009, 2010). Previous researchers have shown that multisensory integration is important for cognitive abilities like target detection, reaction time, and the development of other cognitive skills (e.g., Diederich and Colonius 2007; Lippert et al., 2007; Dionne-Dostie et al., 2015).
Previous research has also shown that multisensory integration is impaired in children with intellectual disabilities and individuals with autism spectrum disorder (Hayes et al., 2003; Oberman and Ramachandran, 2008). Interestingly, the kiki-bouba task is one of the ways researchers test for multisensory integration. Since the kiki-bouba task involves matching auditory information (the names) and visual information (the shapes), abnormal results can indicate multisensory integration problems.
In their recent study, Hamburg et al. used the kiki-bouba task to assess multisensory integration in adults with Down syndrome. Down syndrome occurs when someone has an extra copy of their twenty-first chromosome (for reviews, see: Antonarakis et al., 2004; Kazemi et al., 2016). It the most common genetic cause of intellectual disability in the world, but there it can affect a range of cognitive abilities (Asim et al., 2015). Many of the cognitive abilities that are impacted by Down syndrome involve brain structures that develop relatively late (Edgin, 2013). Since multisensory integration also develops throughout childhood, the authors predicted that this ability could be affected by Down syndrome.
Trisomy 21
To test this prediction, Hamburg et al. first evaluated participants with Down syndrome on several background questions about general cognitive ability and everyday adaptive abilities. Then these participants and typically-developing control participants completed the kiki-bouba task. The authors then calculated the overall correct response rate for both groups of participants. Based on the previous evidence, matching “kiki” to the pointy shape and “bouba” to the rounded shape was considered a correct answer.
The data showed that, among individuals with Down syndrome, the correct response rate on the kiki-bouba task was 72.5% compared to 90% in the typically developing age-matched controls. The authors therefore concluded that multisensory integration deficits are relatively common in individuals with Down syndrome. Additionally, for the participants with Down syndrome, the authors found that there was a significant relationship between individuals’ kiki-bouba task score and both their general cognitive ability score and their everyday adaptive abilities.
The authors found that individuals with Down syndrome who had lower scores for general cognitive ability and everyday adaptive abilities scored close to chance (correct response rates around 57%) while those with higher ability scores scored levels comparable to the typically developing controls. The authors concluded that sound-shape matching ability might be relatively common in the Down syndrome community but are mostly seen in individuals with lower cognitive abilities.
Personally, I enjoyed completing the kiki-bouba task in class as a fun example of multisensory integration. The idea of using this interesting task as an experimental test is exciting but, of course, there are limitations to this approach. Some studies suggest that, across different cultures, there may be differences in sound-shape mapping and other forms of multisensory integration (Bremner et al., 2013; Chen et al. 2016). These differences make the use of the kiki-bouba task as an experimental test concerning as cultural differences could confound results.
Furthermore, in the Hamburg et al. paper the authors noted that the decrease in correct response rate was primarily seen in individuals with Down syndrome who are categorized as severely intellectually impaired. As the authors acknowledge, it is hard to know how much of this effect is due to Down syndrome as opposed to severe intellectual impairments. These possible causes are especially hard to parse because there is little to no research about multisensory integration in individuals who have intellectual disabilities not due to Down syndrome. In the future, further research would have to be done with more precise control groups so that these factors could be dissociated.
While the study is far from conclusive, it is interesting to think about testing for multisensory integration in patients with cognitive conditions. In the future, understanding patients’ ability to combine information from their different senses could help medical professions better understand and support these individuals.
References
Antonarakis SE, Lyle R, Dermitzakis ET, Reymond A, DeutschS (2004). Chromosome 21 and down syndrome: from genomics to pathophysiology. Nat Rev Genet. 5:725–38.
Asim A, Kumar A, Muthuswamy S, Jain S, Agarwal S (2015) “Down syndrome: an insight of the disease”. J Biomed Sci. 22:41.
Bremner AJ, Caparos S, Davidoff J, de Fockert J, Linnell KJ, Spence C (2013) ‘Bouba’ and ‘Kiki’ in Namibia? A remote culture make similar shape– sound matches, but different shape –taste matches to Westerners. Cognition 126: 165– 172.
Burdach KJ, Kroeze JHA. and Koster EP (1984) Nasal, retronasal and gustatory perception: an experimental comparison. Percept. Psychophys., 36: 205—208.
Chen YC, Huang PC, Woods A, Spence C (2016). When “Bouba” equals “Kiki”: Cultural commonalities and cultural differences in sound-shape correspondences. Scientific Reports, 6:26681.
Cuskley C, Simner J, Kirby S (2015). Phonological and orthographic influences in the bouba-kiki effect. Psychological Research
Desai SS (1997). Down syndrome: A review of the literature, Oral Surgery, Oral Medicine, Oral Pathology, Oral Radiology, and Endodontology, 84(3): 279-285
Diederich A and Colonius H (2007). Why two ‘Distractors’ are better than one: modeling the effect of non-target auditory and tactile stimuli on visual saccadic reaction time, Exp. Brain Res. 179: 43–54.
Dionne-Dostie E, Paquette N, Lassonde M and Gallagher A. (2015). Multisensory integration and child neurodevelopment, Brain Sci. 5: 32–57.
Edgin J (2013). Cognition in Down syndrome: a developmental cognitive neuroscience perspective, WIREs Cogn. Sci. 4, 307–317.
Flom, R. and Bahrick, L. E. (2007). The development of infant discrimination of affect in multimodal and unimodal stimulation: the role of intersensory redundancy, Dev. Psychol. 43: 238–252.
Hamburg S, Startin CM, Strydom, A (2017). The Relationship Between Sound–Shape Matching and Cognitive Ability in Adults With Down Syndrome. Multisensory Research 30: 537–547
Hayes EA, Tiippana K, Nicol TG, Sams M, Kraus H (2003). Integration of heard and seen speech: a factor in learning disabilities in children, Neurosci. Lett. 351, 46–50.
Kazemi M, Salehi M, Kheirollahi M (2016). Down Syndrome: Current Status, Challenges and Future Perspectives. International journal of molecular and cellular medicine, 5(3), 125–133.
Köhler W (1929). Gestalt psychology. New York: Liveright
Król ME, Ferenc K (2019) Silent shapes and shapeless sounds: the robustness of the diminished crossmodal correspondences effect in autism spectrum conditions. Psychological Research 1-10.
Lippert M, Logothetis NK, Kayser C (2007). Improvement of visual contrast detection by a simultaneous sound, Brain Res. 1173: 102–109.
McGurk H, MacDonald JW (1976). Hearing lips and seeing voices. Nature. 264:746–748.
Oberman LM, Ramachandran VS (2008). Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: the mirror neuron hypothesis, Soc. Neurosci. 3: 348–355.
Peiffer-Smadja N, Cohen L (2019). The cerebral bases of the bouba-kiki effect, NeuroImage, 186: 679-689,
Ramachandran VS, Hubbard EM (2001). Synaesthesia–a window into perception, thought and language. J. Conscious. Stud., 8: 3-34
Stein BE, Stanford TR (2008) Multisensory integration: current issues from the perspective of the single neuron. Nat Rev Neurosci. 9:255–266.
Images
Bouba and Kiki Shapes: Figure 1 from Hamburg et al., 2017
Trisomy 21: https://upload.wikimedia.org/wikipedia/commons/a/ab/21_trisomy_-_Down_syndrome.png
2 responses to “when senses combine”