Frontiers in Bioscience-Landmark (FBL) is published by IMR Press from Volume 26 Issue 5 (2021). Previous articles were published by another publisher on a subscription basis, and they are hosted by IMR Press on imrpress.com as a courtesy and upon agreement with Frontiers in Bioscience.
Synesthesia literally means a “union of the senses” whereby two or more of the five senses that are normally experienced separately are involuntarily and automatically joined together in experience (1, 2, 3). For example, some synesthetes experience a color when they hear a sound, although many instances of synesthesia also occur entirely within the visual sense. In this paper, I first mainly engage critically with Sollberger’s view that there is reason to think that at least some synesthetic experiences can be viewed as truly veridical perceptions, and not as illusions or hallucinations (4). Among other things, I explore the possibility that many forms of synesthesia can be understood as experiencing what I will call “second-order secondary properties,” that is, experiences of properties of objects induced by the secondary qualities of those objects. In doing so, I shed some light on why synesthesia is typically one-directional and its relation to some psychopathologies such as autism.
Synesthesia is the “union of the senses” whereby two or more of the five senses that are normally experienced separately are involuntarily and automatically joined together in experience (1, 2, 3). For example, some synesthetes experience a color when they hear a sound, although many instances of synesthesia also occur entirely within the visual sense. After making some preliminary distinctions, I first engage critically with Sollberger’s view that there is reason to think that at least some synesthetic experiences can be viewed as veridical perceptions, and not as illusions or hallucinations (4). I also explore the possibility that many forms of synesthesia can be understood as experiencing what I will call “second-order secondary properties,” that is, experiences of properties or qualities of objects induced by the secondary qualities of those objects. Finally, I shed light on why synesthesia is virtually always one-directional and its relation to some psychopathologies such as autism.
Synesthesia or, what we might call synesthetic experiences, often involves instances where two or more of the five senses that are normally experienced separately are involuntarily and automatically joined together in experience (1, 2). For example, some synesthetes experience a color when they hear a sound or see a letter. However, synesthesia can occur entirely within one sense, for example, “grapheme-color synesthesia,” the most common form of synesthesia, involves experiencing (black) letters or numerals as inherently colored. For example, one might always experience the letter “R” or the numeral “2” as red, or the letter “N” and the numeral “8” as purple. All letters and numerals are experienced as having clearly distinct and regular colors. Others experience tastes, smells, shapes, or touches in almost any combination. These sensations are automatic and cannot be turned on or off.
Motion-sound synesthesia involves hearing sounds in response to visual motion and flickers. Saenz and Koch report evidence that, for at least four synesthetes, seeing visual motion or non-moving visual flashes automatically causes the experience of sound (5). These synesthetes outperformed control subjects on a difficult visual task involving rhythmic temporal patterns, for example, judging whether two successive sequences (either both auditory or both visual) were the same or different. This is presumably because these synesthetes not only see but also hear the patterns. Unlike many other abnormal psychological phenomena, however, synesthesia is not a disease or illness and is typically not harmful. In fact, the vast majority of synesthetes prefer to have synesthesia and could not imagine life without it (though there are some exceptions as we will see later). Synesthesia can, for example, aid one’s memory of names and phone numbers and be an asset for creative art. Still, what “it is like” to be a synesthete must be quite different than most of our “normal” conscious experience. In a sense, we might say that they experience an enhanced form of conscious experience as opposed to the typical disorder, that is, something is added to conscious experience instead of the more typical subtraction.
Several key terms and distinctions are important to note at the outset:
1. Grossenbacher and Lovelace use the terms ‘‘inducer’’ to refer to the stimulus that triggers the synesthesia and ‘‘concurrent’’ to refer to the synesthetically induced sensory attributes (6). Synesthetic experiences are highly idiosyncratic and individualized, that is, no two people’s set of synesthetic experiences seem to be exactly the same.
2. There are so-called “higher” versus “lower” synesthetes in grapheme-color synesthesia (1, 7). Higher synesthesia is much more common and has to with the “meaning” or “concept” of grapheme, that is, the concept inherent in a grapheme that induces color, not the visual shape itself. Letter capitalization and font size generally do not change an induced color. For example, J, j, and J evoke the same color experience. Lower synesthesia is rare whereby the inducer is the visual shape itself.
3. Another central distinction is between “projectors” and “associators” in grapheme color synesthesia (8). The concurrent images are either projected onto the external world (projector synesthesia) or perceived in the mind’s eye (associator synesthesia). In projector synesthesia, the projected concurrent may be seen as instantiated like non-synesthetic colors, as floating above its inducer, or even as an ‘afterimage’ that floats close to the subject’s eyes. In associator synesthesia, the concurrent image is seen internally, much like a visual image retrieved from memory or generated by imagination. It is worth mentioning, however, that Cytowic and Eagleman find this distinction inadequate partly because some concurrent color locations need not be right on the grapheme itself (7). Thus they prefer to distinguish between “localizer” and “nonlocalizer” where the former involves experiencing synesthetic colors belong to a specific location (whether or not it is on the inducer or grapheme) and the latter refers to those synesthetic color experiences with no specific location.
There is significant empirical evidence for the view that synesthetic experiences are perceptual in the sense that they are genuinely experienced as properties of objects. For example, it has been shown that grapheme-color synesthetes can perceptually group graphemes according to their synesthetic colors (9). Neuroimaging studies from Nunn and colleagues have shown similar brain activation in synesthetes as found in typical non-synesthetic color processing (10).
One explanation for this kind of synesthesia is that there is “cross-activation” or “cross-wiring” of adjacent brain regions (1). The normal lack of overlap and integration between the brain regions is absent:
“The fusiform gyrus (in the temporal lobes) contains the color area V4 … which processes color information, but … the number area of the brain, which represents visual numbers …, is right next to it … (and) imaging experiments on people with synesthesia suggest that showing black and white numbers to a synesthete produces activation in the color area ….” (11, p. 65).
There is thus a kind of neural “hyperconnectivity” in these synesthetes not found in other people. Other related neural explanations appeal to “disinhibited cortical feedback” between brain areas such that information is processed in a bottom-up fashion but also that later stage brain activation feeds back to activate earlier stages. It is this abnormal feedback that causes the unusual synesthetic experiences (6). Cytowic and Eagleman explain the neural differences between higher and lower synesthetes by pointing out how different brain areas cross-activate with V4 which is the primary color area in the visual cortex (7). For higher synesthetes, V4 cross-activates with the anterior inferior temporal (AIT) cortex which processes conceptual representations of words, letters, and numbers. For lower synesthetes, V4 cross-activates with the visual word form area (VWFA) in the fusiform gyrus which responds to visually presented words, letters, and numbers.
The relationship between synesthesia and hallucination is an interesting one. Is synesthesia a special kind of hallucination or are synesthetic experiences perceptually veridical in some way? I will mainly focus on Sollberger’s discussion since it is an in-depth treatment of the issue (4). He aims to show that “there is reason to think that at least some synesthetic experiences can be viewed as truly veridical perceptions, and not as illusions or hallucinations” (4, p. 171). He mainly focuses on “…a sub-group of synesthetes who meet the following two conditions: (a) They literally attribute the sensory properties of the synesthetic experiences to the distal stimulus itself (and) (b) They do not take their synesthetic experiences to be nonveridical, e.g. illusory or hallucinatory. This means that the following question not only makes sense but is most often answered in the affirmative by such synesthetes: For any synesthetically evoked sensory property F that the distal physical object x appears to have, does x really have F?” (4, p. 173). This would certainly be the case for projector grapheme-color synesthesia. He also cites an interesting case described by Cytowic (12, p. 13): “I remember most accurately scents. We were preparing to move into the house I grew up in. I remember at age 2 my father was on a ladder painting the left side of the wall. The paint smelled blue (emphasis added), although he was painting it white. I remember to this day thinking why the paint was white, when it smelled blue.”
Sollberger offers and defends three reasons for treating synesthetic experiences as truly veridical perceptions:
1. “synesthesia enhances several cognitive and perceptual capacities in its bearer. The additional synesthetic sense can enhance the abilities of reading, writing and spelling and it can also expand the memory faculties (p. 174)…the fact that synesthesia is not a disabling or dysfunctional biological trait, but a condition that can indeed benefit the possessor’s cognition and perception, opens up space for considering synesthetic experiences as potentially veridical perceptions” (4, p. 175).
2. “…the subjective reports of synesthetes [show that they] are firmly convinced that what they synesthetically perceive is real and ‘‘valid,’’ and not hallucinatory or illusory” (4, p. 175).
3. “…from a purely evolutionary perspective, the goal of perception is to maximize fitness, i.e., to raise more offspring. Perception must be viewed as a niche- and problem-specific cognitive function whose purpose is to enhance fitness. Pertinently, the perceiver is able to survive and reproduce only if she can successfully interact with the world” (4, p. 175).
Sollberger carefully considers several objections to each of the above reasons and then offers further counter-replies. I won’t delve deeply into each objections or counter-reply but, for example, he considers and rejects the notion that we should not take reports of most synesthetes at face value. Although I largely agree with Sollberger regarding the plausibility of the above three claims, I wish to critically elaborate on some of them and other points raised in his paper. He does concede that some synesthetes probably do not really take some concurrents to be properties of the distal objects (e.g. that numbers really have personality traits or genders) but he reiterates that he is not claiming that all synesthetes are the same in this respect. Still, he insists that we should not think of synesthetic experiences as involving some special kind of hallucination. He warns against being overly dismissive of what is experientially possible with respect to the following options (4, p. 178):
(A) Strong actual reading: the synesthetic concurrents appear to the synesthete as properties of the distal object x.
(B) Strong possible reading: it is possible that the synesthetic concurrents appear to the synesthete as properties of x.
(C) Weak actual reading: the synesthetic concurrents appear to the synesthete as being bound in some way to x.
Sollberger sees little reason to rule out cases of (A) and allow only for (C). Some might suppose that we cannot make sense of (A): “That is, a skeptic might be tempted to rule out such cases a priori because she thinks that this kind of cross-modal property attribution is inconceivable and hence impossible. Cases of (B) must eo ipso also be rejected by such a skeptic. What is odd about such a dismissive view about what is experientially possible is that it is far too narrow-minded” (4, p. 178). Still, as Sollberger knows, there are cases of “associator” grapheme-color synesthesia which would seem to fit (C) better than (A) which, in turn, better describes “projector” grapheme-color synesthesia. In some ways, Sollberger might simply be making a plausible overall case for the apparent truism that there is more than one coherent way to experience the same world of objects and properties. However, as we will see below, the matter gets more complicated very quickly.
Before going further, it will be useful to have a working definition of a hallucination. On one view, it is “a percept-like experience which (a) occurs in the absence of appropriate stimulus, (b) has the full force or impact of the corresponding (real) perception, and (c) is not amenable to direct and voluntary control by the experiencer” (13, p. 23). But this definition could characterize synesthesia in different ways. Indeed, it is pretty clear that (b) and (c) are present in synesthetic experiences, as we have already seen. The problem, however, might be with (a) and its specific use of the term “appropriate stimulus.” Presumably, this refers to something like the “normal” or “usual” stimulus for typical perceivers. But if this is so, then synesthetes are having hallucinatory experiences since they are not typical perceivers in this respect. If meeting the above three conditions is sufficient for having a hallucination, then it would thus seem that Sollberger’s view could be challenged on those grounds. Still, there seems to be something far more intrasubjectively stable, systematic, and reliable in the synesthete’s experience which is lacking in other random and momentary hallucinations. So we might suppose that synesthesia involves having some sort of regular perceptual “error” as long as it is a stable and systematic natural error. The stimuli in question are, we might say, “appropriate” or “normal” for the synesthete. If they are hallucinations, they are at least different than those caused rarely and somewhat randomly by ingesting drugs or suffering from epilepsy.
There also seems to be an ambiguity in the use of the term “appears” in the above readings (A) – (C). In some cases, such as in projector grapheme-color cases, the term ‘appears’ refers to the way the distal stimulus looks to the synesthete. But, especially in other non-visual cases, such as color-smells or sounds-taste synesthesia, the term ‘appears’ seems to mean something more like “caused by” which is not quite the same. That is, if the paint smells blue, is the claim that the paint causes me to experience a certain smell or am I saying that the smell appears in the paint or as part of the paint somehow? This sort of synesthesia seems at best to meet (C) above, that is, the week actual reading. Indeed, there is perhaps even a further ambiguity in the use of ‘appears.’ Consider another one of Sollberger’s examples:
“The shapes are not distinct from hearing them -- they are part of what hearing is. The vibraphone, the musical instrument, makes a round shape. Each is like a little gold ball falling. That’s what the sound is; it couldn’t possibly by anything else” (12, p. 69).
This seems to be an example of sound-vision synesthesia. Is the concurrent (the shape) experienced as part of the synesthetic auditory experience? It would seem so if we take the report at face value, especially in the first and fourth sentence in the quotation above. However, the second sentence might be taken instead as reporting that the sound, or even the instrument itself, causes (“makes”) a round shape. This seems more like the language of cause and effect. So does the sound itself “appear” round or does it simply cause the synesthete to experience round shapes? Perhaps even more difficult to understand, is the sound itself round in some sense?
So the case for treating synesthetic experiences as hallucinations is perhaps somewhat stronger than Sollberger claims according to the above definition, especially if we interpret the “appropriate stimulus” as the normal stimulus for a typical perceiver. It is worth noting that the official “American Psychological Association” dictionary definition of a hallucination is not necessarily very helpful here. According to it, a hallucination is a “false sensory perception that has a compelling sense of reality despite the absence of an external stimulus. It may affect any of the senses, but auditory hallucinations and visual hallucinations are most common. Hallucinations are typically a symptom of a psychotic disorder, particularly schizophrenia, but also may result from substance use, neurological abnormalities, and other conditions. It is important to distinguish hallucinations from illusions, which are misinterpretations of real sensory stimuli” (https://dictionary.apa.org/hallucination).
It is pretty clear, however, that virtually all cases of synesthesia emphatically do not involve “the absence of an external stimulus” if this means the total lack of any distal object at all. There are no experiences of pink rats climbing on the wall when there is nothing at all on the wall. So this tends to favor Sollberger’s view that synesthesia is not hallucinatory. However, there is still presumably the absence of the property attributed to the object (again, the “appropriate stimulus”), at least according to normal perceivers. In this respect, perhaps synesthesia is closer to an illusion than a hallucination. O’Callaghan (14) seems to have something like this ambiguity in mind when he says that:
“…synesthesia is not necessarily hallucinatory. “In many cases, synesthetes perceive an object but misperceive its features. For instance, a synesthete might see a grapheme but misattribute some color to it. Perhaps, however, this should be understood as involving an attribute hallucination or property hallucination rather than mere illusion. This would require developing and appealing to an independently motivated conception of attribute or property hallucination. Nevertheless, being hallucinatory does not appear to suffice for being a case of synesthesia” (14, p. 53, fn. 13).
In the next subsection, I will explore a view along these lines.
The above discussion leads me to consider how the traditional distinction between primary and secondary qualities of objects can shed light on the nature of synesthetic experience.
Some background first: John Locke famously distinguished between primary and secondary qualities of objects (15). Primary qualities are those qualities that have to do with the object’s microstructure and, according to Locke, are inseparable from the external object itself, such as size, shape, mass, number, and motion. Secondary qualities, however, are those qualities which are “nothing in the objects in themselves but powers to produce various sensations or ideas in us,” such as colors, tastes, and sounds. It is only ideas of primary qualities of a perceptual object that really resemble what is in the object whereas secondary qualities are merely caused by the object’s microstructure. According to Locke’s representative realism, primary qualities are “really out there” mind-independently, especially as compared to secondary qualities which are mind-dependent to some extent. Primary qualities would still exist without minds to perceive them but there would not be any secondary qualities. Locke explains:
“…I think it easy to draw this observation, that the ideas of primary qualities of bodies are resemblances of them, and their patterns do really exist in the bodies themselves, but the ideas produced in us by these secondary qualities have no resemblance to them at all. There is nothing like our ideas existing in the bodies themselves. They are, in the bodies we denominate from them, only a power to produce those sensations in us; and what is sweet, blue, or warm in idea is but the certain bulk, figure, and motion of the insensible parts, in the bodies themselves, which we call so” (15, Book II, Ch. 8, sec. 15).
There is of course still significant debate today as to how best to characterize the mind-dependence or mind-independence of secondary qualities (especially with respect to color). The question might be framed as “What kinds of properties are colors?” or “Are colors mind-independent in some sense?” For example, “primitivism” about colors holds that colors are primitive properties, that is, simple, sui generis, qualitative properties that physical bodies possess or appear to possess. A “reductive physicalist” holds that colors are “hidden” properties of bodies, that is, complex, physical properties that dispose bodies to look blue, pink, yellow, and so on. Another view is “dispositionalism” such that colors are perceiver-dependent, dispositional properties; that is, powers to look in distinctive ways to appropriate perceivers, in appropriate circumstances (16, p. 9). Note, however, that we still have the problematic and ambiguous expressions “appropriate perceivers” and “appropriate circumstances.”
Sollberger also recognizes the issue at hand. He explains that:
“Accepting that synesthetic experiences can be veridical will, of course, have important ramifications for what a metaphysical theory of color properties can look like. For instance, it seems to be immediately ruled out that colors could be construed as intrinsic, categorical properties of physical objects. Instead, it marries up more easily with a form of psychological-dispositionalism about color, according to which x’s property of having a certain color, such as red, is analyzed in terms of dispositions and powers…. In this way, grapheme-color synesthesia can be veridical because there is nothing incoherent in the idea that an object can have the disposition to appear black and the disposition to appear red to the synesthete simultaneously…Moreover, dispositionalists can insist that perceiving distal objects as colored does not involve a kind of massive error or systemic illusion, for the dispositional properties can be grounded in the categorical bases of the objects themselves” (4, p. 183).
Of course, an idealist, such as Bishop Berkeley, would say that even the so-called primary qualities are mind-dependent (at least dependent on God’s mind). I do not wish to try to settle these disputes here. For our purposes, let us simply assume that there are mind-independent objects and that we often do experience secondary qualities as properties of external objects, for example, that objects appear to us as colored. Still, some secondary qualities seem better described as caused by external objects, such as the sound of a guitar string vibration or the smell of a specific food.
So it makes sense that secondary qualities are almost always those experienced as concurrents. In this way, synesthetes can have an intrasubjective coherent stream of conscious perceptions. Since secondary qualities are at least not entirely mind-independent in some sense, they perhaps matter less to coherent conscious experience in the sense that one can have an individual and idiosyncratic way of experiencing the color, taste, and smell of objects. If we treat secondary qualities as themselves appearances of objects, then concurrents are more like unusual appearances of objects or, perhaps even better, “appearances of appearances” in at least cases where a secondary quality inducer triggers a concurrent experience of another secondary quality (such as in sound-color, smell-touch, or sound-taste synesthesia). That is, we can understand many instances of synesthesia to involve what we might call “second-order secondary properties,” namely, the experiences of (secondary) properties or qualities of objects induced by the secondary qualities of those objects (as opposed to primary qualities). So it is crucial to notice that the concurrent is virtually always a secondary quality of objects, such as a taste, smell, or color. Although the inducer is often a secondary quality, it can also be a primary quality such as size, shape, and motion.
Perhaps the fact that the concurrent is rarely a primary quality also accounts for why synesthesia is almost always one-directional, that is, synesthetes who experience an inducer-concurrent pair (I, C) will not experience that pair in reverse (C, I). One exceptional instance is Julie Roxburgh who sees color when she hears sounds and hears sounds when she sees colors (7, pp. 102-103). Each color produces a musical note. However, as we might expect, this leads her to have a kind of psychopathology where there is sensory overload and she has serious problems functioning in everyday life, including walking and navigating through traffic: “The onslaught of cacophony results in considerable perceptual interferences and causes her distress” (7, p. 102). She “feels frightened and exhausted… (it is) difficult to avoid traffic and people and to keep control…every one of her senses is ‘being battered’… (the) neon lights are shouting (and)…flashing lights give her a tactile sensation in her fingers” (7, p. 102). This description of her life, at least, certainly runs counter to Sollberger’s characterization of synesthesia as enhancing cognitive fitness and not as a disabling or dysfunctional biological trait. Nonetheless, Sollberger may unknowingly be pointing to the reason why bi-directionality is so rare.
Perhaps even more important for my immediate purposes is the fact that there are rarely, if ever, cases where there is a secondary quality inducer and a primary quality concurrent. There are some unusual forms of synesthesia which might appear to fit this description, such as audio-motor synesthesia (7, p. 40). However, in this case, we have a boy who felt compelled to move his body into various poses in response to the sounds of words. It was not as if he experienced the motion of outer objects when he heard these words. Otherwise, I would think that his daily life would be extremely difficult as a practical matter, analogous to Julie Roxburgh.
It seems to me that, as long as there is some internal individual experiential consistency among experienced concurrents, there is little worry about incoherent and very disruptive experiences. In contrast, the potential for disruption and difficulty successfully interacting with the world results (or would result) more often in cases where concurrents involve primary qualities such as size, shape, and motion. It is true that some synesthetes do talk about seeing a black letter as, say, both black and orange which would seem contradictory and potentially disruptive. However, these synesthetes are presumably not quite saying that they experience objects or letters as black and orange all over at the same time and nothing is changing in location or size. Some will describe the two colors as though a colored transparency (e.g. orange) is placed on top of the black grapheme. On the other hand, it would be much more difficult to understand what it would possibly be like, say, if a type of colored object appeared to be both large and small, or in motion and at rest, or both square and triangular. Similarly, it is difficult to see how one could coherently experience one type of food smell as inducing the experience of that food moving or being larger than the other food on a plate. The same can be said for systematically experiencing a type of colored object, say, moving in ways that other colored objects do not. For one thing, many objects have more than one color. And try to imagine, for example, judging the distance between objects. How could one drive or play a sport without becoming paralyzed into inaction? How could one engage in the most basic interactions with others? This would seem not only to threaten the coherence of such conscious experiences but, we might say, the very unity of consciousness. Further, these kinds of abnormal conscious experience would most certainly be more noticeable to others and highly debilitating to the people in question.
As a matter of fact, some of the possibilities might even resemble some rather bizarre known psychopathologies. For example, Alice in Wonderland Syndrome (AiWS) is a disorienting neuropsychological condition that affects object size perception (17). People experience distortions in visual perception such as objects appearing small (micropsia), objects appearing large (macropsia), (objects appearing to be closer than they are (pelopsia), or objects appearing to be further away than they are (teleopsia). Size distortion may occur with the other senses as well. AiWS is often associated with severe migraines, brain tumors, and psychoactive drug use. AiWS can be caused by abnormal amounts of electrical activity resulting in abnormal blood flow in the parts of the brain. Although this condition is more often found in young people and often clears up on its own, it is clear that AiWS has a negative impact one’s everyday life.
Let us explore another psychopathology sometimes discussed in connection with synesthesia.
There has been significant discussion of the relationship between synesthesia and autism. Autism is a disorder characterized by impaired social interaction and communication, and by restricted and repetitive behavior. It is a developmental disorder that affects a child’s ability to develop social skills and engage in social activities. It is sometimes thought of as coming in varying degrees and thus called Autism Spectrum Disorder (ASD). Researchers largely agree that autistic persons have impaired empathizing skills and deception detection. There is typically a lack of normal eye contact and gaze monitoring along with a lack of normal social awareness and responsiveness, such as would normally occur when one is embarrassed (18). There are two main points of contact between autism and synesthesia that I will address here:
Autism and synesthesia seem to be in opposition with regard to neural connectivity (19, 7, pp. 241-245). Recall that there is some evidence for neural “hyperconnectivity” in synesthetes, that is, a “cross-activation” or “cross-wiring” of adjacent brain regions (1). For example, the parietal cortex especially has been found to be hyperactivated in different types of synesthesia, which also suggests the idea of top-down modulation of sensory areas by this higher-order associative region (9).
It has been observed that those with autism seem to have the reverse condition, that is, decreased neural connectivity in certain brain areas. Cytowic and Eagleman explain that “it is enticing to consider (autism’s) opposite nature from synesthesia. Neural cross talk is reduced in autism but increased in synesthesia” (and) “autistics are less likely to be fooled by certain illusions…less susceptible to visually induced motion” (7, p. 242). Hirstein also argues independently that some recent evidence points to widespread underconnectivity in autistic brains (19).
The matter is not so simple, however. Hirstein also points out that this lack of frontal connectivity appears to be accompanied by increased local connectivity in the posterior cortex. He explains that “Monk et al. found increased connectivity between the posterior cingulate cortex and temporal regions in subjects with autism. In addition, they found that increased repetitive behaviors, a core diagnostic symptom of autism, were associated with increased connectivity between the posterior cingulate cortex and the parahippocampal gyrus” (19, p. 254; 20). It is also worth noting that Baron-Cohen and colleagues published a case study on a rather unusual man, Daniel Tammet, with synesthesia, autism, and savantism (21, 22). They suggested that co-occurrence of ASD and synesthesia might increase the likelihood of savantism but there is also evidence for a link between ASD and synesthesia, which is perhaps further support for the notion that synesthesia can benefit those who have synesthesia.
Adams and Shreve have argued that both synesthesia and autism are potential problems for the view that perceptual states have conceptual content as well as for the higher-order thought (HOT) theory of consciousness (23, 24, 25, 26). They focus on what we might call “the perceptual-conceptual divide.”
For those unfamiliar with the higher-order thought (HOT) theory, it says that what makes a mental state M a conscious mental state is that there is a HOT to the effect that “I am in mental state M.” One question that should be answered by any theory of consciousness is: What makes a mental state a conscious mental state? So, for example, my desire to drink some water becomes conscious when I am (non-inferentially) “aware” of the desire. Intuitively, it seems that conscious states, as opposed to unconscious ones, are mental states that I am “aware of” being in some sense. For various reasons, HOT theorists believe that it is best to construe such “meta-awareness” as thoughts constituted by concepts. Conversely, the idea that I could be having a conscious state while totally unaware of being in that state seems odd or perhaps even contradictory. A mental state of which the subject is completely unaware is clearly an unconscious state. For example, I would not be aware of having a subliminal perception and thus it is an unconscious perception. HOTs, since they are thoughts after all, are constituted by concepts. It is worth noting also that when a conscious mental state is a first-order world-directed state the higher-order thought (HOT) is not itself conscious. When the HOT is itself conscious, there is a yet higher-order (or third-order) thought directed at the second-order state. In this case, we have introspection which involves a conscious HOT directed at an inner mental state. When one introspects, one’s attention is directed back into one's mind.
So Adams and Shreve first explain that:
“Ramachandran was thoroughly investigating as many ways as he and his researchers could think of to test whether (grapheme-color synesthesia) was conceptual (or ‘top-down’) vs. perceptual (not driven by conceptual association or deployment) (23). In a ‘pop-out’ experiment, Ramachandran produced a grid of 5s and 2s that were mirror images of one another (27). The grid was presented for about one half second. To a non-synesthete, looking at the grid produced only the experience of random figures. The subjects had to press one of two buttons on a computer depending upon whether they saw a triangle or a circle….Twenty ‘normal’ subjects scored about 50% on whether the shapes were circles or triangles….However, when subjects with synesthesia looked at the grid, the colors that they saw on the numbers caused the shapes to pop out. That is, the 2s were arranged either in a circular pattern or a triangular pattern among the 5s (which were randomly placed). The colors seen when observing the 2s and their shapes in circular or triangular pattern were apparent to them at a hit rate of 80-90%. For example, a pattern of 2s that was triangular jumped out as a red triangle….The subjects with synesthesia experienced something the subjects without synesthesia did not. The reason this is interesting in regard to HOT theories, is that the ‘popout’ phenomena is a bottom-up visual experience. The subjects did not first see the shape (triangle or circle) and then have the higher-order thought (‘triangle’ or ‘circle’) causing the experience of the shape to become conscious. Rather, the perceptual pop-out produced the conscious visual experience of the shape prior to the having of the thought about the shape experienced” (23, pp. 253-254).
The experimental results themselves are uncontroversial since it seems rather well established that they show the popout experience is very real to synesthetes. Still, it can be argued that HOT theory does have the resources to account for synesthesia and the specific worries that they advance in their paper, such as the relationship between concepts and experience and the ability to explain instances of “pop-out” experiences (28, 29). Recall from section one that there are so-called “higher” and “lower” synesthetes in grapheme-color synesthesia (1, 7). Notably, lower synesthesia is rare. Adams and Shreve seem to have ignored the prevalence of higher synesthesia which explicitly involves the “meaning” or “concept” of the grapheme (23). It is the meaning of the grapheme that induces color, not the visual shape itself. Letter capitalization and font size generally do not change an induced color. For example, J, j, and J evoke the same color experience. Much the same goes for numbers and number concepts, that is, higher synesthetes experience the same color (e.g. blue) when seeing both the Arabic numeral ‘5’ and the Roman numeral ‘V.’ More generally, the claim might be framed as the view that “conscious perceptual experiences can represent objects as falling into fairly abstract conceptual categories” (22, p. 152). The matter is even more complicated, for example, some grapheme-color synesthetes may have different color experiences when seeing a “5” as opposed to seeing numerous “2’s” arranged in the shape of a “5.” Although HOTs, with their constituent concepts, are necessary for conscious states, the evidence here also seems to indicate that a conceptual component is intimately involved in these synesthetic experiences. This concept application itself can of course also occur unconsciously and almost instantaneously. The conscious experience of the colored number need not occur prior to the HOT and concept application.
The idea that concepts and cognitive states can affect one’s very perceptual experiences is more recently referred to as “cognitive penetration” (30, 31, 32) which also seems supported by the neuroscientific evidence on synesthesia. Recall again our discussion of “disinhibited cortical feedback” between brain areas such that information is processed in a bottom-up fashion but also that later stage brain activation feeds back to activate earlier stages. It is this abnormal feedback that causes these unusual synesthetic experiences (6). In addition, Cytowic and Eagleman explain that, for higher synesthetes, V4 cross-activiates with the anterior inferior temporal (AIT) cortex which processes conceptual representations of words, letters, and numbers (7). Further, it seems that semantic memory can affect sensory perceptions (33).
Thus, I disagree with Adams and Shreve when they suppose that thoughts, unlike experiences, involve concepts. Perceptual experiences are not concept-free. With regard to synesthesia, then, it seems to me that their either/or question presents a false dichotomy: is “synesthesia a conceptual or perceptual phenomenon?” (23, p. 253). My own view is that such experiences are both conceptual and perceptual as is the case with all conscious experience (26).
Regarding autism, Adams and Shreve say that:
“subjects with severe forms of autism are susceptible to pop-out synesthesia of the kind that we described in our initial paper (24, 23). Now a hallmark of severe autism is what Baron-Cohen called ‘mind-blindness (18).’ This is the inability to apply mental concepts to self or others. People with severe autism have no trouble understanding people as physical systems with physical properties…But when it comes to beliefs, desires, intentions, hopes, fears, wishes and other mental causes, severely autistic individuals simply do not understand behavior originating from these causes. Such purposive behavior is a complete mystery to them. Thus, they do not engage in applying mental concepts to themselves or others. Consequently, when a person with severe autism consciously experiences the pop-out of synesthesia, it cannot be the result of applying an HOT to their experience because they don’t employ HOTs about mental states (of self or others)” (24, p. 133).
But the ‘mind-blind’ characterization of autism, even in the more severe cases, is mistaken or at least greatly exaggerated. It is not at all clear that autistic people cannot have or apply any mental concepts to themselves or others (26, 28). One problem with the autism literature is that some authors who argue for a deficiency in ‘self-consciousness’ among autistic individuals leave the term undefined. This is important especially since it seems that self-consciousness, self-concepts, I-thoughts, concept possession, and so on can come in degrees. At the most sophisticated level, there is introspection or reflection. Even if there are deficiencies in introspection, it does not follow that there are no I-thoughts or metacognitive states at all. It is one thing to suppose that autistic people have abnormal or impaired self-consciousness, but quite another to claim that there is no self-consciousness at all. Indeed, despite their own skepticism regarding autistic self-consciousness, Frith and Happé themselves quote numerous cases of first-person reports from autistic people (34, pp. 11-14).
I have engaged critically with Sollberger’s view that there is reason to think that at least some synesthetic experiences can be viewed as truly veridical perceptions, and not as illusions or hallucinations. With the help of the traditional Lockean primary-secondary quality distinction, I explored the possibility that many forms of synesthesia can be understood as experiencing what I will call “second-order secondary properties,” that is, experiences of properties or qualities of objects induced by the secondary qualities of those objects. Depending on the definition of hallucination, it may be that some synesthetic experiences are hallucinatory in at least some sense. In the process, I have also attempted to shed light on why synesthesia is virtually always one-directional, that is, the greater potential for difficulty successfully interacting with the world where concurrents involve primary qualities such as size, shape, and motion. Finally, I briefly addressed synesthesia’s relation to autism partly based on evidence regarding neural connectivity and the conceptual component of some synesthetic experiences.
anterior inferior temporal cortex
visual word form brain area
Alice in Wonderland Syndrome
Autism spectrum disorder
anterior inferior temporal cortex