Some time ago I realized that when I think very abstractly, what's going on in my mind is not for the most part natural language; instead it's more visual. Specifically, the visuals seem to be of a schematic nature, focused on indicating relationships between things. The ideas in this essay came out of my thinking about what sort of cognitive system underlying thought could allow one do it both in natural language and in something as diverse as the 'schematic thinking' I was doing. The more general system accounts for thinking in various domains; e.g. chess or algebra might be other domains you think in.
This is actually part of a larger piece I started on schematic thinking, intending to build up to something like a tutorial—but for now I just have this fragment which I think is kinda neat on its own.
(If you'd like to know more specifically what I mean by 'schematic thinking' check out the two excerpts I include at the end of this document. They are from Hadamard's "The Psychology of Invention in the Mathematical Field," but they are essentially descriptions of what I have in mind when I say 'schematic thinking'.)
----------------------------------------------------
Deliberate thought can take place in a number of different substrates, which I shall call "Domains," each of which corresponds to a domain in a more ordinary sense. The mental counterparts to Chess, algebra, social situations, physical situations, drawing, and playing the flute are all different Domains.
When a Domain is active in the brain, some State is present. A State in Chess is the board layout at a single moment of a particular game; a State in algebra is an algebraic expression or equation; a State in a social situation is mostly made up by the estimated mental state, facial expressions, and bodily gestures of a group of people; a State in a physical situation is a group of objects relatively oriented under the constraints of one's understanding of physical law; a State in drawing is the contents of a piece of paper; a State in playing the flute is the configuration of the fingers of the player combined with an exhalation strength. When thinking in some Domain, words are not present (unless language is the active Domain -- but more on that later), only a chain of States from the Domain. Although, in practice, when playing Chess for instance, the Chess Domain will be your substrate for thinking only a fraction of the time: you may swap in a social situation Domain, or a language Domain, and then jump back to the Chess Domain with new ideas -- but at that point (probably an image of the board is the principle contents of consciousness), words disappear.
Each of these Domains is defined by a set of assertions considered always true while the Domain is the active substrate; these state the elements of a Domain and limit the valid relationships between those elements. In chess the assertions would comprise the rules of the game and constraints on how a board/pieces may appear; in social situations they would correspond to an experience based collection of verities in human interaction: if I behave violently toward a person, they will return with violence -- etc; in drawing the assertions are that a mark will be left at the point of contact between pencil and paper, and a pencil may be moved while in or out of contact with paper. Each of the States mentioned before has a particular format which is a result of the assertions of the Domain; and so, it is useful to think of the assertions of the Domain as a sort of grammar for the Domain -- this will be returned to later. Now let's view the theory in the abstract.
A Domain is a collection of assertions considered always true while that Domain is active, and a collection of valid States.
A State within a Domain is a configuration of mental elements in some arrangement allowed by the assertions of a Domain.
A Line is a sequence of States, corresponding to the notion of a "line of thought."
Deliberate thought within a particular Domain consists primarily of finding a desirable Line. Between each State in a Line there is considered to be an operation which changed one State into the next; this collection of operations for a Line forms a set of actions to take in the external counterpart domain (e.g. actual game of chess).
It is useful to consider a space of all of the possible States within a Domain, called a State-space: each point is a State; each axis of the space corresponds to an operation which may take place in the Domain (e.g. a valid move in chess, yelling in a social situation), resulting in another valid State in the space; any path through this space would be a Line.
Consciousness, the work table of deliberate thought, holds a small list of States in view at any given time, with the most recently occurred being most prominent. These active States are the input to a recursive process for determining which State to add to the list next. Choosing a next State is identical to selecting an operation, i.e. selecting a direction to step in a State-space.
Thought within a Domain is exploration of a State-space. The decision making about which direction to move in next is subconscious, and does not operate in the narrow confines of the Domain itself. Every Domain has two assertions which guide this decision making process: the first is that contradictions may not occur, and the second is that composite mental constructs with similar internal relationships are interchangeable to some degree. The first assertion allows the mind to use logic, and the second enables analogy.
The Domains considered so far are of a primitive type, which presumably showed up in brains so that animals could carry out simulations within particular domains in order to estimate an optimal decision in that domain. These primitive types all seem to have a clear notion of what an operation within the domain is: moving a chess piece, adding to both sides of an equation, grinning, etc., which results in a successor State to the current one. When we consider a language as a Domain, where its assertions are the grammar and lexicon for the language (e.g. "each sentence must contain a verb," "'triumphant' is in the lexicon"), we find that it has the same basic role in the mind as the others: subconscious processes serve up successive States (sentences in the case of language) to consciousness, and the States in consciousness have an important role in determining which States will follow -- but, when we look at two successor states, no operation which transformed the first into the second is apparent with language. Now, in any Domain, the process which selects the operation is always complex and hidden, but in the primitive domains, given a State and a successor for it, we are at least able to identify the operation, once selected; not so with language.
It is a hypothesis of this theory that language is a later evolutionary development which generalized the more primitive Domain "simulation" machinery. But with language, there is no counterpart domain external to the thinker, instead language seems to be a meta-Domain, for thinking about other Domains. Its generality also allows it to translate between Domains. (Perhaps analogy didn't appear as a method for navigating State-spaces until language showed up.)
Before, I said that the assertions of a Domain define a format which the States of the Domain must be in, and that this was good reason for considering the assertions to form a grammar for the Domain -- hopefully this is more clear now that we have a Domain where the assertions are a grammar and lexicon, so that the definition of grammar undertakes only a slight expansion. But viewing them as a grammar is not just an amusing conceit, it gives an idea about how the brain is able to derive meaning from States. In linguistics and computer science, systems exist which can take expressions conforming to a given grammar and analyze them into atomic units which each have a pre-defined role; this is called parsing and is the first step to incorporating semantics in a linguistic machine. Since States in a Domain have a consistent grammar, States can be parsed and each of their elements assigned some role in the context of a semantics for a particular Domain. In this form, a subconscious process could take the labeled elements and determine a value for a State, so that some States can be considered good and others bad, and we have a system in place for preferring certain Lines over others. It is probable that the value determination of a State manifests emotionally.
In this theory then, the complex system of meanings that we experience when thinking in language is an outgrowth of the semantic systems of the more primitive Domain simulators. It is possible that the correct way of viewing a language's State-space is as an infinite-dimensional space: since it is a meta-Domain rather than a proper Domain, and must be able to talk about all other Domains, it doesn't have a finite set of operations for transforming its States -- instead, it has the general notion of operation itself, which includes all operations. Or it may be that there isn't enough structure between its States to usefully view a State-space for it; perhaps its lack of operations is a testament to its freeness -- it can jump between any States that logic or analogy suggest, guided by semantic-emotional responses to candidate Lines.
----------------------------------------
Excerpts from "The Psychology of Invention in the Mathematical Field":
"Indeed, every mathematical research compels me to build such a schema, which is always and must be of a vague character, so as not to be deceptive. I shall give a less elementary example from my first researches (my thesis). I had to consider a sum of an infinite number of terms, intending to valuate its order of magnitude. In that case, there is a group of terms which chances to be predominant, all others having a negligible influence. Now, when I think of that question, I see not the formula itself, but the place it would take if written: a kind of ribbon, which is thicker or darker at the place corresponding to the possibly important terms; or (at other moments), I see something like a formula, but by no means a legible one, as I should see it (being strongly long-sighted) if I had no eye-glasses on, which letters seeming rather more apparent (though still not legible) at the place which is supposed to be the important one."
"About the mathematicians born or resident in America, whom I asked, phenomena are mostly analogous to those which I have noticed in my own case. Practically all of them -- contrary to what occasional inquiries had suggested to Galton as to the man on the street -- avoid not only the use of mental words, but also, just as I do, the mental use of algebraic or any other precise signs; also as in my case, they use vague images."
