Foreword, by Pentti Kanerva

We think of some sciences, by their very nature, as being more mathematical than others. We even refer to physics and other heavily mathematical sciences as "hard sciences," as if to imply that social sciences and humanities are somehow soft or easy. Another way to see this picture, however, is that mathematics itself is hard and that it would be applied first to sciences that themselves are relatively easy. Let us stay with this idea for a moment.

Mathematics is hard, in the sense that it is not in our nature. Like reading and writing, or playing a musical instrument, it takes a great deal of practice to become good at it. Mathematics is also an ever-growing field that builds on its earlier discoveries. Thus it seems reasonable that we would discover the simpler parts of math first and that they would be sufficient for describing phenomena that are relatively simple. However, the thoroughness with which mathematics has imbued the ``hard'' sciences testifies of its power to aid our thinking once we have found the appropriate mathematics.

It matters greatly that the mathematics be appropriate, and there are conspicuous counterexamples. Early math dealt with numbers, ratios, and geometric shapes, and these were sufficient for surveying the land and describing musical sounds pleasing to the ear. Attempts to explain the wandering planets or the basic elements of nature by the five perfect solids, however, have failed the test of time and earned the name pseudoscience. The math that we now use to explain the motion of the planets was only discovered two millennia later.

Once a field finds its math, they begin to feed on each other and coevolve. Old scientific puzzles get satisfactory explanations, new puzzles inspire further developments in math, and the math points to new questions in the science---to what else might be true and worth investigating. That is how physics, for example, has become so thoroughly mathematical.

Can this happen with anything as rich, varied, and complex as human thought and language? My guess is that it will once the appropriate math has been found, and the key is that it be appropriate. This may sound circular and therefore needs clarification.

The common notion that mathematics is about calculating and proving theorems is far too limited. Mathematics is the study of patterns: of things that repeat and recur in different contexts, of structures and relations that are common to different fields and situations. Mathematics seeks regularities in the behavior of natural and artificial systems and so it is the abstract study of things that are understandable to us---assuming that we are finite physical beings and thereby limited in our understanding.

Mathematics is a tool for coping with complexity. The world is infinitely complex from the human perspective, but the more regularity we find in it the better sense it makes to us. Finding the regularities is the key, and dressing them into theorems and proofs is mostly a style of communication among mathematicians. This style is relatively safe and effective, although it belies the process by which mathematicians arrive at their results. Such a style can be compared to rhyming in poetry: how rhyme helps us remember the verse. What matters is the idea behind the theorem and its proof, as is the image or feeling behind the verse.

The mathematics for describing the motions of planets allows us to calculate their positions for centuries to come, whereas no amount of calculation will let us predict exactly what a person will say or do tomorrow. This is commonly interpreted in one of two ways, either that mathematics does not apply after all, or that the proper math here would be probability because we can at least guess what a person might say or do.

The first interpretation reflects the overly limited view of mathematics noted above and can be dismissed on those grounds. The second ascribes too narrow a role to mathematics and thus misses the mark. It overemphasizes prediction---if we cannot predict exactly then let us predict probabilistically---and assumes that the needed math is already fully developed and need only be applied. In language, for example, central aspects such as structure and meaning require their own kind of math in which probability plays only a side role. Much of that math may not even exist yet but will be developed concurrently with our increasing understanding of structure and meaning.

This brings us to the substance of Dominic's book: the exploration of mathematics that would be appropriate for describing concepts and meaning. The branch of mathematics that has traditionally been associated with meaning is logic. Logic is also discussed here, yet the main emphasis is on identifying mathematical spaces that would lend to modeling of meaning.

High-dimensional vectors make up one such space---or, rather, a family of spaces---and the distance between two vectors is a basic notion of a vector space. Distance, or closeness, is a geometric notion and so we enter the realm of geometry. When words are represented by vectors, the closeness of vectors can represent the similarity of their meanings. In mathematics, the elements that make up a space are called points of that space. Here the points are vectors, but a distance between points is a valid idea in many mathematical spaces and thus can model similarity of meaning more generally.

Probability becomes a part of the vectors when they are based on word frequencies in large bodies of text. The essence of this book, however, is not probability itself but the mathematical structures that the probabilities reflect, such as graphs and lattices. By matching more closely the mind's underlying mechanisms, these structures can provide deeper insight into language and meaning than would the probabilities by themselves. With regard to logic, the vectors allow traditional logics to be generalized to "quantum" logics that correspond more closely to how words are actually used.

The human sciences are young from mathematics' perspective, and extremely challenging problems abound. For example, how do words and language expressions acquire meaning? They become meaningful to us in various ways: from contexts in which they appear, from examples of their usage, and by being defined in terms of other words and expressions, as in a dictionary. Meanings are also transferred readily to new domains by analogy and metaphor. But what representations and underlying mechanisms---what physical processes---would have these properties? What mathematical structures would show similar behavior and allow it to be analyzed in depth?

Everyday language is transparent to us but when we try to program computers to use it, it proves to be endlessly ambiguous. Obviously, how we represent language in computers---that is, our mathematical model of language---is grossly inadequate. A more faithful model would have great scientific and practical value.

Our ability to learn language is one of nature's marvels. It gets credited to innateness or instinct, which no doubt is correct, but we should not stop there. We should also ask, what are the underlying mechanisms, and what kind of a model would adequately explain them? The model would be a mathematical abstraction of how the brain's circuits work, but the mathematics to describe the model may not even exist yet.

Complex learning in general is poorly understood in terms of underlying mechanisms. Much of it takes place in social settings in which we observe and absorb into ourselves the behavior of others, and this kind of learning is important in the animal world at large. A mathematical model that ties learning to underlying mechanisms would deepen our understanding of ourselves as social beings. It would also take us a long way toward the building of artificial systems that learn and adapt in ways that humans and animals do.

I may seem to be overselling mathematical modeling, but I am emphasizing it for a reason. We humans are very limited in our ability to imagine and see things in a new light. Even when our concepts and explanations are badly wanting, we tend to hold onto them. Anything that can help us surmount our limitations should therefore be more than welcome. We fare much better when we bootstrap by using analogy---that is, relate the new to something we already know, or model the unfamiliar with the old and familiar.

Here is where mathematics can help. It is a rich source of models, capable of spurring new branches of math and new models to capture whatever regularity interests us, and so it has the potential of serving any field of study. A crucial part is still left for human creativity, namely, seeing some behavior and imagining an underlying system that could produce it. The deeper our knowledge of the behavior and the richer our repertoire of models, the more likely we are to make a meaningful connection. In the interplay that follows, the behavior suggests extensions of the math and the model, and the model predicts new facets of the behavior. When the right conditions for discovery are provided, a stroke of genius is not merely a stroke of luck any more.

This book is a mathematician's invitation for researchers in the human sciences to look into new areas of math, and it gives an illustrated tour with many examples from language. It is not a book of techniques---there are other sources for that---but of the ideas behind the techniques that would be helpful for understanding existing techniques and for discovering new ones. This is useful to us all, yet to young people in particular I recommend an even broader study of mathematics so that you can make it serve you on your terms and not be limited by what others have done already. To use a military metaphor, you want this arsenal at your disposal. The book can also help mathematicians to identify challenging and worthy problems to which to apply their skills.

The writing is congenial, which is rare among texts in mathematics. It conveys the author's appreciation of, and his eagerness to share with others, the beauty in the subject and the humanity in the people who have brought it to us over the millennia. By going to original writings he has discovered how the story has changed in the telling by those who did not fully understand it. This has allowed him to repair the reputation, as a mathematician, of even someone like Aristotle.

I will close with a bold challenge: mathematics is too important---it is too useful in solving fundamental problems in fields other than math---to be left solely to mathematicians. Social scientists, linguists, philosophers, mathematicians, all have much to contribute and gain.


Up to Geometry and Meaning
On to Introduction