The Algebra of Thought: a theory of human creativity

How is it possible to have an original thought?  This question seems to be burning in the minds of various corporate executives; who have apparently come up with the idea that a good solution would be to impose an additional layer of bureaucracy  in the form of a "Director of Innovation", for some current examples see here.  On the other side of the debate, we have the notion that innovation kinda just happens if you leave "creative" people alone to "do their thing" (for example, Google's 20% time, or Valve's unique approach to project management).

It seems to clear to me that more bureaucracy will not help people be more creative, but unfortunately we are left with a difficult and profound question: how does the creative process work?

To summarize my opinion on the matter, I argue that it's not possible to have a completely original idea (the key word being "completely"); but rather that every "creative" idea is really the result of a process of breaking down idea's into their component parts, and then reassembling those idea's in a way that produces something "new".

This raises several interesting and important questions:

If thoughts can be broken down into component pieces, is there such a concept as an "indivisible thought", or a "fundamental building block" of ideas?  And if so, what is the nature of these fundamental building blocks?  Furthermore, is it possible to construct an "algebra" so that we can have a notion of "adding" two thoughts together, or other relevant operations such as subtraction?

One possible answer to the question is to say: since our knowledge of the outside world is purely a result of our 5 senses, then the fundamental building blocks of thought should correspondingly be the sense input we receive from them.  

For example, I know what an elephant looks like, I know what the color pink is, therefore I can "add" these two ideas together to come up with the "new" idea of a pink elephant.

Similarly, I know what an elephant smells like and sounds like, I also know how a flower smells, what an alarm clock sounds like, and what it feels like to be stung by a bee.  So now I can come up with the original idea of a pink elephant that smells like a flower, sounds like my alarm clock, and stings like a bee!  I'm so creative aren't I!

Certainly the above idea is pretty dumb, but the process is actually the process by which humans are creative: we break down idea's we already have into smaller parts and then recombine them to come up with something "new".  There is an excellent series of video's called "Everything is a Remix" which argues the same point, and which I recommend you watch, if you have not done so already.  

A question which should be raised at this point is: if every thought can be broken down into building blocks which come from the 5 senses, then how is it possible that people can understand mathematics, or other abstract concepts?

I claim that the answer is that mathematics is a language, that the human mind has a natural capacity to understand language, and that the building blocks of thought are not sense experiences in and of themselves, but rather these building blocks are "abstractions" which are processed by the language centers of the brain.  Consider that language itself is perfectly suited for "adding" and "subtracting" ideas together, we simply add some words to a given sentence, or else remove some words from a given sentence.  It seems likely that the creative process works something like this: we abstract away from our sense data "ideas" which can be processed by the language centers of the brain, which has a coherent and consistent system for adding, subtracting  and recombining these "ideas", and then we can express those ideas and give them life in the external world in the form of some kind of object that we can experience with our senses.

  

What Separates a Great Programmer and a True Master.

The difference between an average programmer and a great programmer is very large indeed, certainly much larger than say, the difference between an average construction worker and a great one.  This observation has been discussed in numerous places, and I especially like Jeff Atwood's post on the subject over at coding horror.  In fact, I would argue that the difference between a great programmer and a true master is just as large, if not more so.

1. A master programmer is capable of learning a new language extremely quickly; not because they are so much smarter than everyone else, but because they understand how computers work on a deep level, and they understand that programming languages are merely an interface.

Learning a new programming language is certainly much less challenging than learning a new spoken language.  Why is that?  Whereas spoken languages were developed independently by many different groups of people over the entire course of human history, programming languages have been around for less than 100 years and share common historical roots.  Whereas spoken language is by it's nature ambiguous, programming languages are extremely precise.  Whereas learning a spoken language requires a great deal of memorization, learning a programming language requires a great deal of understanding.

There is just way more stuff to memorize when trying to learn a spoken language, even for a simple conversation; and although in programming you must memorize certain things, a programming language is a "machinery", and once one understands how the "machinery" works, memorization is not as useful as knowing how to figure things out from first principles.  The great masters understand that the machinery of programming languages is dictated by the fact that a programming language is merely an interface which communicates with the computer.

2. A master programmer strives to maintain maximum generality whenever possible, and is able to dramatically increase their productivity by manipulating data the "best" way, and choosing the "best" tool for every task,

A great programmer is a hacker by nature, but a true master avoid "hacks" whenever possible because they understand that it's far too easy to get backed into a corner if you do not plan properly, and then productivity suffers because it is necessary to start over from the beginning.   Programming can be extremely rewarding, as well as extremely frustrating; it can be tempting to implement a quick fix, or a "hack", when you get into a difficult situation, and sometimes it's necessary to do just that.

A true master is able to be so much more productive because they organize their code, and their time, in such a way that they maintain flexibility and they can always keep plugging away because by maintaining generality they maximize code reuseability.  Although it might take a bit more time to write code at the highest level of generality one time, the fact that this same code can be reused will pay for itself over and over again in terms of increased productivity in the future.

3. A master programmer is able to very quickly plan out solutions for highly complex problems because they are as much mathematicians as programmers.

Although it's true that a great programmer can pretty much manipulate data however they want, the great masters are able to manipulate data the "right way", in the sense of maintaining maximum generality, and they are able to rapidly find solutions where others cannot because they are able to abstract the mathematics of the situation, from what the data is supposed to represent.  In this excellent answer from math.SE on the question of why mathematicians use single letter variables, Dan Peterson writes:

I think one reason (that mathematicians use single letter variables) is that often one does not want to remember what the variable names really represent.  As an example, when we choose to talk about the matrix A_ij, instead of the matrix TransitionProbability_ij, this expresses the important fact that once we have formulated our problem in terms of matrices, it is perfectly safe to forget where the problem came from originally -- in fact, remembering what the matrix "really" describes might only be unnecessary psychological baggage that prevents us from applying all linear-algebraic tools at our disposal.

A great programmer is able to apply things they learn in one language to other languages, and for this reason learning a new language is a great way to increase one's skill level.  Similarly, a great master is able to apply things they have learned from studying the language of pure mathematics to various programming problems, and this can have an even more profound effect.

On the one hand, being skilled in coming up with and writing rigorous proofs is a great way to train oneself to being able to come up with a plan of attack for a difficult programming problem.  On the other hand, being able to formulate a programming problem as a pure mathematics problem is only as useful as the level of mathematics you have mastered.  Indeed you will never be able to make progress if you get to a point where you start trying to reinvent something like algebraic topology.  It's important to study higher mathematics precisely so that once you have abstracted away the math from the code, that you know where to look to find out how to solve it, or better yet that you remember the solution.  This is the secret by which a great master is able to so rapidly solve problems which a great programmer is either incapable of solving, or else can only solve partially after an extreme force of will.