Want to invent the next iPod? Then don't try too hard. We may be able to train our minds to be better at generating ideas, according to recent thinking on how we think, and often the best way to foster a brilliant idea is not to push it.

Nobel laureate physicist Richard Feynman used to visit a topless bar, sip a soda and scribble quantum mechanics on a napkin. Einstein's theory of special relativity came after he imagined himself a child riding on a beam of light.

And Greg Swartz, director of innovation at the golf company Ping, says he has come up with 36 ideas for better tees and loftier drives by looking at the stars. After immersing himself in his subject matter, he'll go to his backyard at night and let his mind settle into what he calls a "hyper state" when it is firing on all cylinders. He says it's as if he can almost feel the rush of gamma rays that are said to emanate from the right hemisphere when an idea is born.

Brain scans have revealed that when you think you're not thinking, your unconscious mind may be doing wind sprints searching for a perfect solution. As a result, answers sometimes seem to appear out of nowhere. In reality, that "nowhere" is beneath your consciousness. In studies, these out-of-the-blue insights are more frequently associated with novel, creative solutions than those derived from concentrating hard, according to cognitive neuroscientist Mark Jung-Beeman, of Northwestern University.

via online.wsj.com

I agree with this insight entirely. The mind is complex, and its workings are not intuitively revealed through simple introspection. According to at least one credible theory, our conscious thoughts are post-facto "explanations" that our left, verbal, brain invents so as to provide a rational narrative explaining what our left, creative / emotion-driven brain, has already decided to do or worked out as a solution to a problem within an entirely different cognitive framework.

I'm reading Edward Tufte's "Beautiful Evidence" at the moment. It's rich with content and possible application that initiates multiple threads of thought. I find myself pausing, literally putting the book down every page or two, just to allow those thoughts time and space to percolate. Only a small fraction of them have bubbled up to the surface of conscious awareness. Many others are down there, brewing, likely to pop up when I least expect them.

Want to bring your best, yet-to-surface thoughts up to where they can do some good? Here are some ideas I've found to work:

1. Change of venue: Get up, get out, put yourself in a different setting. Listening to the Dead's Truckin' right now, I'm reminded of the time in my senior year at SUNY Stony Brook when, after a whole day and evening struggling to debug the compiler I had designed, I said, "That's it" (or something to that effect), got up, walked across campus to James Pub, and ordered a pitcher. Truckin' Sample

   Sometime later, somewhere in the middle of the third serial playing of Rosalita (Come Out Tonight), loud, the solution to my problem popped up, out of nowhere. I was thinking about the young lady across the room, not software.

2. Draw it out: I often use mind mapping software to sketch out my ideas. By doing so, I'm sure that my left brain gets engaged in ways it wouldn't if I only used words (although outlining also works well for me in the case of problems where the broad idea is already at hand). I like MindManager.
3. Collaborate: Sometimes your best thinking is done alone, but often the creative interplay that happens when you brainstorm with others brings out ideas that otherwise would not emerge.