Recently I’ve been listening to the Talking Machines podcast, a show about machine learning. The machine learning community has a wide variety of researchers in academia and tech companies doing theoretical and applied research to try to solve hard problems (e.g, How can we tailor medical treatments specifically to your DNA?).
One of the tools the community uses to keep pushing forward in specific ways is through contests. The Netflix Prize is a good example of this, where Netflix awarded $1 million to the best team to improve their recommendation algorithm. Contests are great because they motivate many teams to work on the same problem in a way that un-focused research doesn’t.
Another tools the community uses is academic conferences. Researchers write papers describing their methodology and results, those papers are judged for quality and originality and the best are selected for presentation and publication in the conference’s proceedings. This helps researchers build their reputation and satisfies the universities employing these folks (publish or perish).
The problem is, there’s no one publishing negative results (failures).
Failure is simply the opportunity to begin again, this time more intelligently. —Henry Ford
I have not failed 700 times. I’ve succeeded in proving 700 ways how not to build a lightbulb. —Thomas Edison
Without publishing failures, everyone has to independently discover all those ways that don’t work. Imagine Edison had been working in parallel with dozens of others working on the lightbulb problem (he was!). Since he had no way to let others know about his failures, nor any incentive to do so, all of his peers (competitors, in a way) had to repeat those same 700 failures (or maybe they gave up before getting that far), a terribly inefficient progress mechanism.
This may be all fine and good when you think about your competition. Why would you want to help them, anyway? But what about internally, inside your company? What happens to failures? Do you fire the people involved, reassign them to your version of Siberia, quietly bury the project and hope word doesn’t get out? If you do any of these things, you’re replicating the inefficiencies described above.
Instead, why not publicize those negative results internally, giving others the benefit of learning what one more thing that doesn’t work? You could literally publish stories of these negative results in an internal newsletter, start a failure wall (stencil the Thomas Edison quote on a blank wall and tape a Sharpie nearby and watch what happens) or come up with your own creative way to encourage people to learn from failures, their own and others.
Leave a Reply