Publish Negative Results

Recently I’ve been listening to the Talking Machines podcast, a show about machine learning. The machine learning community has a wide variety of researchers in academia and tech companies doing theoretical and applied research to try to solve hard problems (e.g, How can we tailor medical treatments specifically to your DNA?).

One of the tools the community uses to keep pushing forward in specific ways is through contests. The Netflix Prize is a good example of this, where Netflix awarded $1 million to the best team to improve their recommendation algorithm. Contests are great because they motivate many teams to work on the same problem in a way that un-focused research doesn’t.

Another tools the community uses is academic conferences. Researchers write papers describing their methodology and results, those papers are judged for quality and originality and the best are selected for presentation and publication in the conference’s proceedings. This helps researchers build their reputation and satisfies the universities employing these folks (publish or perish).

The problem is, there’s no one publishing negative results (failures).

Failure is simply the opportunity to begin again, this time more intelligently. —Henry Ford

I have not failed 700 times. I’ve succeeded in proving 700 ways how not to build a lightbulb. —Thomas Edison

Without publishing failures, everyone has to independently discover all those ways that don’t work. Imagine Edison had been working in parallel with dozens of others working on the lightbulb problem (he was!). Since he had no way to let others know about his failures, nor any incentive to do so, all of his peers (competitors, in a way) had to repeat those same 700 failures (or maybe they gave up before getting that far), a terribly inefficient progress mechanism.

This may be all fine and good when you think about your competition. Why would you want to help them, anyway? But what about internally, inside your company? What happens to failures? Do you fire the people involved, reassign them to your version of Siberia, quietly bury the project and hope word doesn’t get out? If you do any of these things, you’re replicating the inefficiencies described above.

Instead, why not publicize those negative results internally, giving others the benefit of learning what one more thing that doesn’t work? You could literally publish stories of these negative results in an internal newsletter, start a failure wall (stencil the Thomas Edison quote on a blank wall and tape a Sharpie nearby and watch what happens) or come up with your own creative way to encourage people to learn from failures, their own and others.

About

I free up entrepreneurs for greater impact. Day to day demands of your growing business will sap your energy and blunt your effectiveness. Together we can work to reduce demands by up to 80% while moving more quickly in the direction of your purpose.

Posted in Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: