How do I create lean experiments?

Several people wrote in in response to last week’s post regarding Lean Experiments for finding People of Peace.

All were essentially asking for more ideas about experiments to run, or how to create experiments. In this post, I’m briefly going to share the system I use for creating “Lean Experiments.” The idea is to have something very simple, which enables you to run new experiments just as fast as the last experiment can be evaluated.

On a large table, I’ve set up a series of 9 Post-It Notes, running vertically down the table one right after the other, each forming a “row.”

The Post-It notes are, in order:

• Leap of Faith Assumption
• Hypothesis (Specific, Measurable description of LoA)
• Experiment: minimum req’d to test hypothesis
• Behavior: how experiment shows results (aim for visible action–no surveys)
• Success Goal / Target Metric / Commit to the Finish Line
• Actual Result
• Why / Observed Behavior
• New Insight
• Resulting Decision

I “create” an experiment simply by putting Post-It notes to the right, in a column, answering these questions.

The Leap of Faith Assumption is the broad overarching idea you have.

“There is a Person of Peace in this town.” The Hypothesis is the specific thing you want to test with regard to the LoA: “People of Peace can best be found through interactions in marketplaces.” You can develop hypotheses by mapping out a place and asking questions about who/where/when/how People of Peace might be found.

Markets? Streets? Business relationships? Refugee camps? Clinics? Mornings? Afternoons? Evenings? Now, you need an Experiment to test the Hypothesis (basic science here).

“I will visit the market once a day, using x story set.” As a result of the experiment, what “behavior” would be a success? This behavior needs to be visible.

For some businesses, it would be signups or sales or the like.

For you, it might be a response that suggests a Person of Peace, whatever that response may be (ask for more stories? positive response to suggestion of a Bible study or the like?).

Success Goal: define a level of behavioral response which represents a success line.

Choose the target you’re reaching for before you see results.

(We aren’t shooting arrows at the tree and then painting targets around them.) Actual Goal: the actual results of the experiment you ran.

Why/Observed Behavior: ask yourself why you got the results you did.

Were they close to the success goal, and maybe just need tweaking? Were people not responding at all? Was it a bad time of day–too crowded? Look for insights.

New Insight: this is what you come away with, that forms the basis for your Decision.

Decision: this is what you decide on the basis of the experiment.

It may be to drop the story set, or change times, or whatever.

A decision usually leads into a new experiment.

By using an approach similar to this – doesn’t have to be this exact one, but this has a lot of user testing behind it – you are documenting your experiments and learning from them.

It’s a lot better for you to use a system of rapid experimentation and iteration than for me to try to give you tactical ideas that someone else has used, which you’ll have to adapt anyway.