- What do we need to learn? - What is our riskiest assumption? - What is our one priority? e.g. "To understand the homepage messaging and design theme that drives the most conversions to activated users"
- Is it Relevant to the learning goal? - Is it falsifiable and specific? - Is it causal? e.g. if X then Y. e.g. "If the experiment is successful, I predict that we will create 250 new downloads per week because of a +7% increase in the View > Download conversion rate."
- Is it Specific? - Is it Achievable? - How will you collect the data?
- How much development effort is required? - How much people are required to run this experiment? - How is the Probability of this experiment to succeed? Low -20%, Medium - 50%of High - 80%
- What happened? - Did we succeed or fail? - What data did we collect? - Anything unexpected? - How close have we been to our prediction? - Why did we see the result that we did?
-What did we learn by running this experiment?
- Another experiment for this goal? - How do we progress from here on?