Olsen, Dan
I will start with the conclusion chapter as this is a great mini summary of the book
Chapter 15 : Conclusions
It began with the Product-Market Fit Pyramid—an actionable model that defines the components of product-market fit and how they are connected. Your market consists of your target customers and their needs, and your product is the combination of your value proposition, feature set, and user experience. When you try to achieve product-market fit, you make critical hypotheses at each of these five layers. The Lean Product Process guides you through the formulation and testing of your hypotheses with these six steps:
- Determine your target customers
- Identify underserved customer needs
- Define your value proposition
- Specify your minimum viable product (MVP) feature set
- Create your MVP prototype
- Test your MVP with customers
The process starts in the problem space and progresses to the solution space. You begin by determining your target customers, which you describe using personas. To create the most value for customers, you use the importance versus satisfaction framework to identify their important but underserved needs. Using the Kano model, you define a differentiated value proposition that better meets those needs for your target customers. You then take an MVP approach, trying to identify the minimum set of functionality required to deliver the key parts of your value proposition. You bring your MVP feature set to life by applying the principles of great UX design to create a prototype with a usable and delightful user experience.
To assess product-market fit, you test your MVP candidate with your target customers, who can give you better feedback in the solution space than in the problem space. To save resources and iterate more quickly, you ideally test with design deliverables such as clickable or tappable mockups before actually building your product. You use the knowledge you gain to revise your hypotheses and your MVP candidate. You continue to iterate through the hypothesize-design-test-learn loop with additional waves of user testing, hopefully achieving higher and higher levels of product-market fit. As you test, you may decide to pivot to a more promising opportunity by changing one or more of your fundamental assumptions.
Once you have validated your product-market fit, it’s time to build your MVP. To reduce risk and deliver customer value more quickly, you should build your product in an incremental, iterative manner using Agile development. QA and test-driven development help achieve higher product quality; continuous integration and continuous deployment help improve the speed of your development process.
After you’ve launched your product, you employ analytics to understand how customers are using it. Your retention rate gives you a quantitative measure of product-market fit, and cohort analysis shows you how it changes over time. Once you have good retention, you can focus on improving other macro-metrics in Dave McClure’s AARRR framework (acquisition, activation, retention, revenue, and referral).
Defining the equation of your business helps you identify the key metrics for your particular business, and the Lean Product Analytics Process provides a systematic way of optimizing your metrics, resulting in greater revenue and profitability.
10 best practices for creating successful products.
- Have a point of view but stay open-minded.
- Articulate your hypotheses.
- Prioritize ruthlessly
- Keep your scope small but focused
- Talk to customers
- Test before you build
- Avoid a local maximum
- Try out promising tools and techniques
- Ensure your team has the right skills
- Cultivate your team’s collaboration
Introduction: Why Products Fail and How Lean Changes the Game
The main reason products fail is because they don’t meet customer needs in a way that is better than other alternatives .
startups “ fail because they never get to product – market fit . ”
The framework , which I call the Product – Market Fit Pyramid , breaks product – market fit down into five key components : your target customer , your customer’s underserved needs , your value proposition , your feature set , and your user experience ( UX ) .
Why this book
The Lean Product Process consists of six steps : Determine your target customers
Identify underserved customer needs Define your value proposition Specify your minimum viable product ( MVP ) feature set Create your MVP prototype Test your MVP with customers
host a monthly Lean Product meetup in Silicon Valley , which I invite you to check out at http : / / meetup.com / lean – product
How is this book organized
I’ll use the companion website for this book , http : / / leanproductplaybook.com ,
Part I: Core Concepts
Chapter 1: Achieving Product-Market Fit with the Lean Product Process
“ Product – market fit means being in a good market with a product that can satisfy that market . ” My definition of product – market fit — which is consistent with his — is that you have built a product that creates significant customer value . This means that your product meets real customer needs and does so in a way that is better than the alternatives .
In business , there is a distinction between creating value and capturing value .
market consists of all the existing and potential customers that share a particular customer need or set of related needs .
You can describe the size of a market by the total number of customers in the market or the total revenue generated by those customers .
current size or the potential future size
Within a given market , you can analyze the market share of each competing product —
companies trying to gain market share through product innovation .
Product – Market Fit Pyramid separates the market into its two distinct components : the target customers and their needs .
you probably don’t want to enter a market where customers are extremely happy with how the existing solutions meet their needs .
product is a specific offering intended to meet a set of customer needs .
The real – world manifestation of software products that customers see and use is the user experience ( UX ) , which is the top layer of the Product – Market Fit Pyramid . Beyond software , this is also true for any product with which the customer
interacts . The UX is what brings a product’s functionality to life for the user .
The functionality that a product provides consists of multiple features , each built to meet a customer need . Taken together , they form the product’s feature set ,
I describe the six steps of the Lean Product Process in detail in Part II of this book , with a chapter devoted to each one : Determine your target customers Identify underserved customer needs
Define your value proposition Specify your minimum viable product ( MVP ) feature set Create your MVP prototype Test your MVP with customers
Chapter 2: Problem Space versus Solution Space
high – level concept : separating problem space from solution space .
Any product that you actually build exists in solution space ,
problem space is where all the customer needs that you’d like your product to deliver live .
Fisher , invented the Space Pen in 1965 : a wonderful piece of technology that works great in zero gravity .
Faced with the same challenge , the Russian space agency equipped their astronauts with pencils .
first focus on “ what ” the product needed to accomplish for customers before getting into “ how ” the product would accomplish it .
The “ what ” describes the benefits that the product should give the customer
The “ how ” is the way in which the product delivers the “ what ” to the customer .
“ What ” is problem space and “ how ” is solution space .
The best way to mitigate the risk of an “ inside – out ” mindset is to ensure your team is talking with customers . That’s why Steve Blank urges product teams to “ get out of the building ” ( GOOB for short ) .
Lean product teams articulate the hypotheses they have made and solicit customer feedback on early design ideas to test those hypotheses .
likely true that customers won’t invent a breakthrough product for you ; but that doesn’t mean it’s a waste of time to understand their needs and preferences . On the contrary , a good understanding of customer needs and preferences helps product teams explore new potential solutions and estimate how valuable customers are likely to find each one to be .
You’ve got to start with the customer experience and work backwards to the technology . You can’t start with the technology and try to figure out where you’re going to try to sell it . … As we have tried to come up with a strategy and a vision for Apple , it started with : What incredible benefits can we give to the customer ? … Not starting with : Let’s sit down with the engineers and figure out what awesome technology we have and then how we’re going to market that . And I think that’s the right path to take .
It’s true that customers aren’t going to lead you to the Promised Land of a breakthrough innovative product , but customer feedback is like a flashlight in the night : it keeps you from falling off a cliff as you try to find your way there .
Unlike customers and their needs , which you can target but can’t change , value proposition is the problem space layer over which you have the most control .
“ Customers don’t care about your solution . They care about their problems . ”
Part II: The Lean Product Process
Chapter 3: Determine Your Target Customer
You can develop hypotheses about your target market , but you won’t truly know who your customers actually are until you throw your hook into the water and see what kind of fish bite . Once you have a product or a prototype to show customers , then you can gain clarity about the target market you’re attracting .
You define your target customer by capturing all of the relevant customer attributes that identify someone as being in your target market . These attributes can be demographic , psychographic , behavioral ,
needs .
Demographics are quantifiable statistics of a group of people , such as age , gender , marital status , income , and education level .
Psychographics are statistics that classify a group of people according to psychological variables such as attitudes , opinions , values , and interests .
You can also use relevant behavioral attributes to describe your target customer : whether or not someone takes a particular action or how frequently they do .
Another powerful market segmentation technique is needs – based segmentation . With this approach , you divide the market into customer segments that each have distinct needs .
especially for business – to – business products , the customer who will use your product ( the
user ) is not the same person who makes the purchase decision ( the buyer ) .
In such cases , it is useful to distinguish the economic buyer — the decision – maker who controls the budget and writes the check — from the other stakeholders involved in the decision – making process .
Crossing the Chasm , Geoffrey
Moore’s classic book
five distinct customer segments based on their risk aversion towards adopting new technologies .
Innovators are technology enthusiasts who pride themselves on being familiar with the latest and greatest innovation .
Early Adopters are visionaries who want to exploit new innovations to gain an advantage over the status quo .
The Early Majority are pragmatists that have no interest in technology for its own sake .
The Late Majority are risk – averse conservatives who are doubtful that innovations will deliver value and only adopt them when pressured to do so ,
Laggards are skeptics who are very wary of innovation .
Alan Cooper championed the use of personas as part of his “ Goal – Directed Design ” process . In his book The Inmates are Running the Asylum , he describes personas as “ a precise definition of our user and what he wishes to accomplish . ” Cooper explains , “ personas are not real people ” but rather “ hypothetical archetypes of actual users . ”
personas help people on the product team make decisions about which features are important and about how to design the user experience .
Name Representative photograph Quote that conveys what they most care about Job title Demographics Needs / goals Relevant motivations and attitudes Related tasks and behaviors Frustrations / pain points with current solution
Level of expertise / knowledge ( in the relevant domain , e.g . , level of computer savvy ) Product usage context / environment ( e.g . , laptop in a loud , busy office or tablet on the couch at home ) Technology adoption life cycle segment ( for your product category ) Any other salient attributes
You can see other examples of Tetzlaff’s work at http : / / beccatetzlaff.com .
Before you have any designs or product on which to solicit feedback , you will mainly talk with prospective customers to gain a deeper understanding of their needs , usage of current solutions , and pain points so you can identify potential product opportunities .
“ customer discovery ” interviews .
Your goal is to iterate until you feel confident that you have identified a target customer with an underserved customer need that you believe you can address .
To be useful , a persona should be pragmatic and provide useful information that can help inform product design decisions .
they are no substitute for talking to customers on an ongoing basis .
Chapter 4: Identify Underserved Customer Needs
word “ needs ” to refer to what customers want or value .
Good interviewers excel at listening closely to what customers say , repeating statements back to ensure understanding , and asking additional probing questions to illuminate the problem space .
A well – written user story follows the format “ As a [ type of user ] , I want to [ do something ] , so that I can [ desired benefit ] . ” For example : “ As a Dropcam user worried about the security of my business , I want to quickly view only the suspicious activity that took place without having to watch the whole video , so that I can know what’s going on in my store without spending too much time watching security videos . ”
Good user stories reflect customer needs .
One of the easiest ways to tell that a product team is starting with the solution space is that instead of articulating customer benefits , they list product features .
each benefit begins with a verb : help , check , reduce , maximize . A benefit conveys value , which means it’s doing something for the customer . Finally , many of the benefits speak to increasing something that’s desired ( tax deductions ) or decreasing something that is not desired ( audit risk , time required to accomplish a task ) .
Once you have an initial set of hypothetical customer benefits you feel good about , it’s time to test
them with users . The best way to do so is via one – on – one , in – person customer discovery interviews .
You should share each of your customer benefit hypotheses with the customer during the interviews . You should ask a set of questions about each benefit statement , such as : What does this statement mean to you ? ( to check their understanding ) How might this help you ? If a product delivered this benefit , how valuable would that be to you ?
For a response of high or very high value : Why would this be of value to you ? For a response of low or no value : Why wouldn’t this be of value to you ?
you can keep asking them , “ Why is that important to you ? ” until it doesn’t lead to any new answers .
Originally developed by Toyota , the Five Whys is an iterative question – asking technique to explore the root cause of a problem
The left side of Figure 4.2 shows the five – level hierarchy from the customer’s perspective . To the right of each tier is what it means to us at the company .
you need a good way to prioritize among the different needs — and prioritizing based on customer value is a good approach .
Let’s divide the graph into four quadrants . You can use this framework to evaluate potential product opportunities , either for new products or for additions or improvements to an existing product .
Incremental innovation occurs when you make minor improvements that add small amounts of customer value with each new version of your product .
When a new product enables such a better way of doing something that people can’t imagine going back to the old way , that’s disruptive innovation .
I receive a lot of questions about is how to measure values for importance and satisfaction . The easiest way to think about this is a question that you ask your customers ( or prospective customers ) .
A bipolar scale goes from negative to positive , whereas a unipolar scale goes from 0 to 100 percent of an attribute . It’s usually best to measure satisfaction using a bipolar scale ; since people can be satisfied or dissatisfied , a negative score makes sense . In contrast , importance is just a matter of degree — without any negative value — and therefore better measured with a unipolar scale .
For any bipolar scale , I recommend using an odd number of choices so that there is a neutral option in the middle .
it is generally agreed that 5 – point scales are best for unipolar and 7 – point scales are best for bipolar
Gap analysis and jobs to be done both use importance and satisfaction to quantify the size of different product opportunities to inform your prioritization .
The version to which I’m referring is based on calculating the “ gap ” between importance and satisfaction . So you simply take the rating for importance and subtract from it the rating for satisfaction .
Anthony Ulwick’s book What Customers Want .
His calculation subtracts satisfaction from importance , as in gap analysis . However , he does not allow that difference to become negative ; it can only go as low as zero . To that difference he adds importance so that it becomes a tiebreaker for gaps with the same size .
Using 0 to 10 for each rating , the resulting score can vary from 0 ( when importance is zero ) to 20 ( when importance is 10 and satisfaction is 0 ) . Ulwick considers opportunities with scores greater than 15 to be very attractive , and those below 10 to be unattractive .
Central to Ulwick’s methodology is the idea that customers buy products and services to help them get a task or job done .
Customers decide which product to buy based on how well it delivers their “ desired outcomes ” for the “ job to be done . ”
actually essential to successful innovation — for product teams to create a detailed and precise definition of their problem space .
Steve Jobs shared a similar view , saying : When you first start off trying to solve a problem , the first solutions you come up with are very complex , and most people stop there . But if you keep going , and live with the problem and peel more layers of the onion off , you can oftentimes arrive at some very elegant and simple solutions . Most people just don’t put in the time or energy to get there .
When a product or feature is plotted as a point on the graph , the amount of customer value it provides is the area of the rectangle the point creates with the origin . So you can calculate the customer value delivered with the equation :
The larger the area of the rectangle , the more customer value the product or feature creates .
The opportunity for each point is simply the maximum amount of customer value that can be added to it . Customer value can be added by increasing satisfaction , up to the maximum value of 100 percent ( or 1 ) . This can be expressed quantitatively as :
Opportunity to add value = Importance * ( 1 – satisfaction)
When you are evaluating opportunities to pursue , you should pursue the ones with the highest opportunity scores .
The customer value that a product delivers varies with the satisfaction level ( the width of the rectangle ) , but the maximum customer value it can deliver ( the area of the widest rectangle ) is fundamentally determined by the importance of the need ( the height ) .
another way to express the opportunity is : Opportunity = Importance – Current value delivered
If an improvement increases satisfaction from the current level ( Satbefore ) to a higher level ( Satafter ) , then the customer value it creates is the area of the incremental rectangle , given by : Customer value created = importance * (Sat_After – Sat_before)
the Kano model also plots a set of two parameters on horizontal and vertical axes : ( 1 ) how fully a given customer need is met ( horizontal axis ) , and ( 2 ) the resulting level of customer satisfaction ( vertical axis ) .
The utility of the model is that it breaks customer needs into three relevant categories that you can use : performance needs , must – have needs , and delighters .
important aspect of the Kano model : Needs migrate over time . Yesterday’s delighters become today’s performance features and tomorrow’s must – haves .
You have to meet basic needs before you can get credit for performance features . And your product must be competitive on performance features before delighters matter .
You can apply the Kano model to gain clarity about the problem space . Think about the customer benefits that are relevant in your product category and classify them into the three categories of must – haves , performance benefits , and delighters .
Once you have identified the customer benefits that you could potentially address , you use the importance versus satisfaction framework to determine which ones allow you to create the most customer value . You want to pursue product opportunities in the upper left quadrant of importance versus satisfaction that have as large an opportunity score as possible . For the opportunity you decide to pursue , you will next break down the related benefits and decide which ones you will address with your product . You want to make sure your product delivers enough customer value and is better than other alternatives ,
Chapter 5: Define Your Value Proposition (Step 3)
next step in the Lean Product Process is to define your product value proposition ,
you want to start off by identifying the minimum viable product . Remember , all of your hypotheses about customer needs are hinged on an underlying assumption about your target customer . If you test your MVP and realize that your assumption was wrong , you will have to revisit your hypotheses about the relevant needs to address .
Here’s what Steve Jobs had to say about saying “ no ” : People think focus means saying yes to the thing you’ve got to focus on . But that’s not what it means at all . It means saying no to the hundred other good ideas that there are . You have to pick carefully . I’m actually as proud of the things we haven’t done as the things I have done . Innovation is saying no to 1,000 things .
It’s important to list the must – haves , since they are required . However , since all products in the category have to have them , they are not the core part of your value proposition . The core elements are the performance benefits on which you choose to compete and the unique delighters you plan to provide .
blank template for your value proposition . In the first column , you list the benefits — one per row , grouped by type . You want to include the must – haves , performance benefits , and delighters that are relevant to you and your competitors . You should have a column for each relevant competitor and a column for your product .
Completing this grid allows you to clearly articulate what benefits you plan to provide and how you’re aiming to be better than your competitors .
you want to ensure that you are projecting forward in time , anticipating the important trends in your market and what competitors are likely to do .
Wayne Gretzky said , “ I skate to where the puck is going to be , not where it’s been . ”
you can use separate columns for “ now ” and “ later ”
Chapter 6: Specify Your Minimum Viable Product (MVP) Feature Set (Step 4)
You are not going to start off by designing a new product that delivers on your full value proposition , since that would take too long and be too risky . For your MVP , you want to identify the minimum functionality required to validate that you are heading in the right direction .
You have done all this great
thinking in the problem space and are now transitioning to solution space .
practicing divergent thinking , which means trying to generate as many ideas as possible without any judgment or evaluation . There will be plenty of time later for convergent thinking , where you evaluate the ideas and decide which ones you think are the most promising .
user story is a brief description of the benefit that the particular functionality should provide , including whom the benefit is for ( the target customer ) , and why the customer wants the benefit . Well – written user stories usually follow the template : As a [ type of user ] , I want to [ do something ] , so that I can [ desired benefit ] . Here’s an example of a user story that follows this template : As a professional photographer , I want to easily upload pictures from my camera to my website , so that I can quickly show my clients their pictures .
INVEST : Independent : A good story should be independent of other stories . Stories shouldn’t overlap in concept and should be implementable in any order . Negotiable : A good story isn’t an explicit contract for features . The details for how a story’s benefit will be delivered should be open to discussion . Valuable : A good story needs to be valuable to the customer . Estimable : A good story is one whose scope can be reasonably estimated . Small : Good stories tend to be small in scope . Larger stories will have greater uncertainty , so you should break them down . Testable : A good story provides enough information to make it clear how to test that the story is “ done ” ( called acceptance criteria ) .
identify ways to break each of them down into smaller pieces of functionality — what I call “ chunking . ” The goal is to find ways to reduce scope and build only the most valuable pieces of each feature .
Working in smaller batch sizes increases velocity because they enable faster feedback , which reduces risk and waste .
developers estimate the amount of effort required . They often do so by using story points , a type of currency for estimating the relative size of different user stories . For example , a very small user story may take 1 point , while a medium scope user story may take 3 points , and a large scope user story may take 8 points .
finished chunking your feature ideas , you should perform a second – pass prioritization that accounts for both the value and the effort .
case for ROI in the context of product development . When
you are building a product or feature , the investment is usually the time that your development resources spend working on it , which you generally measure in units such as developer – weeks
context of developing a new product , “ return ” is often not a dollar amount . Instead , it is usually some relative measure of the amount of customer value you expect a certain feature to create .
When you have two feature ideas with the same ROI , it’s best to prioritize the smaller scope idea higher because it takes less time to implement .
The main point of these calculations is less about figuring out actual ROI values and more about how they compare to each other .
ROI calculation can be a measure of value to your business instead of value to the customer .
If you are struggling with creating numerical estimates of customer value or development effort , you can score each feature idea high , medium , or low
Once you have organized your list of feature chunks by benefit and prioritized them , it’s time to start making some tough decisions . You must decide on the minimum set of functionality that will resonate with your target customers . You are going to look down the leftmost column of feature chunks and determine which ones you think need to be in your MVP candidate .
You can create a preliminary product roadmap by continuing this process and creating columns for each future version with each column containing the feature chunks that you plan to add .
At this point in the Lean Product Process , you have done a fair bit of work . You have : Formed hypotheses about your target customers Formed hypotheses about their underserved needs Articulated the value proposition you plan to pursue so that your product is better and different Identified the top feature ideas you believe will address those needs and broken them down into smaller chunks Prioritized those feature chunks based on ROI Selected a set of those feature chunks for your MVP candidate , which you hypothesize customers will find valuable
Chapter 7: Create Your MVP Prototype (Step 5)
The diagram in Figure 7.1 illustrates the difference between this incorrect way of interpreting MVP and the correct interpretation .
The first way you can categorize MVP tests is by whether they are aimed at testing your product or your marketing .
The second dimension on which MVP tests differ is whether they are qualitative or quantitative .
Qualitative means that you are talking with customers directly ,
Quantitative research involves conducting the test at scale with a large number of customers .
Quantitative tests are good for learning “ what ” and “ how many ” : what actions customers took and how many customers took an action
qualitative tests are good for learning “ why ” : the reasons behind different customers ‘ decisions
created the two – by – two matrix in Figure 7.2 to list and categorize the different MVP tests based on product versus marketing and qualitative versus quantitative .
These types of tests involve showing customers your marketing materials and soliciting their
feedback .
One good way to test your overall messaging is the five – second test . The idea is to show customers your home page or landing page for just five seconds and then ask them to tell you what they remember and what they liked .
called a smoke test because there is no real product for customers to use yet . Instead , there is usually a “ coming soon ” page that thanks the customer for their interest and asks for their email address or other contact information .
conversion rate :
handy tools like Optimizely and Unbounce make landing page testing and optimization faster and easier with less development effort .
how he decided to start out with a landing page MVP test ( https : / / blog.bufferapp.com / idea – to – paying – customers – in – 7 – weeks – how – we – did – it ) .
The explainer video is really just a variant of the landing page test that relies on a video to explain the product .
to test a landing page , you need to drive traffic to it somehow — and one way to do so is with advertising campaigns .
you might run three ad campaigns , each with a different tagline : “ We match you with your perfect dream job , ” “ We have the most job listings anywhere , ” and “ We offer the fastest way to scan job listings . ” You would then compare the clickthrough rates of the three ad campaigns to see which performed best .
A / B testing , also called split testing , is a quantitative technique where you test two alternative designs simultaneously to compare how they perform on a key metric , such as conversion rate .
In a true A / B test , the different versions your are testing run in parallel at the same time , for example , 50 percent of traffic to version A and 50 percent of traffic to version B .
Popular A / B testing tools include Optimizely , Unbounce , KISSmetrics , Visual Website Optimizer , and Google Content Experiments ( part of Google Analytics ) .
Multivariate testing is similar to A / B testing , but instead of testing different versions of a page , you test variations of page elements .
Crowdfunding platforms like Kickstarter and Indiegogo can be a great way to test whether or not people are willing to pay for your product and to quantify demand .
Product tests help ensure that customers see value in your actual product .
There are two fundamentally distinct times when you can conduct qualitative product tests : before you’ve built your product and after you’ve built it .
You can test your product’s design with customers before you build your product .
After you build your product , you can test it with users
You want to reduce this risk by showing customers design deliverables earlier in the process
Sketching on a whiteboard or paper allows you to iterate your design quickly .
Wireframes are not “ pixel perfect ” ; rather , they show relative size and position .
You usually only make selected user interface controls clickable — the ones that are pertinent to the design and what you want to learn from the specific test . Such a user scenario is called the “ happy path ” — the one that you intend the user to follow through the user experience you have designed .
Mockups convey visual design details such as colors , fonts , and images . Some are intended to be “ pixel perfect ” while others may only represent the approximate size and position of design elements .
Interactive prototypes provide a level of interaction that goes beyond that of just clickable mockups . For example , an interactive prototype might include many types of functioning user interface controls , such as drop – down menus , hover effects , input forms , and audio or video players .
The Wizard of Oz MVP and concierge MVP allow you to actually test your live product or service ; but instead of the final version , you are using manual workarounds .
The idea behind a concierge MVP is to be very involved with a small number of early customers to really understand your target market ,
Concierge MVPs work best with services , especially those with processes that require a fair amount of interaction
The Wizard of Oz MVP is similar to the concierge MVP in that you perform certain steps manually in the short term . However , the difference is that it’s not obvious to the customer that you are performing these steps manually ;
Once you have a live product with a meaningful amount of usage , you can conduct quantitative product tests .
Unlike qualitative product tests where you are asking smaller numbers of customers for their opinions , quantitative product tests measure the customers who are actually using your product ( usually with large sample sizes ) .
The fake door or 404 page test is a good way to validate demand for a new feature that you are considering building . The idea is to include a link or button for the new feature and see what percentage of customers click on it .
Product analytics aren’t a test per se , but they can give you insights into how your customers are actually using your product . For example , you can see which features they use the most and where they spend most of their time .
Product A / B tests or split tests are used to compare the performance of two alternative user experiences ( A and B ) in your product .
Chapter 8: Apply the Principles of Great UX Design
At this point in the Lean Product Process , you are clear on the feature set you believe should be in your MVP . User experience ( UX ) — the top layer in the Product – Market Fit Pyramid — brings your product’s features and benefits to life for the customer .
Poor UX gets in the way , preventing the user from realizing the benefits . Great UX makes it easy for the user to realize the benefits that the product’s functionality offers .
great UX also achieves a high degree of usability and delight .
usability , which indicates how easy it is for customers to use the product .
usability testing , where you ask users to complete key tasks and observe what they do .
usability also includes efficiency .
The likelihood of a user successfully completing a task is directly related to the amount of effort it takes .
“ Olsen’s Law of Usability ” : The more user effort required to take an action , the lower the percentage of users who will take that action . The less user effort required , the higher the percentage of users who will take that action .
you can ask users , “ How easy or difficult is the product to use ? ” and allow ratings on a seven – point bipolar scale :
You determine a product’s usability with respect to a particular user profile . Your target customers will have a certain level of knowledge or skill . Different target customers can vary in how tech savvy they are and how much relevant domain knowledge they have .
Ease of learning is also an important usability attribute . How much time and effort does it take a user to progress from having no knowledge of how your product works , to working knowledge , to mastery ?
The second key attribute of a great UX is delight . Strong usability helps avoid a poor UX , but it is not enough to deliver a great UX . Usability answers the question , “ Can customers use your product ? ” Delight answers the question , “ Do customers enjoy using your product ? ”
means evoking positive emotions . Products that delight users are enjoyable and fun to use .
aesthetics
Less is often more in UX design
User experiences that seem to read the user’s mind can help create delight .
product can make users feel like it understands them and is empathetic .
Animations and sound effects can also contribute to delight .
The conceptual design , the iceberg’s bottom layer , is the underlying concept that forms the essence of the user experience . The next layer is information architecture , which determines how you structure your product’s information and functionality . The next layer is interaction design , which defines how the user and your product interact with one another . The top layer that sticks above the water — the portion of the iceberg that users see — is visual design : how your product looks .
rather the essence of how they function to create value for the user .
conceptual design of using the checkbook as a metaphor , which customers found very intuitive .
You gain the understanding of your customer through user research , which is a specialized field within UX design . User researchers utilize a range of techniques to learn about customers , such as discovery interviews , usability tests , and surveys .
It’s critical when you conduct user research to ensure that the UX researcher isn’t the only person who gains most of the learning . Product team members should observe as much user research as they can .
Team debriefs , where individuals share and discuss their observations , help maximize learning and should be held promptly after the research occurs . Documenting the summary of results and key takeaways is also important to solidify the learning
The first is how tech savvy the user is . Users who are less comfortable with technology will need very simple interfaces
Another aspect of personas that can help inform good UX design is the context in which customers will be using the product .
If the user is rushed for time , then you need to make key information and frequently used functionality readily available without much effort .
IA is the design discipline responsible for defining how the information and functionality of a software product should be structured .
Good IA organizes a product in a manner that users find intuitive , with labels that are easy to understand , resulting in good usability and findability .
Findability refers to how easy it is for users to find what they’re looking for in the product .
The main IA deliverable used to do that is the sitemap .
A sitemap shows all of the pages or screens , how they are organized into sections , and the high – level navigation patterns provided .
Once you have created a sitemap that defines your product pages and how they are organized , the next step is to identify how the user experience flows across those pages — that is , how the user will interact with the product .
Interaction design specifies user flows : That is , what actions can the user take at each step , and how will the product respond ? It also governs how users enter information , such as in a form .
Another important part of interaction design is the feedback the product gives the user : how the system responds when the user takes a certain action .
Response time is another aspect of product feedback .
Users need confirmation that the system is receiving their actions .
If the product is going to take a while to complete the requested task , it is important to give users a sense of progress and how much time remains .
For long user flows — such as a multipage wizard — giving the user a high – level sense of the steps involved and a progress indicator showing “ you are here ” as he or she completes the steps is helpful .
They specify the possible flows for key tasks in your user experience . They show the actions that can be taken and the decisions that can be made by both the user and your product .
Aside from their role in UX design , flowcharts can help ensure that everyone on the product team understands the end – to – end user experience the product should deliver .
clickable wireframes are often more effective at communicating the desired interaction design .
You are trying to determine the layout of each page or screen : which components should be there and how they should be arranged . You are asking questions like , “ Should this page have one or two columns ? ” and , “ For this page , should I put the image on the left and the form on the right or vice versa ? ”
Instead of designing the wireframe for each page or screen from scratch , good designers look across a product and identify groups of pages or screens that should be similar . Each group will share a distinct template that defines its layout .
It is also called graphic design , look and feel , or chrome . And as with the chrome on a car , it doesn’t determine what the product does or how you use it , but it does impact how it looks .
Warm colors such as red , orange , and yellow are typically more energizing and passionate , whereas the more subdued cool colors such as green , blue , and purple are more calming and reserved .
blue color scheme because it conveys trustworthiness and calm .
Green is associated with nature , growth , and money .
Purple suggests luxury and creativity .
Red is associated with aggression , passion , power , and danger .
Orange is energetic and vibrant .
Yellow conveys happiness and sunshine .
Brown is associated with warmth and the earth .
Black can suggest sophistication , elegance , and mystery .
White is associated with purity , cleanliness , and simplicity .
The idea is to pick a combination of colors that look good together and to use only those colors in your design . Using color with consistency helps create a cohesive visual design .
Typography — defining the arrangement and appearance of text — is another important element of visual design .
Different typefaces convey different attributes : formal versus informal , classic versus modern , light versus dramatic . You should select fonts that reinforce the tone you want to set with your product .
The use of hero images is common — where a large , prominent photo shows your product , a typical customer , or some other artistic or inspirational
If a standardized symbol already exists for an icon in your design , I strongly recommend you use
Consistency in your product’s visual design is important to create a good UX .
style guide is a visual design deliverable that is used to achieve a consistent look and feel .
The idea is to align all page or screen elements to the grid as you lay them out .
higher – fidelity design deliverables that capture your visual design . They build on your wireframes , using color , typography and graphics to create the look and feel of your product .
The word gestalt means “ an organized whole that is perceived as more than the sum of its parts . ”
According to the Gestalt principle of proximity , the brain perceives objects that are closer together as more related than objects that are farther apart .
Therefore , you should put related objects close to one another in your designs .
The Gestalt principle of similarity maintains that the brain perceives objects that share similar characteristics as more related than objects that don’t share those characteristics .
Therefore , in your designs , objects that are similar or related should look similar by having the same shape , size , or color .
The size and color of elements are two of the main attributes that create a visual hierarchy . The brain assumes that larger objects are more important and smaller objects are less important . It also assumes that elements with high contrast — for example , a color that makes them stand out or “ pop ” — are more important .
Unity : Does the page or screen feel like a unified whole or a bunch of disparate elements ?
Contrast : Is there enough variation in color , size , arrangement , and so forth to create visual interest ?
Balance : Have you equally distributed the visual weight ( position , size , color , etc . ) of elements in your design ?
Use of space : How cluttered or sparse does your design feel ? Ensuring your design has enough white space — the space you don’t use on the page or screen — is important to avoid designs that feel crowded to the user .
With responsive design , as the screen width shrinks from wide to narrow , some page elements start “ wrapping ” — that is , getting pushed to the next line .
This is the text that your customers see , whether it’s on your marketing pages or in your product .
Chapter 9: Test Your MVP with Customers (Step 6)
This chapter focuses on how to conduct qualitative user testing of your MVP .
you have “ product blindness ” : blind spots for the issues that a new user will readily encounter within minutes of using your product .
User testing is the antidote for product blindness .
Qualitative user tests require that you show customers your product or design deliverables — wireframes , mockups , or prototypes — to solicit their feedback .
recommend conducting user tests with one customer at a time for the best results .
the more observers you have , the more worried about being judged some customers can be .
I’ve found that testing in waves of five to eight customers at a time strikes a good balance .
You can conduct user testing research either in – person or remotely .
Moderated means that the researcher is present and conducting the test with the customer .
Unmoderated implies that no moderator is present ; instead , customers are provided with the artifact or product to test and guidance on what to do .
I would recommend in – person if possible .
One advantage of unmoderated testing over moderated testing is that there is no risk of the moderator influencing the results .
When you are farther down the road and feeling more confident about your MVP , unmoderated testing can be a useful tool to compliment moderated testing since it takes less time and is less expensive .
You can ensure a good fit by using a screener — a set of questions , like a survey , that you ask prospective participants . For example , if you were targeting younger males , you would ask questions about age and gender .
In addition to demographic attributes , behavioral attributes are typically very useful .
Psychographic attributes — users ‘ opinions and feelings — can also be useful for screening .
One approach is to try to recruit local participants by posting online to Craigslist , TaskRabbit , and similar websites .
A best practice is to include in your posting a link to an online survey hosted at SurveyMonkey , Google Forms , or another survey site with your screener questions .
The best way out of this trap is to just blindly schedule users on a routine basis .
For example , you might schedule three users to come in every Tuesday afternoon
The typical range is $ 75 to $ 125 per hour , but it depends on your target customer and how much their time is worth .
product managers would conduct “ follow me homes . ” They would wait in the store aisle where Quicken was being sold . When they saw a customer who was going to purchase Quicken , they would ask if they could follow the person home , where they would observe the customer install and use the product .
Have customers bring their laptop or device for the test if possible ,
seat the customer at the table with their laptop or mobile device and have the moderator and note – taker sit next to the customer , one on either side .
skip the recordings and focus on watching the live sessions .
helpful to prepare a test script that lists what you plan to show and ask the user .
The test script should identify exactly which design artifacts or parts of the product you plan to show the user , what tasks you plan to ask the user to attempt to accomplish , and what questions you plan to ask the user , all in the desired order .
User tests typically run about an hour plus or minus 15 minutes ,
I recommend spending the first 10 to 15 minutes of the session warming the user up and conducting discovery about his or her needs and current solution . Then I like to spend about 40 to 45 minutes getting feedback from the user on the product or design artifacts . I close with 5 to 10 minutes of wrap – up , where I answer any questions from the user and ask any closing questions that I have .
Discovery questions are great for exploring the problem space and your value proposition with customers . You can start by asking them about their current behavior and feelings about the key benefit you plan to provide .
After discovery , you transition to the product feedback portion of the user test .
The top way that moderators perturb the results is by asking leading questions , such as “ That form was easy to fill out , wasn’t it ? ”
The best moderators engage the user with the product with as little intervention as
possible .
“ echoing back ” is a powerful technique to ensure you understand the user and to probe deeper .
Good moderators often use the judo move of answering a question with a question .
Open questions give the customer plenty of latitude in answering . They usually begin with “ why , ” “ how , ” and “ what . ” In contrast , closed questions limit the customer’s possible responses ( e.g . , to yes or no ) .
get in the habit of saying your next question in your mind before you verbalize it . That way , if it is a closed question , you can change it to an open – ended question before you pose it to the customer .
Another error to avoid is embedding a preferred or possible answer in your question .
You should avoid suggesting an answer and just stop talking after you ask a question to keep it open – ended and give users latitude
You will have to start by showing the user the particular part of your product or artifact on which you’d like feedback ; but once you do this , you should recede into the background .
like to reply , “ Pretend I’m not here . Just do whatever you would do if a friend told you to check out this product and you were by yourself on your computer at home . ”
you should ask probing questions as necessary .
If users have difficulty understanding or using your product , it’s important not to help them , as painful as that may feel .
Your goal is to keep the test as real as possible ; you’re not going to be able to hold every customer’s hand after your product launches , so it’s important for the product to stand on its own .
wrap – up section starts after the feedback portion is over . This is a good time to ask users to reflect on everything they’ve seen and provide overall impressions and feedback . You may want to ask the user to provide some ratings .
I usually ask users to sign a form acknowledging receipt of payment . On that form , I will often include prompts for the users to write their email and phone number if they want . I also like to include two yes – no questions : “ Would you be willing to participate in future research ? ” and “ Would you like to be notified when this product is available ? ”
As the user goes through the test , you’re trying to uncover data that supports or refutes the hypotheses you have made to get to your MVP candidate : your target customers , their underserved needs , the differentiators in your value proposition , and so forth .
There are three distinct elements of your product that users will give you feedback on : functionality , UX , and messaging .
When you receive critical or positive feedback from customers , it can be very helpful to map it to those three high – level categories of functionality , UX , and messaging . Documenting feedback this way after a test allows you to develop a clearer picture of what is and isn’t working well .
Feedback on usability has to do with how easy it is for customers to understand and use your product , whereas feedback on product – market fit has to do with how valuable they find your product .
You need to explicitly assess product – market fit by asking how much they value your product .
you can get very valuable feedback on your hypotheses at multiple levels of the Product – Market Fit Pyramid : underserved needs , value proposition , MVP feature set , and UX .
assumption that you are talking with the right type of customer .
Chapter 10: Iterate and Pivot to Improve Product-Market Fit
what to do after you complete each round of testing .
you want to use what you have learned after you receive a round of feedback to modify your hypotheses and your MVP so that you can test them with customers again .
his book The Lean Startup
“ Build ” simply means having something that you can test with customers , which could be a live product or design artifacts , such as wireframes or mockups .
I use a modified version of the build – measure – learn loop that I call the hypothesize – design – test – learn loop —
It’s easier to make changes near the top of the pyramid , but changing hypotheses near the bottom can have significant ramifications for higher layers .
You should identify which items you plan to address before the next testing round .
There is no hard – and – fast rule to determine when you’ve validated your MVP “ enough . ”
If you find that you are not making progress as you try to iterate , I recommend you pause and take a step back . Brainstorm with your team about what all the possible problems could be . Map each problem back to the corresponding layer of the Product – Market Fit Pyramid in Figure 10.2
You want to start at the bottom of the pyramid and work your way up until you identify which of your hypotheses are incorrect .
When you change one of your main hypotheses , it’s called a pivot .
One of the hardest parts of the Lean Product Process can be deciding whether to persevere with the opportunity you are pursuing , pivot to a new opportunity , or stop altogether .
So how do you decide whether to persevere or pivot if you still have cash in the bank and time on the clock ? You should consider pivoting if you just don’t seem to be achieving gains in product – market fit after several rounds of trying to iterate . If , despite your best efforts , your target customers are only lukewarm on your MVP , you should consider a pivot .
Chapter 11: An End-to-End Lean Product Case Study
Here is the high – level outline of my user testing script showing the time allocation : Introductions and warm – up ( 5 minutes ) General discovery questions ( 15 minutes ) Direct marketing mail The data about you that companies have Comparing yourself to others financially Concept – specific questions ( 45 minutes total ) Discovery questions related to concept’s main theme ( 10 minutes ) Feedback on concept mockups ( 35 minutes ) Review : What did you most like / dislike about what you saw ? ( 5 minutes )
Brainstorm : What would make the product more useful / valuable ? ( 10 minutes ) Feedback on possible product names ( 10 minutes ) Thanks and goodbye
To minimize risk , make faster progress , and avoid waste , I strongly recommend getting feedback on design artifacts before you start coding .
Part III: Building and Optimizing Your Product
Chapter 12: Build Your Product Using Agile Development
At this point , you have validated your target customer , their underserved needs , your value proposition , your MVP feature set , and your UX . As a result , you should feel confident about the blueprint you’ve developed . Validating product – market fit with prototypes is incredibly valuable , but now it’s time to turn your blueprint into an actual working product that customers can use .
teams using Agile methodologies break the product down into smaller pieces that undergo shorter cycles of requirements definition , design , and coding .
I’ll quote epistemologist and former Secretary of Defense Donald Rumsfeld : There are known knowns . There are things we know we know . We also know there are known unknowns . That is to say , we know there are some things we do not know . But there are also unknown unknowns . The ones we don’t know we don’t know .
You can’t completely escape unknown unknowns , but by using smaller batch sizes , you can rein them in to be more manageable and ship product more predictably .
A key part of Agile is defining your product in a customer – centric way with user stories .
Agile also promotes strong cross – functional communication and collaboration , with business people and developers working together daily , ideally face – to – face .
A key aspect of Scrum is that the team works in time – boxed increments
User stories are written and placed on the product backlog by the Product Owner ,
second role is “ development team member . ”
Three other important team roles are UX designers , visual designers , and quality assurance ( QA ) testers .
Well – written user stories include acceptance criteria , which are used to confirm when a story is completed and working as intended .
The ideal size of a Scrum team is five to nine people .
The third role is Scrum Master , whose job is to help the team with the Scrum process and improve its productivity over time .
for a visual depiction of the flow of work , meetings , and deliverables in Scrum .
Teams will often discuss and estimate story points together ,
Some teams develop a reference set of user stories of different known sizes .
Planning Poker is a popular technique for generating quick but reliable estimates as a group .
Teams will often break each story down into the set of coding tasks required to implement it .
When sprint planning is complete , the team should be clear on the set of stories they plan to accomplish in the sprint .
good idea to ensure each story has been assigned to a specific developer
The Scrum guidelines direct each team to define what “ done ” means for them . For many teams , “ done ” means a product that could be shipped , called a “ shippable product ” or a “ potentially releasable product . ”
sprint review meeting ( also called a sprint demo meeting ) where they show what they have built .
teams hold retrospectives
A core principle of kanban is to visualize work . Each card is a user story or a development task that supports a user story . The cards are arranged on a kanban board , which consists of a set of columns , one for each different state of work .
You may have noticed that instead of having just a single state for “ testing , ” Figure 12.4 has two : one state for items being tested ( “ in testing ” ) and a second state for “ testing done ” items . “ Development ” similarly uses two states . This helps create a clearer picture of the status of the team’s work and helps make bottlenecks easier to identify .
the quantity of active work is managed by constraining the amount of “ work in progress ” or WIP . The team decides on the maximum number of cards each column can contain , which is called a WIP limit .
You can further organize your kanban board with swimlanes — horizontal lines that separate cards into rows . There are a variety of ways to categorize cards with this technique . You can use swimlanes to prioritize cards ( the higher the row , the higher the priority ) . You can give each epic or each user story its own row . Swimlanes can also show each person’s workflow more clearly , by having a row for each team member . You can also track multiple related projects on one board by putting each project in its own row .
But you can measure the team’s throughput , which is just the number of work items completed in a given timeframe , for example , 10 items per week .
cycle time — the amount of time on average from when work starts on
an item to when the item is delivered to the customer — and lead time , the amount of time on average from when a work item is created ( e.g . , requested by a customer ) to when it is delivered .
You can visualize the flow of work in a kanban system with a cumulative flow diagram
Trello is a popular visual board application used to manage software development .
Agile depends on strong cross – functional collaboration . There should be free and frequent communication among product managers , designers , developers , QA , and any other team members ,
In a nutshell : product managers write the user stories , then designers create artifacts , then developers code , and then testers test . But product development is a team sport .
You can tell the level of collaboration by how often team members refer to one another as “ we ” instead of “ they . ”
You should maintain an up – to – date , prioritized backlog . It is important to be clear about the next set of user stories you plan to implement
It’s important to provide your developers with the information they need to build the desired product . A set of well – written user stories with accompanying wireframes or mockups usually does a good job of that .
In order for Agile teams to achieve their highest velocity , developers need to be able to hit the ground running when they start on a new user story — which means that the team must finalize the user stories and design artifacts beforehand . Because you want to achieve a steady flow of work , designers need to be at least one or two sprints ahead of the current sprint .
Beyond that , you should strive to break stories down into the smallest size possible .
Coding standards help
code review , one developer examines another’s code — and can catch mistakes that the original developer missed .
pair programming — a technique where two developers work on creating the code together at the same time .
One benefit of having dedicated QA testers is that they are more likely to find unforeseen problems than a developer checking her own code
In automated testing , software is used to run tests on the product and compare the actual results with the predicted results .
A person ( usually the developer or the tester ) has to initially define each automated test case , but once specified , they can be run whenever desired .
validation testing , checks to see if the new or improved functionality works as expected
The second aspect of product testing is to ensure that none of the other existing functionality was inadvertently broken during the process of building the new
This is called regression testing .
test – driven development , a technique where developers write
automated tests before they write code .
use continuous integration to iterate their product development more quickly .
current , stable version of the code base , called the mainline or trunk .
each developer should perform unit testing of his or her code
merging the new code with the trunk and releasing it , all the changes are combined or “ integrated ” to build the new version of the whole product . Integration testing is performed at this point
Continuous integration uses an automated build process to create a new version of the product based on the latest code commits .
continuous deployment , where code that successfully passes all tests is automatically deployed .
DevOps , which focuses on building and operating rapidly changing , resilient systems at scale .
to work well , continuous deployment requires a robust analytics system .
Chapter 13: Measure Your Key Metrics
Because you don’t yet have a customer base before launch, you rely heavily on qualitative research with prospective customers for direct feedback on your product.
Rohrer’s framework .
Attitudinal information is what customers say about their attitudes and opinions .
In contrast, behavioral information has to do with what customers actually do.
Quantitative research can tell you how many customers are doing (or not doing) something. But it won’t tell you why the customers are doing it (or not doing it). On the flip side, qualitative research will help you get at the underlying reasons for why customers do what they do. But it won’t tell you how many people do what they do for each particular reason. In market research, it’s very common to start with qualitative research to understand the relevant questions to ask and the responses that customers give you (the “why”). Armed with this information, you then proceed to quantitative research to find out how many customers give each answer (the “how many”).
So if surveys are bad for certain kinds of questions, what are they good for? Well, they are good for simple questions about the respondent’s attitudes where they have the information required to answer.
Surveys can help you see how people feel about your product and brand.
Net Promoter Score , This metric is based on the results of a single question, “How likely are you to recommend [product X] to a friend or colleague?”
NPS is an attitudinal measure of customer satisfaction with your product, and is a proxy indicator of product-market fit.
you should not invest in trying to grow your business until after you have achieved product-market fit.
you ask the users of your product the question, “How would you feel if you could no longer use [product X]?” The four possible responses are: Very disappointed Somewhat disappointed Not disappointed (it isn’t really that useful) N/A—I no longer use [product X]
Ellis found empirically that products for which 40 percent or more of users reply “very disappointed” tend to have product-market fit.
Analytics and A/B Testing
To paraphrase Peter Drucker, you can’t manage what you don’t measure.
It’s worth mentioning that the full version of Rohrer’s framework also includes a third dimension for “context of use.” He distinguishes between the different contexts of product use for each research method: “natural use” (e.g., analytics), “scripted use” (e.g., usability tests), and “not using the product” (e.g., discovery interviews). I encourage you to explore his full framework, which categorizes 20 different UX research methods. You can find it on the Nielsen Norman Group website at http://nngroup.com/articles/which-ux-research-methods. You can see Rohrer’s other publications and blog posts at http://xdstrategy.com. Now that it’s clear where analytics and A/B testing fit in, let’s discuss some frameworks for using these powerful tools.
it’s very helpful to have a holistic analytics framework that encompasses your entire business.
- Acquisition: How many prospects (new visitors) are our marketing programs driving to our website?
- Conversion: What percentage of prospects that come to our website sign up as customers? Retention: What percentage of our customers remain active over time?
- Revenue: How much money do our customers generate?
This example is a great illustration of how quant and qual can work together. Quant was the smoking gun that told us we had a conversion problem and identified where people were bdropping off the most, but it couldn’t tell us why. Qual gave us the insights we needed to understand and address the issues. After we rolled out the improvements, quant showed us the impact of the changes we made.
At a high level, almost every company has these five common goals:
- It wants to make prospective customers aware of its product.
- It wants to convert those prospects into customers.
- It wants to retain as many of its customers as it can over time.
- It wants to generate revenue from its customers.
- It wants its customers to spread the word about the product to generate prospects.
Dave called his framework “Startup Metrics for Pirates” because if you make an acronym for his five metrics—acquisition, activation, retention, revenue, and referral—it spells “AARRR!” (with an exclamation point added for good measure). In his talk, Dave recommended tracking two or three key metrics for each of the five elements of his framework.
AARRR metrics framework
At any point in the life of your business, one of the five macro-metrics in the AARRR model will be more important than the others. I call this the “metric that matters most”—or the MTMM
Optimize Retention First
Until you know that customers find your product valuable, it doesn’t make sense to spend lots of resources trying to acquire customers.
Optimize Conversion before Acquisition
It usually makes the most sense to focus next on making sure the highest percentage of prospects who show up at your front door make it inside. Conversion, the macro-metric that tracks this, has now become the MTMM.
Optimizing Acquisition
Once you have optimized retention and conversion, it often makes sense to focus on acquisition—
Retention rate is the single best metric to measure your product-market fit.
Retention Rate
¨
percentage of your customers are actively using your product. To calculate it, divide the number of active customers by the total number of customers.
Retention Curves
This “initial drop-off rate” is one of the key distinguishing parameters of a retention curve.
If the curve becomes flat at a certain value, then that is the percentage of customers you eventually retain.
Those three distinct retention curve parameters I mentioned—initial drop-off rate, rate of descent, and terminal value—are direct measures of product-market fit. The stronger your product-market fit, the lower your initial drop-off rate, the lower your rate of descent, and the higher your terminal value. The weaker your product-market fit, the higher your initial drop-off rate, the higher your rate of descent, and the lower your terminal value. Terminal value is the most important of these three parameters, since it answers the question, “What percentage of customers who tried your product continue to use it in the long run?”
This is why it is critical when tracking your user counts over time to distinguish between new users and returning users, the latter being the metric used in the numerator when calculating the retention rate.
Since retention curves measure product-market fit, they give you a way to measure how much you’re improving your product-market fit over time.
Cohort Analysis
A group of users that share a common characteristic—such as the month that they signed up—is called a cohort. Cohort analysis—the analysis of metrics for different cohorts over time—is a powerful tool.
If you are improving your product-market fit over time, your cohort retention curves will be moving up, reaching higher terminal values for newer cohorts.
The Equation of Your Business
Profit = Revenue – Cost
Revenue = Users * Average revenue per user
This equation tells you that there are basically two ways to increase revenue: increase the number of users or increase the average revenue per user (ARPU). Perhaps you’ve heard the term ARPU before; it’s a key metric tracked by many businesses.
Customer lifetime value (LTV) is the profit that a customer generates for you without taking into account the cost to acquire the customer.
When your LTV is greater than your customer acquisition cost, then each new customer generates profit for your business.
You can also increase LTV by increasing your average customer lifetime, which you can do by decreasing your cancellation rate: the percentage of paying customers that stop paying you each time period. This is more broadly called churn rate
Chapter 14 : Use Analytics to Optimize Your Product and Business
The Lean Product Analytics Process
However, there are some opportunities where just a small amount of effort can cause a major improvement in a metric’s value. I call these “silver bullets,”
Once you’ve assessed each metric’s upside potential, you move on to the next step in the process: selecting the metric that offers the most promising opportunities for improvement. This is the “metric that matters most” (MTMM)
You want to brainstorm as many improvement ideas as you can for this top metric.
You then pick the highest ROI idea to pursue.
Next, you design and implement the top improvement idea. Ideally, you would use an A/B testing framework to roll out the improvement to a fraction of your users.
However, sometimes you are stuck at a local maximum but actually could improve the metric further by considering a completely different alternative or approach.
