Launching a rocket ship, or a new company is a bizarre experience. There is the rush of rapidly completing major stages: Delaware c-corp status, app store release, press release, subscription packages, customer #1, ad #1, investor #1, the milestones just keep whizzing by. Then there are the hard questions not answered quickly. Is this a product people want? How engaged are people with the app? There are two groups who want to know the answers: investors and you.

An investor happy with stickiness.

Answering these questions for investors is fairly straight forward. Investors like using a familiar measuring stick across companies: page views, daily active users (DAU), monthly active users (MAU), and/or revenue. If you remove the graphics, that’s what the earnings reports are for Facebook, LinkedIn, and Twitter. But the original questions remain: Is this a product people want, and if so, how much do they want it? To approximate the answers, investors calculate secondary metrics such as page views per member, stickiness (DAU/MAU), revenue per member, etc.  Most are straightforward economics metrics, but I recommend caution when using the stickiness metric. The best ‘stickiness’ is achieved by a company with 1 user who visits every day. If I asked, my mum would oblige.

Now for the tough question, with the toughest critic: yourself. When guiding your product or company, how should you measure user engagement? Choose wisely. Once you define a metric for user engagement, that metric will be owned by a product team who will maximize that metric in ways you never thought possible. If you choose the stickiness metric mentioned above, it will be Mother’s Day 365 days a year.

In general there are two lines of approach for engagement summary metrics: bottom up and top down. A bottom up approach entails measuring every activity a user could do with your product and counting interactions. If you have a basic text messaging app, users can send messages and read messages. A pretty reliable metric then is . Use correlation coefficients between sends and reads for long term user engagement to chose . This approach can rapidly get away from you as your app or site increases in complexity.

Know your options.

For the top-down approach we can tackle measuring user engagement by first solving another tough problem. What is the vision of the perfect user experience with your product? Ideally, this is a question asked in the design phase of the product, but if not, or if the vision has morphed, no worries. Take the time to ask it now. Once the vision is well articulated everything else is simple.

Example 1, short term vision: Let’s say our product is a news aggregator and the vision is to provide valuable content to members everyday. The top level engagement metrics are going to be along the lines of number of members reading news on 5 of the last 7 days, number of members who interacted with a news article today, etc.

Example 2, long term vision: Let’s say our product is a real estate site and the vision is for members to buy a house through our service. If we captured 10% of San Francisco’s homes sales, that would be 11 sales a week. That metric is too sparse to be reliable. For a stable metric, we need to utilize early indicator signals for eventual conversion. Enter data science. I can not predict what the actual metric will be, but it will be of the format , a weighted sum over the actions users can take.

Engagement metrics can seem elusive, but a vision is a good place to start.

For continued reading:

• An excellent workshop on understanding user engagement in lab settings: Lalmas, O’Brien, and Yom-Tov’s slides.
• Concrete values for 80 sites in terms of Popularity, Activity, and Loyalty: Lehmann et al
• Feel free to suggest additional sources in the comments.