Online Community Metrics

Cross-posted from the Online Community Report blog .

How are you measuring the performance and health of your community?

This may seem like a strange or even whimsical question. Communities are organic and ever-evolving entities, right? Trying to benchmark and measure something as inherently chaotic and dynamic as online community performance, health, or the shiny new metric “engagement” can be challenging. But the reality is that this type of data is now expected from community managers by stakeholders and executives.

So where is one to start? I have outlined a few first steps below, based on my experience. You mileage, of course, may vary.

1. Review your “Basic” Web Metrics
Your “basic” web metrics, from sources like Hitbox or even your server logs will reveal quite a bit of data including # of page views, # of unique visitors, browser type, screen resolution, country, language and referring URLs. While this data in isolation won’t paint the “big picture” for you, it will give you some information to make decisions about design (target screen size), what browsers to design to, and potentially what languages to localize your community site in. This information also becomes more meaningful when you look at trends in the data over time.

2. Registrations and Attrition
These numbers are a bit more tricky to get, but you should definitely pay attention to registrations numbers. The ratio of registrations to casual visitors is also a great number to look at if one of your primary goals is conversion from visitor to member. Sign up form abandonment could indicate a usability issue with your signup flow, or could indicate that the value trade off of registering vs. the percieved benefit of the community isn’t clearly being communicated.

3. Look at Member Activity
Depending on how you collect metrics and what you used to instrument your site, data on visitor and member activity may be difficult to get as well. Being able to drill down on member participation in your community helps you understand if there is a good balance of the following functions in your community: contributors (those that produce content), connectors (those that form relationships) and consumers (those that consume content). Member activity data also gives you a good view (depending on the quality of the data) of how engaged (or not engaged) your members are in various content and features.

4. Watch Your Moderation / Maintenance Overhead
Are you constantly editing comments or discussion group posts? Banning members? Removing spam from your forums? Spending all you time approving comments or reviewing video clips before allowing them to be posted? Is hosting an online community inherently painful?

Well, in short, no.

Basically, any activity that detracts from the intended mission and purpose of the community should be tracked, and the root cause should be identified and addressed. This could be as simple as making your Discussion Group guidelines more clear and telling your moderators to lighten up (I had to do this). If you find yourself struggling against parts or all of your community rather than engaging in activity with them, something is wrong. Track the “overhead” for 2-3 months, and then take a time out with your team (and ideally members from your community) to discuss ways to optimize or solve the problem. You’ll be glad you did.

5. Ask Your Members
This should arguably be at the top of the list. The best source of data about how your community is doing? Your community. Creating an open dialog about community health and activity can be as simple as turning on blog comments or creating a new discussion forum for the sole purpose of discussing open issues and ways to improve the community.

I’ve also seen satisfaction surveys used in a very effective way to benchmark community health and satisfaction, as well as to gather feedback on usability issues, moderation activity, and feature enhancements. Several companies I know (SAP, Intiut) use the “Net Promoter” score to quantify the willingness of a member to recommend the community. You can find more information about the Net Promoter concept here:

The One Number You Need to Grow – Reichheld

Other Resources for You
Last March, Jim conducted a metrics survey and put together a white paper based on the results. You can check it out here:

Online Community Metrics Survey 2006

Forum One Web Fun fact: a lot of the snarky comments were mine, made before I knew I would eventually be working for Jim.

If you weren’t included in the 2006 survey, and would like to participate in the 2007 survey, please email me.
Survey invitations will be going out the first two weeks of Februar