A Short Guide to Metrics
Simple rules to follow when creating metrics
- Over time: You must see how metrics change over time. Ideally you can view them by day, week, and month. No pie charts!
- In groups: Metrics require balance. Think of the extremes. If you over-optimize for one metric, what problem will it create?
- Weighted appropriately: If different times of the week or geographic areas are more important, your metrics should reflect this.
- Robust: Chances are your data is not pristine. Be careful of outliers. Median or percentile can be better than the mean.
- Keep it simple: People should be able to understand the metric.
Nice to have
- Responsive: Prefer metrics that change quickly as a result of your actions.