What an awesome question Adam.

Just had a look at our cohort analysis. Happy to let you know that 16% of the December 2010 cohort (those 100 people) are still using Buffer today (27 months later). It's actually fairly representative of what our retention stabilizes to after 4-5 months for almost every cohort, though 100 is not an ideal sample size!

We've certainly worked to improve retention over time, but in addition I think with Buffer I finally had hit upon a problem that was a real pain point. So, there was good retention right from the start.

Joel, what are you using for your cohort analysis? An internally-built tool? A third-party service?

Hey Charlie! We've been back and forth on this in the lifetime of the startup, but we're now doing all metrics ourselves, so it is internally built. I built an early version of the cohort analysis (which didn't scale, but that didn't matter) just a few months in and it was massively valuable. It's not too hard to knock something together.

Thanks for the answer Joel! By the way - I'm curious why you guys had to built this internally - I always get the feeling people prefer using a 3rd party service like kissmetrics than building something.

Internal BI is the way to go. It gives better vision to those who make decisions. Congratulations! :)

Solid. Thanks!

Would be an interesting third party tool. Does anyone do this already? Something as simple as receiving a list of usernames over time.

MixPanel can do this and they have a free plan.

I custom-built my own cohort reports but as number of users grow in my own app, I'm planning to replace it with MixPanel. You don't really want to put stress on your own database servers which are meant for production, not for analysis.

You should not be doing analysis (OLAP) on your main transactional database (OLTP). Moving from transaction-processing system (TPS) to analysis is done through a process call ETL (Extract, Transform, Load.) You will want to transform your data into facts and dimensions -- this is going from a relational model to a multidemensional model (using a star schema.)

but that would be another system to maintain. I'm happy to outsource this to MixPanel or some other company. My needs are pretty generic. I appreciate your response though.

Yes, I 100% believe in focusing on your core competencies. If the OLAP is self-hosted or external, the key takeaway is to not do analysis against production (once your data set is larger than RAM ;) before that, who cares?)

Cohort.ly does a great job. They are in private beta so you can request access. It has been really beneficial to us.

http://cohort.ly/

I really dislike this trend of only having videos to share info about a product or service.

Not even a blurb on there.

have a look at https://github.com/Doist/bitmapist - some more info here http://amix.dk/blog/post/19718