How to catalog community platforms data consistently

Before you count anything about a community platform, decide what one of the thing is. It sounds like a pedantic first step. It is also the step that decides whether the file is worth keeping, because the units in this category are genuinely ambiguous and every product defines them differently.

One community, one member, one message, one active participant: each has at least three defensible definitions. Two platforms that both report a figure for it are usually reporting different quantities. This page is about settling the units first, then about what is worth measuring once they are settled.

What to take away

Four units, and why each one is contested

One community. Is a space with twenty topic channels one community or twenty? Is a group that also runs a mailing list one or two? The answer that works is to define the unit by membership: one community is one set of people with one joining decision. Twenty channels behind one door is one community; a mailing list you have to subscribe to separately is a second one, overlapping.

One member. Registered, present, and participating are different definitions, and the gap depends on the product and the counting rule. The catalog should never store a bare member figure; it should store whichever of the three it means, in a column named for that meaning.

One message. Does an edit create a new one? Does a reaction count? Does a system notice about someone joining? Products differ, and the differences are invisible in a total. If you are counting messages, count only messages a person typed with the intention of being read, and say so.

One active participant. Active over what window, doing what. Reading is activity and is usually invisible. Posting is visible and rare. A definition that requires a post excludes most of the people the community exists for, which is fine as long as the column is named honestly.

Deciding these is a small exercise in ontology: saying what kinds of thing exist in your record and how they relate, once, so that everything after it is consistent. An hour spent here saves rebuilding the file later. A community platforms list can keep those unit decisions visible instead of burying them in a total.

What a platform's own dashboard will and will not tell you

An organizer running a community sees whatever the product shows, which is shaped by what the product is selling. Treat the dashboard as a source with an author.

For a catalog covering Discord, Discourse, Circle, Slack, Mighty Networks, Reddit, and Facebook Groups, use the same unit ledger for each product but retain each dashboard's own labels. Check its current help or analytics documentation for the definitions of active member and message when available; save the wording and collection date, or mark a definition as unavailable rather than infer it.

A single dashboard number is useful as a trend only over a period when its definition did not change. You know that only if you wrote the definition down. In a catalog, keep member count, active members, messages, channel activity, and retention curves in separate fields, with the product's definition attached to each when available.

The notes on how platform figures are produced offer a general reminder: treat each reported figure as a product-defined measure. For a catalog, record its label and definition as displayed at the time of collection, and preserve the date and sample so later readings can be compared honestly.

What is worth measuring instead

Five things, all of which an organizer can produce, none of which requires the platform to cooperate.

Each is defined in one sentence, measured the same way every time, and independent of what any product decides to report. That is what makes them worth more than the dashboard.

Storing it so next year's figure is comparable

One row per measurement, with the unit definition version alongside. When a definition changes, add the new version instead of restating old rows. Start a new series and record the changeover date.

Show both series on one chart with the break marked. Readers then see the method changed, not the community.

The snapshot discipline in the notes on annual catalog snapshots applies to these measurements exactly as it applies to catalog fields. The reason is the same: the value of a series is entirely in whether the two ends were produced the same way.

Store the sample too, not only the result. A hundred message identifiers take no space and they are what lets you recount under a new definition later. Without them a definition change ends the series permanently.

The bias you cannot measure your way out of

Every community you can study lasted long enough to be studied. Those that failed in their first month left no archive, no dashboard, and usually no trace.

Any pattern you extract from the communities in front of you is about the ones that made it. This is survivorship bias in its most ordinary form. Advice derived from studying successful communities tends to be confident and unreliable.

There is no correction available. What there is instead is a discipline: state that the sample is survivors, avoid causal claims, and be especially suspicious of any property that all the survivors share, because you have no idea how many failures shared it too.

The community platforms overview makes structural claims rather than empirical ones: it describes what a platform is, not how well it performs. Keep that distinction when classifying products in a catalog.

Common questions

Can I compare two communities on different platforms?

On the five measures above, yes, if you measure both yourself with the same definitions in the same period. On anything either platform reports, no. The second comparison is the one everybody wants and the one that is not available.

Is a falling member count bad?

It is usually meaningless, because member count is registrations and registrations rarely fall. Falling distinct posters is bad. Falling newcomer response rate is worse, because it precedes the other one by months.

How much of this should a small community bother with?

Distinct posters and newcomer response rate, counted by hand, once a month, in a spreadsheet with four columns. That is twenty minutes and it catches almost everything worth catching. The rest is for someone maintaining a catalog rather than a community.

What about counting lurkers?

You cannot, in any product that does not report view counts, and where one does the figure counts loads rather than people. Accept that most of a community is invisible, design for readers anyway, and resist any measure that treats the visible few as the whole.