Reading major social networks data without fooling yourself

When you collect major social networks data, the file does not fail on the day you build it. It fails about six months later, quietly, with nothing in the file to say which rows went stale and which are still good.

That problem is solvable. The solution is not more research, but a record structure that knows how old it is.

This page is about how to build one: what a row should hold besides the answer, how to handle a service that renames or absorbs another, and why an empty cell is the most dangerous thing in the table.

What to take away

Every field has a half-life

Group your columns by how long an answer stays true. A record that mixes a field good for years with a field good for weeks, and dates them together, is wrong about half of itself most of the time.

Write the definitions down once, in a data dictionary kept beside the file. Otherwise the next person to open it will guess at what each column meant.

Date every field, not every row. That costs one extra column. It is the difference between a file you can trust in a year and one you have to rebuild.

Structural fields sit at the top of that table for a reason. A service moves from one broad form to another perhaps once in its life, and the sequence those forms run in is short enough to hold in your head. A structural entry is therefore cheap to keep current and expensive to get wrong.

Store the observation, not the value

The instinct is to write down the answer. The better habit is to write down the act of finding it out.

A value on its own is an assertion by an anonymous author. An observation is a claim you can re-run: it says who looked, at what, on what date, and what exactly they saw. When two people disagree later, a table of values produces an argument and a table of observations produces a check.

One fact becomes four columns, not "export includes captions." The columns are the claim, the page it came from, the date it was read, and whether it was read or tested.

That distinction matters more than it looks, because documented and observed behavior are different evidence. They disagree often enough that merging them into one column destroys information you paid for.

MAU means monthly active users, and DAU means daily active users, but each platform sets its own rules for what counts as “active.” Registered accounts count accounts, not necessarily people or people active during a given period, so keep these measures distinct.

Named sources are not interchangeable. Meta Content Library, TikTok Research API, and X API are platform-specific research or API routes subject to access rules; Pew Research Center surveys report people’s self-reported platform use, not account counts, while DataReportal compiles published and advertising-reach figures. Check current definitions, eligibility, and any costs before budgeting staff time or relying on a source as a complete, comparable count.

For company-reported figures, publicly traded platforms may disclose metrics in filings such as Form 10-K, available through the SEC’s EDGAR system. These filings are not a standardized register of every social network, and their metric definitions may differ.

Blank means at least four different things

An empty cell in a shared file is read by the next person as whichever of these suits their argument.

Use four distinct markers and never leave a cell truly empty. If your tooling cannot hold four, use three and put the fourth in a note, but do not collapse "we did not look" into "there is nothing there".

The hard part is deciding what counts as one thing

Every catalog of services eventually meets a service that stops being one service. The record breaks in five ways, and all five have to be planned for before they happen.

Renaming. The name changes and everything else stays. If the name is your key, every historical row is orphaned. Give each entry an internal identifier that never changes and treat the name as just another dated field.

Twitter’s change to X is a concrete example: keep the stable identifier and date the display-name change so older records remain connected to the same entry.

Merging. Two entries become one. You need a rule for what happens to the older histories: keep both and point them at the successor, or you will lose the ability to explain where a claim came from.

TikTok’s merger with musical.ly is a real example of two service histories needing a successor link; retaining both histories makes the transition traceable.

Splitting. One service becomes two, often with the same interface for a while. Deciding late means backdating, and backdating is where records get quietly falsified.

Absorption. A service continues to exist as a feature inside something larger. It is neither alive nor dead, and a status column with only two values cannot say so.

YouTube Gaming is an example of absorption: its standalone app was discontinued and its gaming features moved into YouTube. A catalog should record that transition rather than treating the service as simply active or shut down.

Dormancy. Nothing is announced, nothing is shut down, and nothing changes for two years. This is common and almost never recorded, because there is no event to record. A "last confirmed alive" date is the only thing that catches it.

Myspace is a useful dormant-status test case: mark its catalog entry dormant only if it fails the catalog’s stated activity check, and preserve the last-confirmed-alive date. An accessible page alone does not prove that a social network is active.

Corrections have to keep the old state

When a value changes, resist overwriting. The previous value, the date it was replaced, and the reason are all worth more than the tidiness you gain by deleting them.

Two reasons, both practical. Keeping the old value beside the new one is an old problem with an old name, the slowly changing dimension. First, a reader who quoted your old value needs to know what happened to it, and a silent overwrite makes your file look like it always said the new thing.

Second, the pattern of changes is itself the most useful signal in the dataset. A field that has been corrected four times in a year is a field to stop trusting, and you cannot see that if each correction erased its predecessor.

Keep it simple. One append-only change log with the entry, the field, the old value, the new value, the date, and one line of reason. That is enough, and anything more elaborate does not get maintained.

A structure that survives contact with reality

The descriptive fields worth recording for a large service are set out in the major social networks overview. What follows is the housekeeping wrapped around them, which is what actually decides whether the file is still usable next year.

If the file exists to support a choice between two services, not to describe all of them, keep the side-by-side questions in a separate sheet with its own columns.

Mixing a catalog with a comparison answers neither question well. A comparison needs fields chosen for one decision; a catalog needs fields chosen for every future decision.

That last one prevents the slow drift that ruins most catalogs. Without a written rule, each new entry is judged against the last few entries rather than against a standard, and after a hundred rows the file contains something nobody would have accepted on day one.

Common questions

How often should the file be rechecked?

By field type rather than on one schedule. Structural entries can go a year. Anything about policy or interface needs a look before you rely on it, which means the recheck is triggered by use, not by the calendar. Record the trigger next to the field so the next person knows why the date matters.

Is it worth recording things that will obviously change?

Only if you record them as observations with dates attached, and only if someone actually needs them. A fast-moving field in a reference file is a liability unless it is clearly stamped with when it was true. If nobody uses it, drop the column: an unmaintained field is worse than a missing one, because it looks maintained.

What if two sources disagree?

Keep both rows. Note what each one counted and when, and let the disagreement stand in the file. Picking a winner throws away the most informative thing you found, which is that the record is contested. Resolve it only when you can say why one source is measuring the thing you actually care about.

Should the same file hold measured values and derived ones?

No. Keep anything you calculated, ranked, or scored separately from anything you observed, with the formula stored beside it. Derived numbers get copied out of context and quoted back as measurements, and once that has happened twice, nobody can tell which columns were ever real.