Platform comparisons by year: A method for tracking change

Platform comparisons by year are most useful as a series. A single comparison is a photograph; repeat it and the movement between runs becomes the interesting content.

Most of the value in comparing platforms comes from that movement. It shows which platform is drifting, and in which direction. Drift is what costs people years. One example: a feed that switches from follower order to ranked order changes reach while the platform keeps its name.

Getting a series is harder than it sounds, and the difficulty is not the measuring. It is keeping the method still. A comparison re-run with a slightly different account, a slightly different definition, or a slightly different set of criteria has produced a second photograph rather than a second point on a line.

What to take away

Why a re-run is not automatically comparable

Between two runs, at least three things change, and only one of them is the thing you meant to measure.

The platforms change. This is the signal.

Your accounts change. They are older, they have more history, and the systems that decide distribution treat them differently as a result. An account that has been posting for a year is not the same instrument as a new one.

Your method changes. You learned something, so you measure a little differently, and you do not notice because the improvement felt obvious.

The second and third are confounding factors: they move at the same time as the thing you are studying, so any difference you find is a mixture of all three and cannot be separated afterwards. The only defense is to hold two of them still on purpose.

The frozen method

Write it down before the first run, and treat it the way a laboratory treats a protocol rather than the way a blog treats a checklist.

Give the whole thing a version number. Every result row carries the version. That single column is what makes the difference between a series and a pile.

The figures in this example are illustrative, not measured.

The v1.0 rows form one series. The v2.0 row starts a new one, because the criterion was reworded that year.

Platforms to cover. YouTube, TikTok, and Instagram are the default set. They cover the three distribution models most comparisons care about: search and subscription, ranked short video, and a social graph. Add a platform only when a criterion applies to it.

When the method has to change

It will. A criterion stops being meaningful, a platform removes the thing you were measuring, or you realize a definition was wrong.

Do not restate old results under the new method. You cannot: the old evidence came from the old method, and the new method's needed information was never recorded.

Instead, close the series with a dated note and start a new one. Where the change was small enough, run both methods side by side for one cycle so a reader can see the offset.

The same discipline applies to a catalog's fields in an annual snapshot.

Reading a series once you have one

Repeated network maps can be compared when each uses the same conventions. Platform comparisons need the same consistency from one run to the next.

For your own runs, state what each result represents and whether it is verified or the best information available.

Three questions, in order.

What a series cannot do

It cannot tell you a platform improved, only that it scored differently on a fixed set of criteria that you chose. Benchmarking always measures what the benchmark measures, and a platform that got better at something outside the criteria looks unchanged.

It cannot show growth, decline, or popularity, because none of those is in the method and none of them is measurable from outside anyway.

It cannot be transferred to another reader without its weights, its accounts, and its content. Publish the method with results or the results are decoration.

The same warning applies to any published comparison you read, which is why the constraint-based reading of a category is more durable than any dated scorecard. Constraints explain why a convention exists, and a score explains nothing.

A cadence that is worth the trouble

Run the annual pass in the second week of January, after holiday traffic settles and before the year's platform changes land. Add a second pass in the second week of July for categories where distribution behavior is the point, because that is where movement is fastest. An announced change to a criterion you measure is the other trigger.

Between runs, log events instead of re-measuring: a pricing change, a policy revision, an ownership change. These prompt an off-cycle run on the affected criteria only, done with the same method and recorded as an interim result with its own date.

The criteria that structure the comparison also show which events matter. An event that cannot affect a measured criterion cannot affect a result.

Common questions

Can I use my real accounts instead of fresh ones?

For measuring your own outcomes, yes, and that is a different exercise. For comparing platforms across years, no.

How many cycles before a series says anything?

Two, for a specific movement on a specific criterion with a date attached. Three, for a direction.

What if a platform changes so much that a criterion no longer applies?

Record it as not applicable with the date, and keep the criterion. A criterion that stops applying is itself the most interesting result the series can produce, and deleting the row hides it.

Is it worth publishing a year with no movement?

Yes. A stable year preserves the interval that dates the next change. A series with gaps in it cannot bracket anything.