Rules
Part of How to make platform comparisons that hold up
Building a platform comparisons directory from substitution sets, not guesses
Platform comparisons directory built from substitution sets: which pairs are worth comparing, which should not exist, and how the index stays honest.
What to take away
- Two services belong in the same comparison only when a reader with a stated job could pick either one and get that job done.
- Substitution sets come from jobs, not category names, and each set needs an admission test with observable answers.
- Every index entry carries six fields. The fifth names what does not separate the pair.
- Retire a pair when either service stops being a plausible candidate for its job, and archive it with dates.
- Searches for pairs that fail the test get an explanation, not a table.
Comparability is a property of a pair
Most published comparisons pair things no reader would swap. A messaging service set against a broadcast network, or a hosted forum set against a video host. Those pages get written because people search the phrase, not because a reader could switch.
Comparability applies to a pair of services, never to one service alone. The operational version is a single question. Could a reader with a stated job choose either service and get the job met?
A network against a tool is the clearest failure. A tool has users; a network is its people. Platform alternatives reviewed works through what that difference costs a buyer.
Here is the worked case. A reader wants a private real time chat for a paying member group of 300. Discord serves that. YouTube does not, and nobody joins a video host to talk in a room. The pair fails and gets no entry.
Building substitution sets from stated jobs
A substitution set is a group of services that could each satisfy one stated job. Build it in five steps.
Building a Substitution Set
- Write the job in one sentence
- List every plausible service
- Apply an observable admission test
- Record rejections with failed test
- Let services appear in several sets
- Write the job as one sentence.Include a verb, a reader and an outcome. "Let 300 paying members talk in private and keep a searchable record" qualifies. "Community" does not.
- Read the candidates' own documentation.Read the pricing page and the export guide. Skip category pages, which group services by the industry's naming rather than by jobs.
- Apply an admission testwith observable answers, in the way the written inclusion rule for a list does, so the boundary is one a second person could reproduce.
- Name the set after the joband record the date you built it.
- Recheck on change.New terms of service, a new price page or a changed export policy triggers a fresh pass.
The result is a small taxonomy of jobs with membership lists underneath. That structure beats category names, because readers supply the jobs and the industry supplied the categories.
Example: a private member discussion set
Job: keep 300 paying members talking in private with a searchable record. Candidates taken from their own documentation: Discord, Slack, Discourse, Circle, Facebook Groups.
The admission test asks for answers a second person can check:
Private member discussion set
- Entry is private and by invitation.
- Posts persist and members can search the archive.
- A departing member can remove their own posts.
- The operator can export the full archive.
- The cost at 300 seats is published on the provider's pricing page.
Discord clears the first two checks on its own documentation. Export and removal terms move with each release, so read the current Discord notes and pricing first.
Once admitted, Discord against Slack passes the comparability test. Both can host the private room and the record. Which one wins depends on the axes.
The six fields in one index entry
The index lists pairs. Each entry stays short, because the detail belongs on the comparison page.
The six fields
- 1. Job
- The one sentence the pair serves
- 2. Set
- The substitution set both services sit in
- 3. Pair
- Both services, named exactly
- 4. Axes
- The fixed axis set used for every pair
- 5. Not comparable on
- The points that do not separate the two
- 6. Review
- Date built, date checked, recheck trigger
Index Entry Fields
- Substitution setthe shared job
- Pairtwo services by stable key
- Why this pairgenuine uncertainty
- Deciding axeswhat separates them
- Not comparable onidentical or irrelevant axes
- Last testeddate, tier, region
Field five stops padding. Naming what does not separate two services keeps a page from running to twenty rows of ties, which buries the three rows that matter.
Axes come from the six-axis method and stay identical across every pair. A different set per pair lets a writer choose the one that produces a preferred answer.
Pairings that fail the admission test
A network against a tool. Compare them and you compare a room with a person. File it under the network, or under the tool, but not as a pair.
A service against a category. "This service versus open source alternatives" puts one thing against an idea. Name the candidates, or write the category.
A live service against a discontinued one. History, not a decision. Send it to the archive with its dates.
A service against doing nothing. The pairing that should exist and rarely does. For many readers the honest choice is the incumbent against dropping the job. It never appears because doing nothing is not a product.
Keeping the index accurate as pairs multiply
Pairs grow faster than services. Cap the pairs per set at the ones where the choice is genuinely open, usually a handful.
Anything beyond that is filler nobody retests. Retire a pair when either end stops being a plausible candidate, and archive the entry with its dates.
An entry with two ends inherits the recheck triggers of both, which is why the directory of substitutions tracks both sides of each pair.
Common questions
Readers search for pairs that fail the test. Should the index serve them?
Serve the intent, not the query. Explain why the two are not alternatives, then point to the substitution set each belongs to. Do not build the table because the search exists.
Can a substitution set have one member?
Yes, and publish it. A job with one plausible candidate carries no competitive pressure, which shapes what a reader should expect on price and exports.
How specific should a job be?
Specific enough that two people list overlapping candidates, general enough that more than one service qualifies. A job that admits everything is a category.
Who decides the jobs?
Readers do, through what they ask for, filtered by whether the job can be stated in one sentence with observable membership.
