Decentralized networks search: what catalogs should record

Search on a federated network does not work the way a newcomer expects, and the mismatch produces more complaints than any other property of the architecture. Someone types a name they know exists, gets nothing, and concludes the network is broken or the person has left. Neither is true. Their server simply has not been told.

Decentralized networks search varies by software and instance. This page explains how discovery works on ActivityPub-based platforms, what a catalog should record about search, and how to find something when the search box is not going to help.

What to take away

Why the search box returns so little

On an federated network, ActivityPub-based services such as Mastodon, Pleroma, Misskey, Lemmy, and Pixelfed usually do not share one network-wide search index. Each instance indexes its own members' posts and material it has received; details vary by software, instance configuration, and content type.

Examples of named instances include mastodon.social (Mastodon), lemmy.world (Lemmy), misskey.io (Misskey), and pixelfed.social (Pixelfed). They are places to examine, not evidence that every instance has the same search behavior.

The consequence is direct. A new server has almost nothing in it. An old server with many members has a lot, but still only the part of the network its members happened to be interested in. Two members on two servers searching the same term get different results and both results are correct.

Statistically this is a convenience sample of the network: not random, not complete, and shaped by exactly the people who are already there. Anyone reasoning about what is on a federated network from what their own server's search returns is reasoning from a biased sample and should say so.

Finding a person: resolution, not search

The reliable way to reach an account is to have its full address, which includes the server, and to ask your server to fetch it. Most software does this from the same box that runs a text search, which is why the two get confused. They are different operations.

The convention for that lookup is WebFinger, and it is why a federated address looks the way it does. On ActivityPub-based services, WebFinger helps discover an account's actor endpoint, while ActivityPub carries federated activities between servers.

Once resolution succeeds, your server fetches the account. From that moment the account exists in your server's index, and a text search will find it.

The practical rules that follow are short.

What a catalog should record about search

Search behavior is a field like any other, and it varies enough to be worth its own row in an instance entry. The three-level directory structure puts it at the instance level, because the same software configured differently gives different answers. The levels are protocol, software, and instance: for example, ActivityPub, Mastodon, and mastodon.social.

Record the software version alongside, because several of these change between releases. Where the behavior differs from what the software documents, record both, dated: that contradiction is the most useful thing in the row.

An illustrative entry for mastodon.social might read: software: Mastodon; software version: [observed value]; text-search scope: [observed scope]; account lookup: WebFinger; anonymous fetching: [observed setting]. Check behavior on that instance rather than inferring it from the software name.

Searching a federated network in practice

When the built-in search is not going to answer, four approaches do most of the work.

Start from a person you already reach. Look at who they follow and who follows them. This is slow, and it is the mechanism the network is actually built around: discovery through people rather than through an index.

Use a hand-made directory. Fedi.Directory is a curated directory of Fediverse accounts, while instances.social helps people find Mastodon instances. These are starting points, not complete indexes, so check whether a listed account or server is active before relying on it.

Search the open web instead. Where a server allows anonymous fetching, its public posts are ordinary web pages, so a general web search restricted to that hostname sometimes finds what the network's own search cannot. This only reaches servers that permit it, and permitting it is a choice a catalog should record.

Ask. Posting a question in public and asking to be pointed somewhere works far better here than on a large centralized service, because the audience is small enough that a specific request reaches specific people. That is a property of small networks in general rather than of federation, and it is one of the honest advantages.

What this means for a catalog's own search

A catalog of these networks has the opposite problem: it holds everything it lists, so its search can be complete over its own contents, and it should say so plainly. Readers arriving from a network where search is partial will assume the catalog's search is partial too.

State the scope in one line at the top of the search page: this searches every recorded entry and field.

Make the negative result explicit as the general notes on catalog search describe. Say what was searched and what was not found, because a reader from a partial index will assume the answer is missing, not absent.

The classification fields themselves are the most valuable thing to search here, since a reader looking for a server usually wants a property rather than a name. For this search-focused view, the three questions in the mechanism-based classification are: Which protocol and software does the instance use? What content can its text search return? Can an account be found by full address when name search fails? These questions make useful facets because their answers can be recorded per instance.

Common questions

Is partial search a bug that will be fixed?

No central index causes this, so no design that keeps the architecture removes it. Shared indexes servers opt into, author consent for wider text search, and better directories narrow the gap.

None produces the complete view a central service has, and a network that offered one would have rebuilt the central index it set out to avoid.

Why do some accounts refuse to be found by text search?

Because being findable by text is a different exposure from being findable by address. A member who is happy to be read by people who sought them out may not want everything they said surfaced by a keyword. Software that treats these as separate settings is respecting a distinction the member cares about.

My server shows a post; my friend's server shows the same post with fewer replies.

Normal. Replies arrive at each server independently, and a server holds only the ones delivered to it. Nobody has the full thread except, sometimes, the server the original post was made on. Treat any reply count you see as a local count.

Should a catalog record how many accounts a server holds?

No. The figure is the server's own, defined by the server, and it says nothing a reader can act on. Whether registration is open, whether anyone posted recently, and what the rules say are all more useful and all checkable, as the pillar notes on decentralized networks argue at more length.