Unscalable, Hand-Crafted Lists of Links
Or, Technology vs. Taste
The first search engine wasn’t one at all. It was a webpage with a list of links on it. There were many of these, some freely accessible, and others within the first walled-gardens of America Online, Prodigy, and so on. But the most famous one is probably Yahoo.com.
Yahoo! started as a very simple idea and its title reflected that: Jerry and David’s Guide to the World Wide Web. Jerry and David made this webpage on their Stanford.edu student web space — akebono.stanford.edu/~yahoo. They found things on the web and typed each and every a href link into a text editor writing a file that ended in .html. This very unscalable technique earned their little company $33.8 million just two years later. Despite the insane tech IPOs we’ve become used to, that’s a lot of money by any decade’s standards.
By the time Google became firmly established, hindsight mocking of Yahoo! became standard. Of course it was absurd to portal the web! Of course humans could never hope to map its contents! Of course an algorithm is a better idea! Very few tempered their hindsight with much of any humility. Rarely did you hear an it turns out that or a we’re learning that. But we’re not as smart as we think we are.
^ That old Yahoo! sure was handsome.
Nevertheless, by the mid-aughts, curation was the thing. The list of late-Gen-X/early-millenial influencers that made their name on human scale curation is… not short! Did anyone mock them for doing something absurdly unscalable? No. Why? Because they never said they were trying to map the web. The whole point of curation is choice; it’s saying, “Of all the things, I have chosen these.” You cannot search for what you don’t know exists. That’s curation.
Post-algorithm curation was all about technology versus taste.
For me, this was a very good time for the web, and for people. Yes, some became famous for their taste. Some of them even became rich. I don’t have a problem with that. Because at the root of it was something I deeply value and hold dear as a distinction between human and machine: curiosity. You can write an algorithm to act as if it is curious, but you cannot create it to be so. At least not yet.
We are in this cycle again. Google’s search results page now prioritizes AI-driven results. Google offers this as a convenience: our machine has read web pages so that you don’t have to. Mark this week in your analytics — it will be a profound cliff for website traffic and attention; many will never leave Google.com again. Good for Google — they win at scale again. But, in the end, convenience is an antithesis to choice. (To say nothing of its relationship to quality — AIs are often wrong, after all.)
Times like these reinterpret the entire history of the web. They should change the ways we think of the halcyon days of unscalable-human effort. No longer naive, but necessary. No longer inadequate, but intentional. Yahoo! made many mistakes once they had a lot to lose. One wonders how things would have gone had they recognized what made them different at the start.
Even today, thirty years after Jerry and David, I still visit bookmarked webpages that maintain human-curated lists of links. I do this far more regularly and often than I search for web pages. Some of these portals haven’t updated their design in a decade or more. But what sometimes seemed like a stubborn affectation — on their part for maintaining them and on mine for visiting — now looks, if I may say so myself, like wisdom.
No one person sets the pace of the web. But one person should set the pace of their experience of it. Scale begins as a proposed benefit to individuals, but the farther it goes, the more its end-user returns diminish. That’s where we are. The scale of the algorithm exceeds even our own understanding; its returns benefit only its owners and leave the rest of us awash in noise and bereft of understanding.
Information technologies have always been finely-balanced metaphors. We’ve gone from maps to portals to oracles. Each one has depended more upon technology to scale and trust to satisfy. You can buy one, but not the other.