30 March, 2024

The Moral Economy of Tech

https://idlewords.com/talks/sase_panel.htm

Techies will complain that trivial problems of life in the Bay Area are hard because they involve politics. But they should involve politics. Politics is the thing we do to keep ourselves from murdering each other. In a world where everyone uses computers and software, we need to exercise democratic control over that software.

Second, the surveillance economy is way too dangerous. Even if you trust everyone spying on you right now, the data they're collecting will eventually be stolen or bought by people who scare you. We have no ability to secure large data collections over time.

The goal should be not to make the apparatus of surveillance politically accountable (though that is a great goal), but to dismantle it. Just like we don't let countries build reactors that produce plutonium, no matter how sincere their promises not to misuse it, we should not allow people to create and indefinitely store databases of personal information. The risks are too high.

I think a workable compromise will be to allow all kinds of surveillance, but limit what anyone is allowed to store or sell.

More broadly, we have to stop treating computer technology as something unprecedented in human history. Not every year is Year Zero. This is not the first time an enthusiastic group of nerds has decided to treat the rest of the world as a science experiment. Earlier attempts to create a rationalist Utopia failed for interesting reasons, and since we bought those lessons at a great price, it would be a shame not to learn them.

There is also prior art in attempts at achieving immortality, limitless wealth, and Galactic domination. We even know what happens if you try to keep dossiers on an entire country.

If we're going to try all these things again, let's at least learn from our past, so we can fail in interesting new ways, instead of failing in the same exasperating ways as last time.