Okay, against my better judgment, I’ll bite. But first, a story:
At the start of the Covid pandemic, my research institute shut down and I found myself stuck at home and bored out of my mind. For lack of anything better to do, I decided to try my hand at picking locks. I taught myself the basic principles with the help of YouTube, I ordered a few picking sets from Amazon, and after a few weeks’ practice I found I could open most basic padlocks inside of a few minutes. I was hit-or-miss on more advanced padlocks, and I never came close to mastering what I regarded as the summit of the discipline, the six-pin cylinder locks that secure most German residences. (I did manage to open my practice ABUS double cylinder with different pin configurations two or three times, but only after extremely frustrating multi-hour attacks, and I was never able to defeat any of the various security pins.)
I gave up on lockpicking after a few months, when I realised that picking locks is probably the least impressive parlour trick you can cultivate and that increasing one’s lockpicking proficiency requires refining the same extremely narrow and tiresomely acquired skillset over months and years as you work your way up the hierarchy of increasingly sophisticated lock designs. To my mind, and with all due apologies to lockpicking enthusiasts among my readers, there are many more rewarding pursuits out there for people who like small mechanical puzzles.
Of course, nothing is wasted, and you learn something even from abandoned pursuits. In this case, I gained first-hand experience with the fact that all security measures are only ever just as good as they have to be. Many locks objectively speaking are not very secure, and a surprising number are plagued by egregious design flaws that you need only a little knowledge and practice to exploit. Pin-by-pin picking, it turns out, is for hobbyists and tinkerers; in the real world locks are mostly shimmed, bumped, combed, rapped, raked, drilled, bypassed, snapped and cut. Yet locks keep most people out most of the time anyway, because many of these exploits are brand- and model-specific (requiring generalist attackers to amass a great deal of specific knowledge), because opening locks you are not authorised to open is highly illegal, and because our world is awash in locks, most of which secure things like rusty bicycles and boxes of Christmas ornaments that nobody much cares to access. The risk that any particular lock may be attacked is therefore low and most locks can get away with their flaws.
Put another way, our existing system of lock security exists in delicate balance with the low density of human aptitude and enthusiasm for achieving unauthorised access to the things that these locks secure. Any technique upsetting this balance would prove very awkward and potentially very expensive for a great many people. A novel attack requiring basically no skill, an investment of only a few Euros, mere seconds to deploy and possessing potential against a wide variety of systems – from the dumbest padlocks to the most sophisticated bank vault combination systems – would land us in a real security crisis. Our dilemma would be even greater, were this novel attack method merely enabled by an otherwise useful and widely available technology.
Obviously, I am not really talking about locks any longer. This is a metaphor for the security risks posed by Large Language Models. As metaphors go, it is a crude and imperfect one, which I chose because it will help me emphasise two points: