The more “user friendly” our machines become, the harder it gets for humans to truly master them. This phenomenon is known as the automation paradox.
The problem is very real for pilots. The more sophisticated aircraft become, the more they depend on an ever increasing number of systems that automates what used to be human tasks. That’s fine in fair weather, but spells trouble when things get out of hand.
Programmers report a similar cognitive rip current, as they‘re getting comfortable with using GitHub’s code automation tool Copilot. It boosts productivity by orders of magnitude and is so good at second guessing the coder’s intention, that many seasoned programmers starts relating to it as some sort of sentient being.
But for all its capabilities, Copilot has already been caught introducing some very subtle bugs into programs it helps writing. Bugs that are exceedingly hard to catch, since the very smartness of Copilot means that the human-in-the-loop programmer is gradually losing touch with the actual code.
The movement from nitty gritty details towards ever more lofty levels of abstractions has been a trend in technology for so long, that it almost seems inevitable.
But loss of control is not the only tradeoff in this development, we also tend to lose sight of what’s possible. Some of the most interesting pieces of technology I see coming out of the labs, are extremely close to the machine. Starting from first principles, these teams go slow as they carefully lay out the foundations for whole new paradigms. I think in the end, that’s the only way real innovation can happen.