Kevin Kelly believes the A.I. of the future might require built-in intellectual blinders:
Most of the commercial work completed by AI will be done by special-purpose, narrowly focused software brains that can, for example, translate any language into any other language, but do little else. Drive a car, but not converse. Or recall every pixel of every video on YouTube but not anticipate your work routines. In the next 10 years, 99 percent of the artificial intelligence that you will interact with, directly or indirectly, will be nerdily autistic, supersmart specialists.
In fact, this won’t really be intelligence, at least not as we’ve come to think of it. Indeed, intelligence may be a liability – especially if by “intelligence” we mean our peculiar self-awareness, all our frantic loops of introspection and messy currents of self-consciousness. … As AIs develop, we might have to engineer ways to prevent consciousness in them – and our most premium AI services will likely be advertised as consciousness-free.
The Dish 