Suppose we develop a superintelligent system tasked with making as many paper clips as possible. It might simply convert everything, humanity included, into paper clips. Even a modestly capable AI might undergo a rapid period of self-improvement to achieve its goals, bypassing humanity in the process.
↧