The singularity, or more precisely the technological singularity, can be most simply defined as the emergence of super-intelligence, i.e. a turning point in the course of artificial intelligence development where the intelligence of machines exceeds the highest level of human intelligence. Although various definitions have been proposed for the term singularity, it seems that the emergence of super-intelligence is one of the most important ramifications of this phenomenon.
Consider an AI that is capable of designing a software system more intelligent than itself. Then the resultant AI will be capable of designing an even more intelligent software system. Some scientists believe that such a situation will lead to an explosion of intelligence.
This cycle will continue until the limitations of the laws of physics kick in and prevent further advancement of the intelligence level.
Although there are various opinions on this, but many scientists believe that the technological singularity will occur within a few decades from now. For example, Ray Kurzweil predicts that in a few decades, the computing power of ordinary computers will exceed that of the unenhanced human brain.
Although this topic has been frequently the subject of science fiction works, but many scientists have expressed concern over the future of AI and its impact on human lives.
If robots become more intelligent than humans, it may not be possible for us to understand the logic behind their decisions. They may choose goals based on their own understanding that may not be compatible with the values generally accepted by humans.
This is probably one of the reasons that some prominent scientists, including Stephen Hawking and Elon Musk, have asked for setting strict regulations for AI research.
