I can see two ways of 'general AI' coming into existence:
1) A dramatic change in our level of understanding at a fundamental mechanical level of what consciousness is and how it works, and the ability to replicate it artificially and arbitrarily
or
2) An evolutionary development from incremental improvements and combinations of 'limited AI'.
In the case of 1), if we have that knowledge than we can similarly create and manipulate human consciousnesses as merely a special case of AI, and the issue is obviated by the more obvious one of the arbitrariness of humanity in the first place. In the case of 2) we would expect AI to evolve gradually in much the same was as any other life evolves. It's a similar situation to the silly 'grey goo' alarmists: any 'grey goo' would need to outcompete the existing 'green goo' occupying every ecological niche that has also had several gigayears head start. Given that an evolved distributed AI would need to exist within human-created infrastructure, and would evolve in a niche where no humans exist[1], it is more likely that there would be no competition for resources than for an AI to decide it needs to replace its entire 'supporting ecosystem' out of whole cloth.
[1] If we crack 'mind uploading' before we do AI creation, then we're back to 'grey goo' sillyness where AI would need to output humans who have a massive head-start.
I think you need to take a trip to Cambridge Massachusetts. These machines were doing just what you have described since the early 90s.
https://en.wikipedia.org/wiki/Thinking_Machines_Corporation
https://en.wikipedia.org/wiki/FROSTBURG
The system had a total of 500 billion 32-bit words (≈2 terabytes) of storage, 2.5 billion words (≈10 gigabytes) of memory, and could perform at a theoretical maximum 65.5 gigaFLOPS. The operating system CMost was based on Unix, but optimized for parallel processing.
And you can't forget about good ol, watson. https://en.wikipedia.org/wiki/Watson_(computer)
The point is, I don't think becoming "self-aware" will ever happen, unless Musk pushes his research into that, and no-one is quite as resourceful as he has been lately..
Last edited: