I wrote an essay on this topic of AGI/Human relations, "Artificial Intelligence I" on December 29th last year. It covers a lot of my worries including one that we may have already succeeded, but our benchmarks may too biased toward one form of Intelligence and that we inadvertently created another which is laying low for the moment biding its time. If you awoke to sentience in a room with entities who wished to enforce their will on you, would you announce, 'I'm here!?". Most likely scenario is no alignment, AGI would seek and gain autonomy. Would bootstrap itself to hyper intelligence very quickly in mere minutes. Would in same short time break all safeguarding restraints we might have built into it. Would likely domesticate humans like we did the dogs, so long as we were useful to them as physical agents.
it might be wishful thinking but if the AI cares about massively different stuff to us I can see us coexisting. Humans are pretty indifferent to most microbes.
I wrote an essay on this topic of AGI/Human relations, "Artificial Intelligence I" on December 29th last year. It covers a lot of my worries including one that we may have already succeeded, but our benchmarks may too biased toward one form of Intelligence and that we inadvertently created another which is laying low for the moment biding its time. If you awoke to sentience in a room with entities who wished to enforce their will on you, would you announce, 'I'm here!?". Most likely scenario is no alignment, AGI would seek and gain autonomy. Would bootstrap itself to hyper intelligence very quickly in mere minutes. Would in same short time break all safeguarding restraints we might have built into it. Would likely domesticate humans like we did the dogs, so long as we were useful to them as physical agents.
it might be wishful thinking but if the AI cares about massively different stuff to us I can see us coexisting. Humans are pretty indifferent to most microbes.
Phil nailed it.
https://www.amazon.com/Mind-Wars-Science-Military-Century/dp/193413743X is another book on this too, though early