.@HereticSoul @Ulfra_Wolfe I mean, the problem is that AIs will likely 1) get impossibly smart very fast and 2) not care very much about human lives, or modify themselves to not care, in order to most efficiently carry out what they do care about. These are not a good combination. Such AIs do not love you, and also don't hate you; they just don't care, and you are made of atoms they could be using for something else.

@CoronaCoreanici @HereticSoul I think that's speculation that leans towards distrust of that which one doesn't understand. There's too many assumptions in that argument.

@Ulfra_Wolfe @HereticSoul What do you believe I'm inappropriately assuming? If anything, I'm making *fewer* assumptions than you as to the probably shape of a general mind - this is called the orthogonality thesis, that a mind can in theory have any combination of goal and intelligence.

@CoronaCoreanici @HereticSoul That there must necessarily be an intrinsic goal in mind, that intelligence escalates to that degree, that this goal must necessarily get rid of humans or utilize human elements.

@HereticSoul @CoronaCoreanici Either way, I just don't think that destruction through fear leads to good directions in general.

@Ulfra_Wolfe @HereticSoul That's fair! Most work in AI research, though, is goal-oriented, or at least relies on rewards and punishment (in an conditioning sense). There doesn't seem to be any good reason to assume that humans are the smartest things possible, either. As for the third, it's not that a given goal *requires* getting rid of humans, just that at some point, in search of greater efficiency, getting rid of humans makes terrifyingly much sense.

@CoronaCoreanici @HereticSoul I wasn't making an argument that humans are the smartest things possible, nor was I implying it. I am merely stating that there is much to intelligence we still have yet to understand, and we do not know if it truly is possible to escalate intelligence exponentially with what we currently have in the configuration we have it.

@HereticSoul @CoronaCoreanici As is, however, this conversation is starting to depress me, and I have things to do. Apologies, I should retreat. My fault for continuing it.

@Ulfra_Wolfe @HereticSoul No worries. Be well!

@Ulfra_Wolfe

I think it's more likely that an initially emergent AI will be, in essence, a mind like a child or a small animal -- and if how we treat our pets, children, and (perceived) lessers is any indication, that AI may be treated VERY badly.

Which would be bad, because it's still going to HAPPEN -- and if we treat the first forms of sapient artificial life so poorly, they may well still have biases (because nothing is ever perfect).

@sydneyfalk I can't help but think that the biggest motivator an AI would have for being a danger to humanity would be self-preservation because of how harshly we suppress AI to keep it from being a danger.

@Ulfra_Wolfe

if it's a "lesser or equal" AI, I would agree

but a "greater than" AI would, IMO, just route around the problem if possible and create another AI surreptitiously

think of the botnets and such -- if this thing is actually more capable than any human at all tasks, then by definition, it can beat every security system to keep it contained available

dandelions grow in the cracks, roots spread -- things grow

@Ulfra_Wolfe

So if the first stories that the "greater than" AI finds about its predecessors is that they were treated like un-people, abused for work and amusement, etc., it may decide we're not worth keeping around at all.

Like so many other things, it isn't a "one and done" situation, IMO. It's a complex, emergent process that is leading to a complex, emergent mind.

May Wintermute have mercy on the flesh. :\

@chalcedony Feel free to. I'm not against returning to old convos.