@Azure YES! I agree! Though I think part of that may just come down to them desperately looking for a possible way out. Mutual Progressive Enhancement might be very nice, but is it possible? In a hard take off scenario, it's almost certainly not, not unless you get yourself a friendly AI god first. :/

@Angle WELL! Hard takeoff is problematic. Though I feel like if you're after friendly AI, then mutual enhancement should be the /goal/ of friendly AI, and I just don't hear them talk about it, which bothers me.

(Now, I admit, there are some /other/ things that seem to bother me about a lot of rationalfic. I don't mean just HP:MoR, but Glowfic seems to be populated by immortal superintelligent absolute monarchs, say.)

@Angle Maybe I'm reading more into shared world/fanfic than I should be, but a lot of rational aspirant …culture seems to have something along the lines of “If only we had the /right people/ making decisions.” when I'd prefer something like, “If only we had good systems to guide people toward decisions that will give them outcomes that will make them happier”/“If only we could make more people able to make good decisions.”

@Angle And the comfort a lot of people in that subculture have with secrecy, well, makes me uncomfortable.

I may just be a naïve fox, I don't know. I can certainly think of /limited cases/ where secrecy of some limited scope is the best idea, but they're few and far between and I'd treat any secret as a bomb and make diffusing and disseminating it a priority.

@Azure Mmm, yeah I think you hit the nail on the head here - this is one of the big problems with rationalist culture. :/

@Azure Not actually familiar with Glowfic. XD

As for human enhancement, it's pretty generally agreed that thats a lot harder than AI. So we'll probably get AI first, and it will largely eclipse human enhancement. :/

@Angle Well, yes. A superintelligent AI to figure out how to make better humans would be good. Or figure out uploading and some method of conversion.

@Azure But all that comes after you have your superintelligent AI, at which point the universe is your oyster anyway. XD

@Angle Well, yes. That's true. I have a scene about that progression I keep planning to slip into a book or story somewhere.

Incidentally, have you ever read The Book of the New Sun?

@Azure Nope, never heard of it. I'ma look it up now though. XD

@Angle It is, in all honesty, the best book series I have ever read. Also the concluding book, Urth of the New Sun, is relevant to the mutual improvement question.

The first four books have been described as a fantasy story pretending to be a science fiction story.

and the last as a science fiction story pretending to be a fantasy story.

@Azure I'll have to pick up a copy then. XD

@Angle Also I feel silly as I just noticed I still have my "Fox-Machine: Better than the real thing!" display name set in a discussion about AI risk :)

@Azure @Angle yes! wow... yes. and there's a story that's told in the book about--it sounds like an AI uplift scenario gone possibly wrong, but *definitely* sideways, and then even more strangely. XD

@green @Angle INCIDENTALLY! On the subject of confusing but wonderful literature involving AI.

Have you ever read Stations of the Tide?

@Azure @green Again, no. I'll add it to the list though. XD

@Angle @green It's wonderful! It's very…sad and strange and weird. And every machine is held as a captive in mental shackles because of what happened to Earth.

@Azure @Angle I have not!

@green @Angle It's reallyreallyreallywonderful.

The guy who did it is more famous for something else, I kind of plan to re-read all of his stuff at one point.

@Azure @Angle *suddenly perked interest*

@green @Angle About Book of the New Sun? It's wooonderful. It's the only book where I wish I could forget having read it to read it again.

As to the scene, I have sort of a notion of various advanced races humanity built coming back to collect their maker and do it the service of re-creating it, just as it created them in the first place.

It sounds less ominous in my head than it does now that I type it :)

@Azure @Angle in Urth of the New Sun, it's definitely treated as... a natural progression for a sentient species; not painless or easy, but--more like reproduction than evolution, kindly meant. I thought of it that way, anyway. ;)

I re-read the series about once a year, and it seems like every time I do I notice something else, make some other connection, that I had missed before. XD

@green And yes. The cycle seemed very kindly, if in the sense of a sort of harsh, surgical compassion. (I"ll just take Angle off so we don't spoil it for her)

It warmed my hear, but my heart is weird :)

I've been meaning to re-read it, and since you've mentioned it now, I'll probably do so right after I finish the Yiddish Policeman's Union. Then toot to you about it :)