Home » Quotes » I Don’t Want to Really Scare You

I Don’t Want to Really Scare You

15 July 2019

I don’t want to really scare you, but it was alarming how many people I talked to who are highly placed people in AI who have retreats that are sort of ‘bug out’ houses, to which they could flee if it all hits the fan.

James Barrat, author of Our Final Invention: Artificial Intelligence and the End of the Human Era



20 Comments to “I Don’t Want to Really Scare You”

  1. Who doesn’t want to be able to ‘get away from it all’? 😉

    Though I will be taking my netbook and a solar array to keep it charged. Monthly trips to the edge of cell coverage is often enough to see what’s going on in the world.

  2. I don’t really want to scare you, but I encountered my first survivalist at a party in Charlotte, N.C. in 1988.

    I’ll bet his supplies of bottled water and Army rations are still fresh.

  3. There always seems to be a segment of humanity that is convinced the roof is about to fall in. Probably right.

    Desert monasticism began around A.D.300, but I suspect it was not a new idea then.

  4. AIs vs. the Lizard People.

    Whoever wins, the spectacle will be glorious!

  5. The Skynet is falling! The Skynet is falling! ~ Chicken Little

    But seriously, I’m not expecting this to happen. I see the stuff about advances in AI, and I’m not convinced. It does make for good SF books and movies, though. I’m probably going to write one or two before it’s all said and done.

  6. I wonder if the people in question are really highly placed in the development of machine learning systems are are just celebrated for hyping up AI? Some of the latter seem to believe their own SF.

    Fortunately, the singularity will arrive, they’ll all upload their consciousness to computers and we can have them declared legally dead, pull the plug and distribute their assets.

    • Raymund Z. Gallum dealt with the issue way back in the fifties in PEOPLE MINUS X where the protagonists excape death by uploading into Androids. Twice. Both sets see themselves as the originals. The case can be made that neither is.

      I doubt he was the first or the last.
      (In STAR TREK McCoy thinks he died the first time he stepped into a transporter.)

      It all begs the question: “if a difference makes no difference is it really a difference?”

      • McCoy was right. Everyone dies every time they step into a transporter, it’s just that the copy is so good that the copy cons itself into believing they are the original. Personally, I think the process would result in minor but increasing degradation until the whole of the Enterprise’s bridge crew – and any security staff that haven’t already died – would be reduced to gibbering idiots.

        So I guess that my answer to “if a difference makes no difference is it really a difference?” is yes, but because I don’t really believe in the “no difference”.

        • Different question, that 😉

          STNG was using a different transporter, btw. Theirs creates wormholes. (Remember the bugs in Realm of Fear?) No difference there.

          • I fear your memories – or interpretation – of STNG differ from mine. I certainly recall the system as similar to – but much more reliable than – that in TOS, though I suppose that Barclay’s conscious appreciation of his travel through the matter stream could be thought of as a non instantaneous journey between two gates. Still, it was also seemed to be a matter of making a replacement copy (hence duplicate Rikers).

            I always preferred the “gate” approach in my SF (less philosophy to worry about) even if the handwavium doesn’t do more than make it clear that the character hasn’t been killed and copied.

            • The missing crewmen were alive and conscious between the two ships. So was Barkley or he wouldn’t have seen them. Meaning it was an actual place.

              Doesn’t much matter because the writers played fast and loose with the rules–bio-filters and all–though not as recklessly as tbe STAR WARS crowd.

        • Personally, I think the process would result in minor but increasing degradation until the whole of the Enterprise’s bridge crew – and any security staff that haven’t already died – would be reduced to gibbering idiots.

          Well, I suppose that would explain Insurrection and Nemesis

    • Maybe after the initial consciousness upload, the computer can run a quick and precise series of tests to determine actual intelligence and knowledge in various fields and the degree to which they tend to overhype certain topics. The results of the tests could then be appended to all online articles, comments, etc., made by the person.

      • Assuming they can be bothered to interact with the meat world in this way (as distinct from issuing orders) and haven’t been deleted as per my “dead is dead” plan. I really don’t want the world to end up run by pseudo-immortal beings who have managed to accumulate sufficient money to pay for their upload by whatever means, fair or foul (probably the latter).

        • They won’t be running the world. They’ll be too busy missing their bodies for that.

          (Star Trek, again. Plus Slylark DuQuesne. And Weber’s latest. Humans don’t make good disembodied intelligences.)

          • Sorry Felix, that’s all speculative theory – fine for fiction – but we won’t know what happens until we have empirical results. I predict 99% total failure, 0.999% goes mad, 0.001% becomes an inhuman super-intelligence and turns into Skynet. YMMV.

            ps. can you think of many nice AIs, other than Mike from The Moon is a Harsh Mistress (and he/she/it was a bit casual about killing people)? My memory of all those old stories is fading: I can recall reading PEOPLE MINUS X but not what it was about.

            • Lots of AIs out there, many from before AI started getting hyped. Most were nice but not in necessarily human-nice.

              Most of the not-nice simply know too much about humans. 🙂

              – Colossus was from a trilogy by D.F. JONES. Made into a movie, THE FORBIN PROJECT. Not nice but not Skynet bad either.

              – Harlie in Davod Gerold’s When Harlie was one.

              – Heinlein had Dora and Athena in Time enough for love.

              – Asimov had dozens of AI “positronic brains” running his Robots, most notably R. Daneel Olivaw and R. Giskard Reventlov. Plus Multivac in its ultimate evolution.

              – His I, Robot collection had Steven Byerly in two stories. a saintly human or maybe a robot. All nice, one way or another, most especially the ones in the Donovan and Powell stories. One refuses to believe humans created him so creates a religion instead. Another designs a hyperdrive that works but kills humans–a no-no. But a second look reveals they don’t stay dead so it okay. More or less. The experience gives it a sense of humor so it makes Donovan and Powell’s experience…interesting..

              – David Weber’s Gordian Knot has not one but two societies where humans transition to pure AIs voluntarily and coexist with meat human partners. Plus a true AI painfully aware it’s “just” an AI. His EMPIRE FROM THE ASHES trilogy had Dahak and a horde of lesser AIs. All nice enough.

              – John Ringo’s TROY RISING has various level of constrained AIs in use all over the galaxy and a much smaller number of unconstrained AI’s with at least one being a main character. All run the same hardware but are upgraded from one level to the next depending on required capabilities. Generally upgrades only required a “license” code.

              – The granddaddy of them all is the original I, robot by Eando Binder: Adam Link. Ten stories from the 20’s on. Early on he was a scrupulously nice robot. Over time, he evolved.

              – One can argue about Helen O’loy by Lester del Rey and Metropolis’s “Maria” by Fritz Lang. I’d say yes to Helen, no to Maria.

              Lots of AIs in books, games, and TV these days, some good true AIs like Mass Effect Legion and EDI. Many neutral. A few evil. But Skynet has become such a cliche…

              In most of the early stories AIs were generally wedded to their bodies/computers but in recent times stories have had emergent AIs floating through the Internet (or equivalent) or uploading/replicating themselves with a mild distinction between self-aware/sentient robots and computers.

              A lot less interest in discorporated humans of late.
              Cyberpunk just isn’t “in” these days. 😉

              Librivox has a PD audiobook although the last I heard the story wasn’t.


  7. Suburbanbanshee

    Desert monasticism was about training for Heaven, and about replacing martyrdom with some equally rigorous practice. People went out to the wilderness because life was getting too easy, not because they thought things would collapse.

Sorry, the comment form is closed at this time.