The upheavals

This content has been archived. It may no longer be accurate or relevant.

The upheavals [of artificial intelligence] can escalate quickly and become scarier and even cataclysmic. Imagine how a medical robot, originally programmed to rid cancer, could conclude that the best way to obliterate cancer is to exterminate humans who are genetically prone to the disease.

Nick Bilton

10 thoughts on “The upheavals”

  1. originally programmed to rid cancer

    So, who changed the program? And if someone did change the program, is everyone else sitting in the bleachers watching it kill people? It has a plug. Pull it.

  2. And exactly *how* would such a robot go about such a genocidal spree, pray tell?
    Only an idiot would enable such a scenario to come to pass.

    First, because that scenario has been explored for over a century (more, if you consider Shelly’s FRANKENSTEIN and its creature) and counters conceived. It even has an official name Frankenstein syndrome:

    Second, because genocide is a uniquely human (absolutist) endeavor and any AI capable of even contemplating it would require a human level intellect “and* human level emotional responses. Neither is likely to appear any time soon. Or ever.

    Third, that particular scenario involves medical technology which is by design single purpose and fixed. Assumes the robot decides to kill humans. It kills the patient. Then what?

    Fourth, any device capable of such action needs a power supply. There’s this thing called a power button. AKA, kill switch. So not only would a fool have to do the improbable, everybody else would have to be stupid enough to sit around doing nothing. Riiiight.

    Fifth, pretty much everybody working in robotics and “AI” (unlike luddite pundits) ​is aware of the THREE LAWS OF ROBOTICS from fiction as well as the Frankenstein syndrome. In the very first of his celebrated ROBOT STORIES 80 years ago, Asimov said it best: No robot (or AI) can do anything it hasn’t been told how to do. Is the AI trained by a covert assassination expert?

    By now we know enough to understand that the “rogue AI” exists solely in the realm of dated SF and bad movies. And in blurbs from pundits trying to appear wise and thoughtful.

    • Mind you: killer faux-AI systems are a certainty and are imminent if not already deployed. An obvious extension of existing area denial hardened Sentry Gun systems:

      Both Israel and South Korea have deployed them on their militarized borders to scan for, identify, and eliminate intruders–under human command. Like the military drones in combat use all over, they are remote controlled and allow a single human to oversee multiple posts. Inserting an “AI” in the loop would reduce the number of humans supervising the turrets, much like the convoy concept of self driving long distance haulers or the swarm attack drones a modern fighter aircraft can guide.

      Technically, yes, killer “AI” but only because humans tell them.
      And not even tbe russians have removed humans from the loop.
      They too are familiar with the BOLO and BERSERKERS of SF.
      And they’re not idiots.

        • Uh, that’s the NAVY.
          Wholly different domain and mission. More, they’re late to the party.
          Air Combat is a different story. Try this instead:


          Drone swarms are already winning wars.

          The USAF and RAAF are already test flying air combat drones. Both collaborating on slightly different products and (because of the different procurement systems) RAAF is slightly ahead with their Loyal Wingman; their prototypes are already flying

          The USAF is looking to deploy swarms of different types of cheap single function (often single use) drones controlled by F35, F32, and brand new F15EX (non-stealth planes). Likely supported by (relatively) dirt cheap ($35,000) smarter JDAM glide bombs upgraded to be able to hit mobile and swarms of long range missiles deployed by cargo craft. Stealth is useful but expensive. Visible but cheap weapons are a viable alternative to lead the waym

          The idea is a layered attack force that uses different kinds of “AI” automation to magnify the manned platforms power and minimize risk while saturating the defenses. Small recon drones provide realtime sensor data for the manned platforms that manage the drones, missiles, and gliding bombs. For example, a typical chinese destroyer carries 20-30 air defense missiles with a 20m range and close defensd gatlings good for another dozen or two. A single long range F15EX can drop 28 JDAMS to be guided by a single stealthy F22 or F35 without using any of its armament. A flight of a dozen F35s can send hundreds of missiles and bombs in a single attack, forcing a destroyer to use up its entire defensive load and either sink it or force it back to rearm before getting to attack range.

          The Armenians were prepared for a tank-tank war, expensive platform vs expensive platform, instezd they got cheap and expendable kamikaze (highly manuevrable) drones; likewise the chinese fleet is geared to fight off a surface fleet, not a dispersed cloud of air delivered weapons. In a confrontation with the US they may instead be faced with thousands of drones, tens of thousands sublaunched cruise missiles and air launched long range missiles from cargo planes, and hundreds of thousands of glide bombs.

          Stealth has ruled for 40 years but the next war looks to be about missiles in vast numbers, either long distance ground launched (expensive) or short/mid range assets. And “AI” and computer systems have a role to play in making those unmanned weapons more survivable and deadlier.

          You’ve heard about AUKUS, of course.
          Most of the media focus has been on “Australia is getting nuclear attack submarines!” Less publicized is that the subs will show up next decade. Even less discussed is the near term cyberwarfare, communications, drone, and “AI” tech and missile stores Australia is getting.

          Odds are that if China engages the US this decade, the only part of the US Navy to see aggressive action will be the subs and carriers. The rest of the fleet will be mostly missile defense. All the open publicity out there is intended to get China to rethink the balance of power and buy time for the next gen “conventional” systems. Lasers, rail guns, orbital bombers… 😀

          Russia will be a different story. That will be mostly up to the Army and the new hypersonic artillery. And they’re less likely to be spooked by drone swarms.

          Frankly, robot ships aren’t likely to be relevant (or needed) any time soon.
          What the Navy needs “yesterday” is semisubmersible “arsenal boats”. One or two per carrier group. And more missile colliers.
          Logistics, logistics, logistics.
          No public mention of those ship classes, though.
          Not sexy.

        • The Loyal Wingman party line, with a link to the test flight video from last summer:

          The US has several candidates from the usual suspects and at least one more from an AFRL/KRATOS precursor for their own Skyborg program.

          Early US talk was initial deployment by 2023. Riight.
          Expect the RAAF to get there first and way cheaper.

          The whole point of these combat drones is to make them significantly cheaper by reducing reusability/maintenance costs. (Take those claims with a pound of salt.)

          Still, Tomahawk cruise missiles cost $3m+ and the cheapest manned fighters in 3rd world use run $17-30M. By contrast F35 costs well over $100M up front and $25,000+ to run. Per hour. If they were serious, they could probably hit $20M for a drone good for a couple of sorties.

          The tech makes sense.
          DOD management rarely does.
          We’ll see.

          • I knew that, but that wasn’t my point, which was rather unstoppable killing machines need maintenance and a logistics train. But, fair enough.

            • Not if tbey are disposable: think of them as more flexible cruise missiles instead of unmanned vehicles. That’s where the “bodyguard” part of Loyal Wingman/Skyborg comes in: they can sacrifice itself to protect the manned platform. (Much like Asimovian robotsunder First Law.)

              And if the things fly themselves to tbe front logistics gets easier, not more expensive.

              The US M.I.C. might still inflate the cost beyond usability but other countries won’t suffer those scytams.

Comments are closed.