From Futurism:
An artificial intelligence dubbed Claude, developed by AI research firm Anthropic, got a “marginal pass” on a recent blindly graded law and economics exam at George Mason University, according to a recent blog post by economics professor Alex Tabarrok.
It’s yet another warning shot that AI is experiencing a moment of explosive growth in capability — and it’s not just OpenAI’s ChatGPT that we have to worry about.
. . . .
Claude is already impressing academics with its ability to come up with strikingly thorough answers to complex prompts.
For one law exam question highlighted by Tabarrok, Claude was able to generate believable recommendations on how to change intellectual property laws.
“Overall, the goal should be to make IP laws less restrictive and make more works available to the public sooner,” the AI concluded. “But it is important to still provide some incentives and compensation to creators for a limited period.”
Overall, Tabarrok found that “Claude is a competitor to GPT-3 and in my view an improvement,” because it was able to generate a “credible response” that’s “better than many human responses.”
To be fair, others were less impressed with Claude’s efforts.
“To be honest, this looks more like Claude simply consumed and puked up a McKinsey report,” the Financial Times wrote in a piece on Tabarrok’s findings.
While Claude and ChatGPT are similar in terms of user experience, the models were trained in different ways, especially when it comes to ensuring that things don’t go out of hand.
Claude makes use of “constitutional AI,” as described in a yet-to-be-peer-reviewed paper shared by Anthropic researchers last month.
“We experiment with methods for training a harmless AI assistant through self-improvement, without any human labels identifying harmful outputs,” they wrote. “The process involves both a supervised learning and a reinforcement learning phase.”
“Often, language models trained to be ‘harmless’ have a tendency to become useless in the face of adversarial questions,” the company wrote in a December tweet. “Constitutional AI lets them respond to questions using a simple set of principles as a guide.”
Link to the rest at Futurism
Folks, what we are seeing here is the blue-sky ecstatic initial phase of the hype cycle. Remember was Google (not yet Waymo) was going to be manufacturing cars without steering wheels, just a couple of years out? That was about 2015 or so. They could put on a demo of a technology that, while not quite there yet, seemed awfully impressive. Here were are, sadder but wiser, with a tech that can do this under extremely strictly and narrowly defined (and expensive) circumstances, and only the die-hards are predicting that any time soon you will be able to punch in any address and take a nap while the car takes you there.
AI has gone through the hype cycle before. They come up with something new, and the think pieces immediately appear reporting that we are all obsolete and SkyNet is taking over. Then it turns out that while this new thing can do interesting, and perhaps even useful, things, it is is far more limited than the hype told us. This is the same thing happening yet again. It will have some interesting uses. It will force some parts of society to adapt. But let’s not get all hot and bothered. The quoted bit really does come across like a McKinsey report. We will adapt. Make all exams open book and don’t worry about the writing itself. Make questions that require more than a a writing prompt to answer.
I remember all the folks in 1981 who assured us the hype about personal computers was just a pipe dream.
Funny: I don’t remember that. I remember all the folks who in 1981 pointed out that personal computers couldn’t do all that much that was actually useful to ordinary people, and they were right. I fondly recall my brother trying to sell my Mom on the idea that she could create a database for her recipes. My mother correct suspected that he really wanted to play games, which even then was a primary force driving the market. A few years later, personal computers with WYSIWYG capabilities hit the market, and prices came down to the point where it was not completely ridiculous for ordinary people to buy. Also, games.
You would have a better case for naysayers about the internet, with Microsoft famously being late to the party. This was a brief phase. The internet for ordinary people hit in the mid 1990s, and by 2000 was utterly mainstream.
He may be thinking of Ken Olson or his precurdors:
https://quoteinvestigator.com/2017/09/14/home-computer/?amp=1
A more recent and ongoing example are ebooks.
https://www.alphr.com/features/242919/why-ebooks-are-a-fad/
Articles along those lines, citing screen fatigue, the smell of print, etc still pop up regularly, willfully ignoring the billions (yes, with a “B”) being spent annually in the US alone. Idiots gotta idiot.
Expect more of the same about “AI”, harping on what it isn’t and doesn’t do, instead of what it can and does do, allthewhile Microsoft and the rest of the tech world deploy the tech everywhere, raking in boatloads of money.
(And, for that matter, the real money isn’t in chatbots or prose compilation. Language processing is but the least of the uses of that tech. It’s just the one the media only now noticed. The real world changer is in places like these:
https://www.deere.com/en/sprayers/see-spray-ultimate/
John Deere, industrial scale gardening. 60% less pesticide and fertilizer, 25% higher yields.
Timely develooment, too.