How Forbes Built Its AI-Powered Tool, Adelaide On Google Cloud Vertex

From Forbes:

Forbes recently announced the beta launch of its new AI-powered news search engine called Adelaide.

. . . .

What does Adelaide do?

Adelaide provides a more intuitive search and discovery experience by having natural conversations with users to understand their interests and needs.

As one of the first major publishers to leverage generative AI for news curation, the launch of Adelaide represents an exciting development in how media companies can enhance audience engagement.

. . . .

Exploring Generative AI for Enhanced User Experience on Forbes.com

Our first step in our process of exploration of generative AI tools was to examine all of the potential use cases that might enhance the experience on forbes.com. While our team was excited about the prospects of many different utilizations of Gen AI, we quickly settled on a tool that could help improve search for users that were interested in a more conversational approach to exploring the Forbes archive.

The Search for the Ideal AI Tool

Finding a tool that was able to combine a fast, reliable and accurate search experience with the flexibility and creativity of a large language model was our first requirement for our new product.

Our team prioritized finding a tool that would be able to surface supporting articles that were closely related to user queries while also being context aware.

If at all possible, we wanted to be able to have both a chatbot-like experience that also surfaced articles for users to dive deeper into their queries.

We believed it was essential for our AI products to be deeply integrated with Forbes data. To set apart our use of large language models, we aimed to leverage the writings from our hundreds of journalists and contributors.

This approach helps the model minimize inaccuracies and offer responses that are more contextually relevant in a broad range of scenarios.This combination of requirements lead us to Google Cloud Platform’s new Vertex AI Search & Conversation product.

Seamless Integration with Vertex AI Search & Conversation

Getting set up with Vertex Search & Conversation turned out to be incredibly straightforward and fast. Since our team already has a lot of our data in BigQuery and Google Cloud Storage which directly integrate with Vertex, building out the datastore which would power the large language model was a simple matter of determining a criteria for inclusion, formatting and uploading.

We were also able to include article metadata in our datastore which improved model performance and made surfacing responses more relevant. From there, all we needed to do was make a call to the Vertex Search API and we were off and running. The simplicity of getting started in using Vertex Search ended up being a huge plus for our team.

Development resources could be used on other projects while we focused product development on the user experience and usability.

Refining and Testing the AI Tool

Once we had a functional proof of concept to use, we wanted to focus on testing, refining and structuring the responses from our new tool. We had a lot of productive back and forth with GCP Support and our account management team which led to fast iterations on bugs and configurations on both sides. Our first version of the tool was as an app in our internal messaging and communication platform, Slack.

Since many of our internal, non-technical stakeholders were already familiar with Slack, allowing them to test out Adelaide’s functionality there was a no brainer. This allowed us to quickly iterate on the feature set for Adelaide without having to rebuild the entire front end each time.

We wanted to be absolutely sure that the responses coming from Adelaide were suitable for the millions of users that would have access to the tool so thorough testing was a huge priority for our team.

Designing a User-Friendly Interface for Adelaide

Our design and front end teams were focused on building an experience for Adelaide that was clean and clear and focused on showcasing both the generative summary produced by the large language model as well as the related articles from Forbes authors.

Since generative AI is such a new technology, we also wanted to be able to provide some suggested prompts for users who were experiencing this kind of search experience for the first time.

Another key feature of Adelaide is the ability for users to have long conversational journeys, asking as many follow up questions as they would like while still retaining the context of their previous asks.

Launch and Reception of Adelaide

As we approached the launch date for the product, we were cautiously optimistic about how Adelaide would be received. As with every product launch, we worked through last minute bug fixes, security patches and technical glitches but were able to roll out the tool to a subset with great success!

Reception for the product was positive among senior leadership and in the press and we were happy to see the number of searches using Adelaide was steadily growing. Our team is continuously monitoring the tool and testing out improvements to make the experience the best possible one for our users.

Reflections and Future Directions

For many on our team, this was one of the more highly visible projects that we have worked on with lots of attention being paid to our work. A key takeaway for us internally is that this project worked the same way as many others that were much more internal or backend and that our process should be the same for projects that will be seen by dozens of people or by thousands. From the generative AI side, we also had a few learnings.

Link to the rest at Forbes

PG suggests that Forbes made a far more intelligent decision regarding its publication than The New York Times did with its retrograde copyright infringement lawsuit against Microsoft and OpenAI.

For one thing, Forbes is aggressively making money from an AI program by using its content while The New York Times will be spending a great deal of money on a multi-year lawsuit against Microsoft, an organization that has much deeper pockets than the newspaper and its investors do.

16 thoughts on “How Forbes Built Its AI-Powered Tool, Adelaide On Google Cloud Vertex”

  1. I’m not a fan of the glasses notion. It assumes no one ever uses prescription or bifocal lenses, and contact or cataract etc. surgery only cover so much.

    Still, I’ve wanted Trek Tech since I was home dateless on a Friday night in 8th grade watching the very first season. They can stick one in my coffin if they don’t speed up about it… 🙂

  2. I haven’t been captive in a corporate workplace for 20 years. I remember having to help employees get enough assistance learning Excel (and Outlook and Word).

    What I can’t figure out in these sorts of articles is how much of the use cases are from the IT group experimenting vs actual Subject Matter Experts (without the IT perspective) trying to use it.

    • Fair point.
      Can be either but with all the AI hype out there, chances are the latter are going through the former because the tech is datacenter hosted. The “experts” can’t work it on their own just yet. That’ll have to wait for NPU driven PCs and Windows 12 sometime in late 24/25. That’s when it’ll go *really* crazy when we start seeing task specific models at retail.

      For, its really an enterprise product which is why Microsoft ldads and others are trying to catch up.

      But unlike the dotcom bubble where Web training was almost “everybody on their own”, commercial “AI” development for now is tied to, and promoted/supported by, the big cloud services. So if Forbes wanted to experiment they would have assistance from Google (since that seems to be who they’re wedded to) in creating their custom LLM (or SLM) ADELAIDE.

      For example, Amazon has this:

      https://nypost.com/2023/11/20/business/amazon-to-offer-free-ai-courses-as-competition-for-talent-heats-up/

      “Amazon announced free courses to teach workers skills around artificial intelligence as the tech behemoth competes for talent with rivals including Microsoft and OpenAI.

      Dubbed “AI Ready,” the initiative includes eight “free AI and generative AI courses,” which Amazon hopes will teach as many as 2 million people by 2025 about the increasingly ubiquitous tech, the company said in an announcement shared Monday.

      “Courses are geared either towards business and nontechnical audiences or professional developers, covering topics such as “introduction to generative artificial intelligence,” “foundations of prompt engineering” and “low-code machine learnings on AWS,” Amazon’s cloud-computing company, Amazon Web Services.”

      Note the third one.

      Microsoft offers similar “assistance” through Microsoft Certifiation and Azure support. Some paid, no doubt. They started in Sept or earlier.

      I would be shocked if Google and IBM don’t do the same, soon if not already. AWS might be coming late to the party, hence the focus on “free”.

      Some lessons were learned from the dotcom era, apparently.
      Getting to the developers being more important than getting to the masses chief among the lessons. 😀

      • But doesn’t that also mean that AI usage is Always-On dependent? MS Word can stuff a spell-check into its product for offline use, but that won’t work with AI.

        Hard to make a feature completely useful as an always-available assist at the retail level (like SpellCheck) if it can’t be used offline.

        Yeah, yeah, I know — I’m a little old lady in the middle of nowhere who has, like, actual weather that interferes with web access. But still… no one wants to depend on an intermittently-available tool for hard-core work. If corporate environments are the primary place it can be used, then it won’t trickle everywhere in general use, just for always-connected use. Even corporate types want to use their laptops in other environments from time to time.

        • Not necessarily.
          Current implementations (mostly) go online when they need to query but the output stays local.

          Mostly but not all.
          And that is a limitation of do-everything models and current hardware.
          STABLE DIFFUSION is one example of a model that runs local on PCs…
          …with high end NVIDIA Graphics cards preferably.
          Check this from TOM’S HARDWARE:

          https://www.tomshardware.com/pc-components/gpus/stable-diffusion-benchmarks

          On the software side, small single task models are starting to emerge where the monster server-based models train smaller subsets. Microsoft ORCA 2.0 is one small model that is still multifunction, still effective, but much smaller than GPT and its peers. Orca 1.0 was built off 13Bilion parameters vs 100Trillion+ for GPT4.
          Orca 2.0 comes in a 7Billion parameter version as well. The “trick” is that instead of being fed massive amounts of raw data from which the training software extracts relationships and rules to build the model, the smaller models are trained from the output of those models instead of the raw data. Software writing software.

          There will be even smaller ones, focused on specific tasks, much as the existing non-LLM models in smartphones for imaging.

          So there is no reason why a Word co-pilot can’t run a local contextual, full sentence/paragraph spell checker that ensures not only valid spelling but also valid *usage*.

          I fully expect somebody, maybe MS, maybe Grammarly, maybe somebody else, to provide a local creative writer’s assistant than can analyze a manuscript and point out inconsistencies, open plotlines, character arcs, etc so the writer can see if they are doing what they mean to do. Maybe even compare it to the norms and expectations of a given genre.

          Now, these tools will most likely *not* run on today’s PCs. Hence the need for the intermitent datacenter connectivity. But that is today. By this spring we will see the first PCs with dedicate NEURAL PROCESSING UNIT (NPU) chips. Along with either a Windows 11.5 or Windows 12 or whtever name MS chooses for a version of Windows with an “AI” API set of calls to mediate between apps and AI models that call on the NPU.

          Intel has already announced their next generation of *laptop* chipsets that have multicore CPUs, game quality GPU, and NPU modules. The first AI PCs should be starting to show up rightbabout now.

          Microsoft is rumored to be preparing similar devices for their SURFACE PC line. And since Surface is (aside from being a billion dollar revenue stream) the tool MS uses to prod OEMs to do better products at better prices, there should be a good selection of laptop and desktop PCs at a variety of price points by the time the full Windows 12 (which *will* require an NPU) in 2025 or so.

          Naturally, some of those “AI” tasks the datacenter versions can complete in minutes will require hours or even days on an affordable PC. At first.

          I’ll link a few relevant looks at the future below.

          Things should get real interesting, real fast. As long as China stays out of Taiwan. 😐

          • Intel’s version of AI PCs as of Dec 14, 2023:

            https://www.tomshardware.com/laptops/intel-core-ultra-meteor-lake-u-h-series-specs-skus

            “And then there’s the NPU, which Intel claims is one of three “AI engines” in the processor (the other being the GPU for high-throughput and the CPU). The Core i7-165H can deliver up to 34 TeraOPS across the CPU, GPU, and NPU combined, but Intel hasn’t broken out TOPS across each individually.

            One question is how useful the AI features will be, and to whom. Intel says it has over 100 software vendors partnered to come up with over 300 “features,” and that its support for OpenVINO should allow for great development support.

            For AI performance, Intel matched its Ultra 7 165H up against a Core i7-1370P And Ryzen 7 7840U with Ryzen AI, with the new chip coming easily in first across a series of benchmarks. It’s interesting to see what Intel has chosen, as these are early days for these kinds of features.

            Overall, Intel says it offers 1.7 times more generative AI performance over its last-gen P -series chip, uses 38% less power in Zoom calls due to NPU offloading, and offers 2.5x Int8 power efficiency in UL Procyon Ai, also due to NPU offload. Additionally, Intel says it has Meta’s LLama2-7B running offline using Core Ultra and Superhuman.”

            “Intel says these laptops should be available in stores today, with more coming in 2024. This first batch includes PCs from MSI, Asus, Acer, Lenovo, and others. We’ll see soon how much the so-called AI PC, and Meteor Lake, have to offer.”

            I wouldn’t take these as mainstream plug and play AI devices just yet. I see them more like the original ALTAIR microcomputers from 1977. Seeding the developer field, consumer apps to follow.

          • A quickie overview of NPUs and how they differ from the GPU, graphics processors running “AI” code for now.

            https://www.windowscentral.com/hardware/what-is-npu-vs-gpu

            In a nutshell, they are specialized number crunchers that forgo the image processing pipelines that are the biggest part of GPUs. So they can more efficiently execute *some* “AI” calculations that are the bottlenecks in generative software.

          • And finally, this is AMD’s NPU PC story from Dec 6, 2023:

            https://arstechnica.com/gadgets/2023/12/amds-new-ryzen-8040-laptop-chips-look-a-lot-like-the-ryzen-7040-cpus/

            “Both Intel and AMD usually have processor updates to announce at CES in January, but AMD isn’t waiting to introduce its next-generation flagship laptop chips: the Ryzen 8040 series is coming to laptops starting in early 2024, though at first blush these chips look awfully similar to the Ryzen 7040 processors that AMD announced just seven months ago.

            Though the generational branding is jumping from 7000 to 8000, the CPU and GPU of the Ryzen 8040 series are nearly identical to the ones in the 7040 series. The chips AMD is announcing today use up to eight Zen 4 CPU cores and RDNA 3-based integrated GPUs (either a Radeon 780M with 12 compute units, or Radeon 760M or 740M GPUs with 8 or 4 CUs). The chips are manufactured using the same 4 nm TSMC process as the 7040 series.

            There’s also an AI-accelerating neural processing unit (NPU) that AMD claims is about 1.4 times faster than the one in the Ryzen 7040 series in large language models like Llama 2 and ONNX vision models. Both NPUs are based on the same XDNA architecture and have the same amount of processing hardware—AMD says that the AI performance improvements come mostly from higher clock speeds.”

            Note that these are not the first PC NPUs for AMD (or INTEL).
            They won’t be the last.

            Basically we’re back to the days of new PC generations every six months when the mantra was: “don’t buy a PC until you’re absolutely going to use it right away” because it would be superceded by a newer model within months.

              • Change is coming at warp speed and the ChatBot side is already moving post PC and muscling into the phone space in Trekish ways.

                https://www.androidheadlines.com/2023/11/ai-pin-is-offical.html

                Coming in march for $699, always connected via T-mobile. And yes, it is a phone too.

                “The AI Pin has access to a suite of AI services stored on the cloud, and this gives the device a seamless AI-powered experience. So, instead of going to a specific app to do a certain task, all you have to do is ask the AI PIn whatever you want, and it will automatically choose the AI service and perform the task you asked.”

                It’s not voice only:

                “The AI Pin is outfitted with a laser ink display that projects a simple UI onto the palm of your hand which you can interact with using simple hand gestures. This means controlling your Pin with a tilt and roll of your hand to highlight different options and making selections by closing your fingers like you’re about to enter a state of Zen meditation. Through this interface, you’ll be able to quickly control your music or read messages on the go without reaching into a pocket to pull out your smartphone.

                “You can also interact with the AI Pin using its touchpad to make use of the onboard multimodal AI through speech. Here you can converse with the onboard AI and ask it questions, set reminders, compose messages, or even ask it to look at what’s in front of you using the Pin’s 13MP camera to give you more information about a location or answer specific questions about particular items with more context.”

                So it goes Star Trek one better.

                Another thing to watch.

                • My hand started to follow along with your comment here as an unconscious “try it out” echo, and the twinges coming back reminded me of the whole notion of Repetitive Injury issues and what the consequences of “air-typing” at speed and in volume (and while driving) might be.

                  And then there’s the populist issues: speech instead of typing (but regional/foreign accents/vocabulary?), getting tattoos on the back of the non-typing hand of one’s favorite brand styles of keyboard (talk about locking in a brand), etc., etc., etc.

                • If you look carefully, the HUMANE AI pin creators are ex-Apple. Probably from the Watch division.
                  The device clearly is a concept the watch dept cooked up and saw Apple dawdling and decided to walk instead.

                  The pin itself is almost certainly not going to get much use from the projected UI since that tech is old (dating back to the PDA era) and never got anywhere. Now the voice actuated and phone functions would work fine with an ear bug.

                  As I said, very Trekish.

                  Hand contorsions? Doubtful.
                  It hasn’t gone far with VR/AR products.

                  But it’s a first gen implementation.

                  Another implementation to look forward to is frames for glasses. The limitation for smart glasses (and watches) is the display tech. Remove that, add effective voice controls, and you have a basic phone. Make calls and texts, receive both. Maybe GPS, take photos and videos, and music playback.
                  Somebody will try it.

Comments are closed.