Microsoft’s new Copilot will change Office documents forever

by
This content has been archived. It may no longer be accurate or relevant.

From The Verge:

Microsoft’s new AI-powered Copilot summarized my meeting instantly yesterday (the meeting was with Microsoft to discuss Copilot, of course) before listing out the questions I’d asked just seconds before. I’ve watched Microsoft demo the future of work for years with concepts about virtual assistants, but Copilot is the closest thing I’ve ever seen to them coming true.

“In our minds this is the new way of computing, the new way of working with technology, and the most adaptive technology we’ve seen,” says Jon Friedman, corporate vice president of design and research at Microsoft, in an interview with The Verge.

I was speaking to Friedman in a Teams call when he activated Copilot midway through our meeting to perform its AI-powered magic. Microsoft has a flashy marketing video that shows off Copilot’s potential, but seeing Friedman demonstrate this in real time across Office apps and in Teams left me convinced it will forever change how we interact with software, create documents, and ultimately, how we work.

. . . .

Copilot appears in Office apps as a useful AI chatbot on the sidebar, but it’s much more than just that. You could be in the middle of a Word document, and it will gently appear when you highlight an entire paragraph — much like how Word has UI prompts that highlight your spelling mistakes. You can use it to rewrite your paragraphs with 10 suggestions of new text to flick through and freely edit, or you can have Copilot generate entire documents for you.

. . . .

Microsoft has customized this Copilot system for every Office app, so there are different ways to command it. Friedman demonstrated to me how Copilot can help you write emails in Outlook, offering up short or long message drafts with options to change the tone. It even works in the mobile version of Outlook, which got me thinking about the ways this could speed up work on the go.

“Outlook mobile is the first place where we’re doing a big push,” explains Friedman. Outlook can summarize all your emails on the go, generate drafts, and generally make it easier to triage your inbox. But imagine creating entire Word documents from your phone without having to type on a tiny on-screen keyboard. “We’ll have more to talk about mobile in the coming months,” says Friedman. But you can imagine where things will go.

Link to the rest at The Verge

Click to Tweet/Email/Share This Post

21 thoughts on “Microsoft’s new Copilot will change Office documents forever”

  1. So, will MS also improve their grammar checker so that it’s as good as, say, Google Docs and doesn’t give outright incorrect advice on a frequent basis? And is of use to fiction authors as well as business users?

    (He says, expecting the answer “no”.)

  2. I have Viasat at the moment (in my collection of defunct dishes on my farm at the base of the Allegheny Plateau). It’s been a while since I’ve comparison-price shopped.

    • Thanks.
      I looked it up.
      It uses geosynchronous satellites just like Hughes (at 22,300 miles) instead of a constellation of low orbit satellites at 300-500 miles. It makes a difference not only in real world data speed but also in responsiveness (half a second lag to geo vs 20 milliseconds or less at leo).

      If you’re not happy, you might want to look up Starlink reviews. Youtube has a bunch. Starlink currently runs off 3500 satelites and counting, with a target of 40,000 as customer base grows.

      The startup cost is non-trivial but performance is comparable to cable starting at good (50Mb minimum) to great 150Mb+. Not cheap. $110 per month. It works best in low population areas but it is good enough for online gaming, even multiplayer shooters.

      https://www.pcmag.com/reviews/starlink

      What holds me back is that I’m on the edge of a small city (200,000) with both cable and 5G. Neither is cheap but still less than Starlink. But since I don’t watch cable TV the higher availability of Starlink come disaster time is enticing. Keeps me on the fence.

    • No prob.
      BTW, if you don’t mind, which satellite Internet do you use?
      Hughes or Starlink?
      After the big hurricane I spent a couple months without broadband. I’m keeping an eye on STARLINK and Amazon’s project Kuiper. Starlink by all reports works well (Ukraine loves it and DOD is signing up) but it’s a bit pricier than I’d prefer especially since the cableco is running scared and offering free boosts. 🙂

      I started at 40Mbps, which was fine; got boosted to 100 for $3 and 200 for free. More than I need but it *is* “free”. 😉

  4. I expect those of us using the separate MS Suite (vs Office 365) to not have this option any time soon. On the other hand, I don’t want to give up my access to tool use locally for Word & Excel (in particular) in the midst of power outage and heavy rains (satellite internet). Sigh…

  5. One wonders if this new chatbot-assisted system will be embued with the same hostility to legal and government documents as the rest of Office is — or if it will evolve…

    Here are some examples:

    • There are built-in templates, styles, numbering systems, and so on that comply with multiple editions of the APA Style Manual; just try finding “American legal numbering,” lined pleading paper, or a compliant small caps feature.

    • For almost all “special characters,” one can use an <alt->[numpadunicode] representation. This is pretty universal across text editors for a wide variety of graphical systems. However, one cannot, in Word, enter § (<alt->[021]) or ¶ (<alt->[020]) any way except <alt><ctrl>s (or p, respectively) — and similarly with m-dashes and n-dashes. Care to guess which broad area of writing requires the use of those symbols the most frequently? Care to guess how hard it is to train law students and old-fogey lawyers to cross the streams?

    • Let’s not start with the “tables of authorities” problem, or alphabetization issues, or whether and when to allow/require line breaks in citations, or counting words correctly (it’s been 24 years and Word still doesn’t do it correctly).

    I say send the new chatbot to law school, let it learn, and then watch the fights with the programmers and management… because I’m evil.

      • You owe me a clean keyboard, Felix. Fortunately, it’s late enough in the day that I’m not drinking hot coffee, so at least there are no burns in my nasal passages.

        To pick at one thing, the idiots who wrote that sales-and-marketing pitch have clearly never handled information subject to a segregated-confidentiality order. If they had, they would know that Office365 doesn’t support segregated-by-user access to parts of documents, and by design cannot. It’s an immense conceptual and practical problem; there’s no reason that a commercial/consumer-grade system should expect to solve problems that led right up to the reason that the cloud will never be a complete solution: The Snowden problem.

        Then notice, too, that they say nothing whatsoever about the problems I noted above. I want my WordPerfect back, dammit! Word is a useful program for many kinds of documents; with long experience (since 1983…) with its quirks and shortcomings, I’ve developed workarounds for its inability to deal with basic legal and government documents, but I’m annoyed at the continued-through-however-many-versions kludges.

          • We still use it in my office. The usual objection is swapping documents with other offices that use Word. This is a legitimate concern if those docs involve fancy formatting or series of edits that need to be preserved. It matters not one whit for routine pleadings. And yes, modern Word Perfect still has reveal codes. Life is good.

            It is a testament to the power of marketing that I have had this discussion many times over the years, and nearly invariably get a wistful response that this sounds lovely, but is out of the question of course.

            • Marketing?
              More like the tyranny of the installed base.

              MS doesn’t need to market office and Corel can’t afford it.
              The nature of platforms: one misstep and you’re history. And Word Perfect Corp didn’t make *one* mistake, but a dozen. Ditto Lotus. Ashton-Tate, Borland, etc.

              Microsoft, too: they killed WinCE just as mobility became significant. No repeat of that this time. This time Nadella is going for the jugular.

            • Unfortunately, I inherently have to swap documents with other offices — and worse yet, with clients and publishers and music/film-industry maroons! — with change-tracking active. And RTL quotations (the other weakness of WordPerfect, but that’s a long and ugly story involving undocumented critical “features” in Windows95, idiot managers at the new owners, and ossification of the codebase since the 1990s).

              It’s very much like learning to drive in the UK (or most of the Commonwealth and/or Japan): Driving on the left side of the road puts the most-critical-to-avoid-a-collision hand-operated controls, and the interior mirror (which given the one-lane roads is probably the only one that hasn’t been dinged/cracked or bashed off), on the dominant side of only 11% of the population† and exposes the dominant side to incidental debris damage through an open window. But if you have to go to London, you have to drive on the left. That I’ve developed all the kludges (except the ones that rely upon reveal-codes… but I even have solutions for that when it’s truly critical), and can swap back and forth between RTL and LTR in the same document in Word’s inept system, doesn’t make me like it — and makes me notice the overt hostility to this substantial subset of users even more.

              † Which was no problem for me, because I’m a left-winger in my right mind/brain.

  6. I have one major question about the implementation of the whole AI-assist phenom: doesn’t it require (and will continue to require indefinitely) “always-connected” access? That’s a significant… pain-in-the-ass for anyone whose Internet is not 100% reliable/available, or who is off-line for any of a variety of reasons.

    Yeah, yeah, I know that won’t stop it (and shouldn’t) but… sigh…

    • Not all GPT implementations requite “always online”.
      Remember, it is a multistage process: the bulk of the valueadd work is upfront, creating the *model* that processes the data. Some models are massive and need an online platform but some are small enough to run on a PC. Stable Diffusion is one such:

      https://www.howtogeek.com/830179/how-to-run-stable-diffusion-on-your-pc-to-generate-ai-images/

      “What Do You Need to Run Stable Diffusion on Your PC?

      Stable Diffusion won’t run on your phone, or most laptops, but it will run on the average gaming PC in 2022. Here are the requirements:

      A GPU with at least 6 gigabytes (GB) of VRAM
      This includes most modern NVIDIA GPUs
      10GB (ish) of storage space on your hard drive or solid-state drive
      The Miniconda3 installer
      The Stable Diffusion files from GitHub
      The Latest Checkpoints (Version 1.4, as of the time of writing, but 1.5 should be released soon)
      The Git Installer
      Windows 8, 10, or 11
      Stable Diffusion can also be run on Linux and macOS”

      Not a low end PC by any means but hardly as expensive as a MAC PRO. (Say $1-2,000).

      That said, the real gains ($$$) come from running the big apps on datacenters. Massive files and massive processors. Microsoft runs their apps on AZURE (at last count they had over 200 data centers spread over 32 countries). In fact, owning the datacenters is one of the ways they can make money off this tech.

      Until some clever folk in a garage figure out how to replicate the same functionality on cheaper hardware, “AI” startups have to pay *somebody* to use their cloud platform, be it Amazon, Microsoft, Google, or a smaller regional provider, which either raises price or eats into the profits.

      Owning the hardware (which operates at obscene margins–look at AWS) MS is paying themselves so they can bundle the cloud support with the consumer app “free”.

      (For various values of “free” of course.)

      Note that by not charging extra for the new features that are running on established products, MS doesn’t have to convince anybody to buy a new product. The new capability is just an added frature (and thus optional). They are using this exact strategy to great effect with online gaming, making it a feature of their Game Pass gaming subscription instead of a standalone product. Google’s STADIA had nominally better tech but failed to convince enough customers to (literally) buy into their business model. Now gone the way of a couple hundred other Google offerings.

      It’s not just a matter of having good new tech but also figuring out how to get prople to use it while making money off it. Not easy.

      And finally, the products that are getting the new features either run online or run locally but use online for some features (advanced help for OFFICE, for example). When it comes to online appos, though, properly designed apps only need enough bandwidth and latency to run the GUI. And are robust enough to survive glitches and cut-offs.

      One example I know of: A PC WORLD reviewer got an early look at MS DESIGNER last november and was trying it out when he got called away. When he returned the session had timed out but when he reloaded the page in it brought him back to exactly where he was. Zero work lost.

      Online apps understand that stuff happens (and not everybody has gigabit internet) and allow for a lot of disruptions.

      New times, new ways of living and working.

  7. As impressive as GPT4 in MS Office is, yesterday Microsoft made a much bigger announcement that isn’t getting anywhere the attention it deserves:

    https://www.windowscentral.com/microsoft/microsoft-brings-gpt-4-to-healthcare-professionals-with-dax-express

    The product is DRAGON AMBIENT eXperience Express, aka DAX Express. And it brings GPT4 to medical document management systems. Corporate hype:

    “The solution is called Dragon Ambient eXperience (DAX) Express, or DAX Express for short. It brings together conversation AI and OpenAI’s GPT-4 to automate clinical documentation.

    The goal of the tool is to reduce the burden on medical professionals created by recording patient data accurately. The medical field requires a massive amount of notes and documentation to ensure proper care, and automating some of those notes should reduce burnout, according to Microsoft.

    Microsoft outlined five outcomes that it believes will come as a result of today’s announcement:

    Relieving workforce burnout
    – Our solutions are proven to lead the industry in addressing this seemingly intractable problem – with physicians reporting up to 70% reduction in feelings of burnout and fatigue.

    Supporting specialty workflows
    – From surgeons to radiologists, our solutions analyze vast amounts of patient data, deliver workflow automation, facilitate reporting and communication, and provide AI insights that support more informed decision-making, planning and treatment—improving radiologist efficiency by 50% and reducing time-to-intervention by 74%.

    Improving adherence
    – By analyzing data to uncover findings, simplifying patient and physician communication, and providing comprehensive care plan tracking, our solutions deliver a 52% improvement in follow-up adherence.

    Increasing access to care
    – Our automated clinical documentation solutions give back time to clinicians who often choose to see more patients, adding five appointments per average clinic day – enabling clinicians to provide their best care to more people.

    Enhancing patient engagement
    – AI-powered chatbots are just one example of how AI is providing patients with quick and accessible information using built-in medical knowledge bases and triage protocols, which can trigger seamless handoff from bot interaction to a doctor, nurse or support agent. And, by delivering consistent and contextually relevant patient experiences, healthcare organizations are realizing 30% increases in patient self-service rates and 50% reductions in patient support costs.

    Similarly to Copilot, which Microsoft announced earlier this month, DAX Express is a tool that uses AI to help professionals without replacing them. Medical experts can review the notes generated by DAX Express before saving them, which is especially important within a medical environment.”

    Of course, corporate hype can promise anything and everything but it turns out that DAX Express is an existing product with a massive userbase. Not vaporware.

    Backstory:

    Two years ago, MS bought NUANCE COMMUNICATIONS for $20B, commonly known for their Dragon dictate software for PCs but quietly a *major* player in the health care field.

    With “major” defined as “Nuance solutions are currently used by more than 55% of physicians and 75% of radiologists in the U.S., and used in 77% of U.S. hospitals, according to Microsoft. Nuance’s Healthcare Cloud revenue experienced 37% year-over-year growth in fiscal year 2020, ending September 2020.”

    Nuance healthcare products include Dragon Ambient eXperience, Dragon Medical One and PowerScribe One for radiology reporting, all leading clinical speech recognition SaaS (Software as a Service) offerings built on Microsoft Azure. Nuance’s solutions work with electronic health records on clinical documentation.

    Now, put GPT4 in between the voice recognition software and the medical document systems doctors and hospitals already use. Healthcare SaaS is a $57B a year market and growing.

    (Healthcare isn’t all of Nuance’s value to MS but it is the biggest. “Beyond healthcare, Nuance provides AI expertise and customer engagement solutions across interactive voice response, virtual assistants and digital and biometric solutions to companies around the world across all industries. This combines with Microsoft’s cloud, including Azure, Teams and Dynamics 365, for customer engagement and security solutions.”)

    So, while Google and Facebook and a dozen startups dabble in Chatbots, MS has quietly put the same tech to work, without significant competitors, in a major growth industry.

    And that is far from the last of the uses of the tech.

    Today’s Microsoft is a very different beast than what it was at the turn of the century but no less dominant.

    • Another day, another GPT release from Microsoft.
      This time it’s Bing Image Creator:

      https://www.cnet.com/tech/services-and-software/microsofts-bing-adds-ai-image-creator-powered-by-dall-e/

      TL:DR – It’s a DALL-E derivative embedded in the browser.

      Not to be confused with the similar standalone Microsoft Designer announced last October as coming to Microsoft 365, though both will share tech.

      They seem to be serious about putting that $10B investment in OpenAI to work.

    • Two responses:

      As a patient, interacting with an AI chatbot is about at the bottom of the list of things I want to do, down there with getting a colonoscopy. AI chatbots are never for the benefit of the customer, and claims otherwise are flagrant bullshit.

      As a reader of medical records in my professional capacity, I see potential for yet more bloat. Twenty years ago, a routine emergency room visit resulted in a medical record of a few pages, or a bit more if there was radiology. Nowadays that same visit will produce about an order of magnitude more pages. This isn’t because there is more information recorded, but because the electronic format encourages, and often mandates, repetition. The challenge now is to find that one paragraph with what you are looking for. This is fine for me sitting at my desk, engaged in contemplative reading of medical records. I have seen stories of doctors missing important information in the moment because it is buried in blather. Perhaps they have managed to reverse this trend. Perhaps they will make it worse. My guess is the second one, but I would be happy to be proven wrong.

      • Well, one of the alleged strengths of GPT4 is summarizing long text.
        We’ll have to see what happens when the rubber meets the road.

Comments are closed.