Microsoft, like Apple and Google, is adding a bunch of kind of questionable AI features to their OS
The two big ones announced so far are Copilot and Recall
Copilot has been around for a while (since fall ‘23) in both Windows 10 and 11. It’s purported to be an AI helper, but in my experience it’s been kind of meh
In true Microsoft form, they’ve added Copilot branding to everything: GitHub, Edge, Bing, etc
Worth mentioning GitHub Copilot has been around since 2021 for code generation, so the branding ascended from that
It’s a signature Windows feature, to the point that they’re adding a key to the keyboard for it
Replaces the Right Menu key that Windows keyboards have had forever and no one ever uses
Win+C for folks who don’t buy new keyboards
Even Microsoft’s own pitch for Copilot is pretty anemic. Lots of stuff about how it can help you but not much that’s actually concrete
I was Googling for some real-life use cases for Copilot to see what people use it for and the first few search results were pretty much just machine generated crap, so maybe we’re just doomed
Computerworld’s 7 Ways to use Copilot Right story starts with how to turn it on, recommends it for reverse image search and writing a bad first draft, and then warns you that it lies and also that it’s less useful than it used to be before closing with a “here’s how to hide it”
Doesn’t seem to touch GPU usage at all, presumably all the compute is happening remotely?
I wonder what the economics are on that at the moment, and how sustainable this is
I’ve tested it on stuff like pretty basic time conversation formulas for Excel and doing the most basic data analysis stuff and it’s hasn’t really saved time
It’s OK at summarizing web articles or things like origins of holidays
Brad tried some complex and poorly documented questions like “When do Oracle’s ZFS patents expire?”
IP and Legal Ramifications of using Copilot
Two sides to this, data leakage and copyright ramifications
MS says Copilot isn’t trained on your data or prompts for Microsoft 365 customers, so data leakage shouldn’t be an issue
The basic idea here is that MS is good at siloing your data in non AI applications if you’ve got org perms set up right, and they’re carrying that over to the Copilot stuff too
Copyright is more complicated and varies based on region, but the early rulings in the US is that you can’t copyright AI generated work.
The secondary question of whether using AI generated work based on other artists copyrighted work is also unresolved in a legal sense. The Copilot image generator uses Dall-E, which was created using an enormous number of copyrighted images scraped from the web. As we record this, there hasn’t been a ruling about whether artists whose work was scraped to train Dall-E can attempt to claim copyright on derivative works.
I wouldn’t, for example, want to make a game that used a bunch of AI generated textures because of this
At launch, these are Qualcomm Snapdragon X Elite ARM-based laptops with CPU + GPU + NPU processors so a lot of the AI can run on the PC locally
Includes a new version of Prism to run x86 compatible software too
Copilot+ PCs include several new AI features, which we’ll get to, but the one everyone is talking about is Recall
The promise is straightforward, what if your computer remembered everything you saw or did with your PC and then it could help you find something you’ve forgotten using the context of the bits that you have.
Can you open the spreadsheet I was working on on Tuesday afternoon?
What was I working on before I found out that Trump was convicted?
I’m convinced that MS has some enormous user data signal that shows that this is something people actually want, since they’ve been working toward this for a long time (see also Task View and timeline feature (clipboard history as well))
Recall saves regular snapshots of what is on your PC’s screen at any given moment, and saves a UserActivity associated with that moment for each app that’s open (window state, what file is open, where in the file the user is, etc)
Copilot parses what’s on screen, so if you tell it “I was reading an article on a website with a dark background and there was a picture of a little girl in a yellow dress” it can theoretically find what article you were reading
It’s worth reiterating here that this comes in Windows 11 24H2, but will only work if you have a computer with a SnapDragon X Elite processor, which don’t start shipping until mid-June 2024.
You can also turn off Recall if you don’t want to burn the space on it or don’t like the privacy implications
Wait, Privacy implications?
Yeah, the recall screenshots don’t redact things like passwords, financial information, or private or sensitive info
You can easily disable Recall, either temporarily or permanently or on a per-application or per-website basis
They list support for all the major browsers out of the box (Edge, Chrome, Firefox, and Opera). APIs are available for other browsers
Disabling isn’t a registry key, like Copilot
All info is processed and stored locally (hence the need for new hardware)
Seems like this won’t be particularly useful if you use multiple PCs daily
Snapshots are encrypted by default, either using Device Encryption or Bitlocker
Can’t be accessed by other users on the same PC. Microsoft can’t see them either
There are some security holes here: it’s fine if your laptop is stolen and logged out, but if an attacker gains access to your computer remotely they’ll have access to this huge database of what you’ve been working on
Also seems like an enormous potential issue if your PC is seized by law enforcement/a government
Probably not, mainly because of the hardware concerns
Eventually this will roll out to other hardware, but you can disable it.
FWIW, I don’t think this is quite as bad as the internet made it seem
Where do we land?
Will: Generally speaking, I’m not interested in running this.
Brad: I already disable all the activity history/timeline features, so I think I’ll pass.
Ways MS could have improved this:
MS’s documentation implies that the snapshots are encrypted because encryption is on by default for this class of PC, but it’s unclear to me if they’re requiring encryption to turn on Recall.
Running this without disk encryption on is a problem
Generally feels like more of a security problem than a privacy problem
Likewise, MS should create a default blocklist of applications that contain sensitive information that should be added to the per-application or per-site blocklist. Banks, password managers, etc
Turning it off by default and making it explicitly opt-in per-app would be even better
Honestly, if MS had let people easily disable Copilot instead of making it a Registry hack that hides Copilot but makes it unclear if it’s actually disabled or off the Recall thing probably wouldn’t have blown up as much