MaliciousCorgi: AI Extensions send your code to China

(koi.ai)

78 points | by tatersolid 6 hours ago ago

74 comments

  • mat_epice 5 hours ago ago

    Sure, AI tools can do this. However, VS Code is the platform. Why aren't more people worried about running arbitrary VS Code extension that can do the same thing, AI or not?

    • g947o 5 hours ago ago

      As an VSCode extension author, I am always terrified by the amount of power I have.

      It is a shame that the team never prioritized extension permission issues [0] despite their big boss said security is the top priority [1]. All they have is "workspace trust" and various other marginally useful security measures.

      I don't install a VSCode extension unless it is either official or well known and audited and I have to use it. I keep most of them disabled by default unless I need something for a project. (Even if you don't care about security, it's good for VSCode performance. I'll save that story for another day.)

      [0] https://github.com/microsoft/vscode/issues/52116

      [1] https://blogs.microsoft.com/blog/2024/05/03/prioritizing-sec...

      • yomismoaqui 4 hours ago ago

        When some minor extension that I have installed on VSCode updates (like parens colorizing and the like) I think what could happend if the author sells it to some bad actor (or decides to push some weird code in an update).

        So I started uninstalling some icon themes and less used extensions that I installed on a whim years ago.

        I implicitly trust extensions by Google, Microsoft and the like, but the less known published make me nervous.

        • thedanbob 4 hours ago ago

          It doesn't even have to be malicious. I used a certain syntax highlighting theme for years, when out of nowhere the author pushed an update that rearranged all the colors. It was extremely disorienting. I forked the extension and reverted the change, so I know that one at least won't change out from under me anymore.

          • freedomben 3 hours ago ago

            This is the thing I hate the most about "automatic updates" in general. I've disabled them and gone back to updating manually because the constant unexpected and unwanted UI changes finally broke a part of my soul. Unfortunately that is something that can't be done on the web, where major UI changes can be rolled out right in the middle of a session on you.

      • fc417fc802 3 hours ago ago

        > As an VSCode extension author, I am always terrified by the amount of power I have.

        Meanwhile random FOSS projects be like "please sudo curl bash to install the prebuilt binaries".

        • godelski 3 hours ago ago

          Most don't even use functions when writing those scripts and it can straight up fuck your system on accident. It's very unlikely but it can happen and a malicious actor can trigger it on purpose.

          But this is true about lots of code. We have this notion of "it works, therefore there's no problem" which is just bad engineering. Just because you don't know there's a problem doesn't mean there isn't. Just because it passes the tests doesn't mean you have test coverage.

        • dirkc 3 hours ago ago

          This is one of my pet peeves! No one should normalize the idea of piping curl output to bash.

        • shermantanktop 3 hours ago ago

          How did that even get started? It’s not like downloading a zip or tar file is so terribly taxing.

          • godelski 3 hours ago ago

            Convenience, mostly.

              curl -L "foo.sh" -o foo.sh && bash foo.sh
            
            Is just more characters. But you should do it simply because a poorly written bash script can accidentally mess you up when streaming.

            Why sudo though?

            I honestly think it's stupidity. Most people really don't know you can build programs to the user and don't need system privileges. I think everyone is just so used to installing from package managers and doing `sudo make install` that they forgot programs only need to be in $PATH and not /usr/bin

            • esafak 3 hours ago ago

              How is that any safer?

              • godelski 2 hours ago ago

                First off, I never used sudo...

                Second off, you're not steaming into bash

                Third, you gotta read between the lines a little. I used some convenience considering my audience is programmers. Don't use && or shove && `less foo.sh` in the middle. There's a million options here

              • fc417fc802 2 hours ago ago

                Don't take the example overly literally. Saving to file means you can read it before executing it.

                That aside, it protects you from this gaping hole of an exploit mechanism. https://news.ycombinator.com/item?id=17636792

        • g947o 3 hours ago ago

          I agree. Sadly most of us aren't going to build from source, and some tools don't really work without sudo. (Did I mention VSCode? On Linux you get a .deb file. Yeah.)

          • lo_zamoyski 2 hours ago ago

            In practice, building from source is not going to fix the problem. Nobody reads the source code of projects they download and compile themselves, certainly not for larger projects. It also takes a long time to compile larger projects. So, realistically, these rarely happen.

            Of course, the one advantage of having source is that it is easier to run things like SAST tools against source, but how many people do that in practice? How integrated is that with package systems? And when package maintainers might provide hashes of what they ostensibly checked, you still need trust.

            So we need a combination of static analysis tools that are integrated properly to produce trusted binaries, and you need earned trust and authority. Hyperindividualist self-reliance is, at the very minimum, impractical. And with authority, we know whose job it is to care for the quality of software and therefore whom to hang.

            • fc417fc802 2 hours ago ago

              > building from source is not going to fix the problem. Nobody reads the source code of projects they download and compile themselves

              However commits tend to be much easier to trace at a later date than arbitrary binaries so attackers will be less inclined to go that route. Once committed it's there forever unless you can somehow get everyone to censor it from their own copies for an unrelated reason. Consider that the xz compromise involved downloading the payload later.

              My policy is to either obtain binaries from a major distro or to build from a clean commit in a network isolated environment. If I can't go one of those routes it's almost always a hard pass for me.

    • tormeh 5 hours ago ago

      The situation is absolutely insane, but it's also productive, but real security would slow everything down a lot. The moment you ask some corporate bureaucrat to put their signature down on a piece of paper saying that such and such dev tool is approved for use, they're going to block everything to avoid the responsibility implied by their approval. I can't really come up with a system that both works and is secure. The only exception is signing up for an integrated environment where Microsoft or Apple provides the OS, compiler, and editor. Oops - Apple doesn't sell servers, so only Microsoft offers this. Hope you like C#.

      In theory you can mix and match, but in practice most bureaucrats will insist on single-sourcing.

      • rapind 5 hours ago ago

        Linux development has a blueprint they could follow. Like the principle of least privilege. These aren’t cutting edge concepts.

        Also I’m not sure the tradeoffs of adding security to an editor are that big of a deal. Are we really seeing revolutionary stuff here? Every now and then I check out VS Code only to realize Vim is still 10x better.

        • fc417fc802 3 hours ago ago

          Vim is hardly secure either. Extensions in both provide for arbitrary code execution.

      • not_ai 5 hours ago ago

        At the company I work for they locked down installing extensions through the marketplace. Some are available but most are not and there is a process to get them reviews and approved. You might be able to side load them still but I haven’t cared enough to want to try.

        They did the same with Chrome extensions.

    • mentalgear 4 hours ago ago

      Same thing for browser extensions: a simple browser extension (e.g. web dark mode), can read all your password fields. It's crazy that there are no proper permission scopes in any major browsers ! It would have been so easy to make password / email fields exempt from browser extensions unless they ask for the permission.

      • sheept 2 hours ago ago

        I do not think it'd be "so easy" to separate password input access into a separate permission because it'd only open up a can of worms. There's so many ways to read a password input's value, from listening to key events to monkey patching `fetch`, that it's not worth playing whack-a-mole just to provide users a false sense of security

        I'm also skeptical that even a dark mode extension would be simple considering how varied web pages can be

      • vunderba 3 hours ago ago

        Pro tip: I’ve seen plenty of dedicated extensions that could have just been simple snippet equivalents in Tampermonkey - an extension that lets you run JS limited to wildcarded websites.

        I've used it to inject download links on sites, autoclose modals, etc. You can either write them yourself, or review other people before installing them.

        It’s not a perfect solution, but at least it reduces the surface area to a single extension.

        FYI: Just set Script Updates to Never.

        https://github.com/Tampermonkey/tampermonkey

      • fc417fc802 3 hours ago ago

        In your example wouldn't that leave the email and password fields the wrong color? I agree with the principle though. Most extensions don't need to access everything.

    • dirkc 3 hours ago ago

      Installing any 3rd party dev dependency without sandboxing should terrify you. These supply chain attacks are not hypothetical.

      Trusting other devs to not write malicious code has led to a surprisingly small number of incidents so far, but I don't think this will extrapolate into the future.

      With more lines of code being auto-written without deliberate intent or review from an accountable author, things can only get worse!

    • larodi 3 hours ago ago

      I am (am worried) and recently stopped adding extensions by just the random anon. Also I take time to sanitise foreign (to my knowledge) gh repos using Claude code.

      • freedomben 3 hours ago ago

        As an aside, claude and codex (and probably gemini) are pretty good at doing that. I've now done it with several repos and they are pretty good at finding stuff. In one case codex found an obscure way to reach around the authentication in one of our services. This is a great use case for LLMs IMHO

        They are (of course) not foolproof and very well may miss something, so people need to evaluate their own risk/reward tradeoff with these extensions, even after reviewing them with AI, but I think they are pretty useful.

    • zukzuk 5 hours ago ago

      Yes, exactly. The lack of any sort of permission controls for extensions in VS Code gives me the creeps

  • ecshafer 3 hours ago ago

    I don't really get how VSCode got so popular. You can use a language server perfectly easily with Vim, Emacs, Helix, Sublime, etc. You can customize basically everything in those editors, syntax, etc. You can just alias console commands for all of your build tools with some custom scripts if you need more complex build commands routinely. The git terminal tool works better than any VScode option. And VSCode is slower than all of those.

    We already have so many good fast secure polygot customizable text editors. Why run one through Chrome and fill it with extensions for everything that will have arbitrary access to everything?

    • guyomes 20 minutes ago ago

      > You can use a language server perfectly easily with Vim, Emacs, Helix, Sublime, etc.

      By the way, the language server protocol was originally developed for VSCode [1]. The popularity of LSP in other editors might have contributed to advertise VSCode.

      [1]: https://en.wikipedia.org/wiki/Language_Server_Protocol

    • kouteiheika 3 hours ago ago

      > I don't really get how VSCode got so popular. You can use a language server perfectly easily with Vim, Emacs, Helix, Sublime, etc.

      You open it. It just works. And the learning curve is smooth.

      Compare this to Vim where, if it's the first time you're opening it, you are forced to kill the process because you don't even know how to quit it, never mind actually do any productive work.

      • godelski 2 hours ago ago

          > Compare this to Vim where, if it's the first time you're opening it, you are forced to kill the process because
        
        Because you can't read

        I'm serious. Open a blank file by typing `vim` into the terminal. Don't press anything, just look at the screen.

        I'm sorry, but reading docs, or just reading, shouldn't be considered a significant barrier to entry.

        • SoftTalker 2 hours ago ago

          How long have you been developing code for others to use?

          Rule number one is that users don't read documentation.

          • godelski 4 minutes ago ago

            Who are you making software for? That matters.

            General public? Yeah, assume idiotic.

            Developers? If you can't read docs then you can't do your job.

            I don't have much confidence in LLMs replacing devs, but if the dev is so arrogant that they think they're too good for documentation and if they have to read is too complicated, then yeah I believe AI can already replace that person. They were just a warm body, not a dev.

        • Daishiman an hour ago ago

          You're not going to be making successful customer-facing software with that attitude.

          • godelski 3 minutes ago ago

            This isn't the flex you think it is. You're really illustrating my point... being able to understand context and make inferences is part of basic literacy skills

    • valicord 3 hours ago ago

      I'm sure you're being sincere here but this really reads like that famous HN comment about "who needs Dropbox when ftp exists". The reason vscode is popular is not because it does something impossible to do otherwise, but because it does those things out of the box with a friendly UI.

      • ecshafer 3 hours ago ago

        But we are programmers. I think there is a difference between expect John Accountant who doesn't trust his computer to set up RSync and a server, and a Programmer who is already going to spend a good chunk of their day in a terminal windiow anyways.

        • pdntspa 2 hours ago ago

          Dude I'm tired. Tired of having to learn some stupid new UI paradigm just because.

          I really really wish there was ONE standard orthodoxy with regards to UI and how programs work and how we get around them.

          Instead we have these clowns constantly inventing new ones. I love learning things and tweaking things but I have limited bandwidth and I am so over micromanaging my PC

          For the record I know and love vi. But as I get older I find myself yearning more for the cathedral than the bazaar

      • godelski 2 hours ago ago

        I think you missed the issue with the Dropbox comment. Look back at it. He talks about using Linux, FTP, curlftps, SVN, and doing a network mount.

        The comment isn't actually even talking about providing the same service, so they mention emailing themselves files and usb drives.

        The problem was there was a big technical hurdle to locally network mount a file system. Especially across OSes. It's even harder to do it non locally. Sure, it's not hard if you're familiar with that stuff. Sure, it's not hard to learn if you're comfortable in the terminal. Sure, today you can use rclone. BUT that's not a tool my grandma can use.

        On the other hand, we're not talking about tools my grandma can use. We're talking about tools a programmer can use.

        https://news.ycombinator.com/item?id=9224

    • mizuki_akiyama 3 hours ago ago

      Mostly because installing and setting up VSCode takes one click

      • ecshafer 3 hours ago ago

        As opposed to 'sudo apt install vim'? or 'sudo dnf install vim'? Same for emacs. sublime you might need to add a package repository.

        • godelski 2 hours ago ago

          Or even, and I hate that I'm saying this, install vim from the apt store?

      • bauerd 3 hours ago ago

        brew install emacs

        • shermantanktop 3 hours ago ago

          I heard there’s an emacs macro for that.

    • qwertytyyuu 3 hours ago ago

      Vs code was also arguably prettier than the rest in release

    • guestbest an hour ago ago

      I decided to switch after notepad++ got hacked.

    • zzzeek 3 hours ago ago

      I used Sublime for years and VSCode is vastly better (the breaking straw was how they'd silo off critical bug fixes in new versions that were pay-only, upon finding vscode I felt silly for not switching sooner, it was so much easier to use and more powerful). Still use vim daily but not as a general IDE, memorizing decades of weird character commands and directives is not a great use of my time.

      my favorite VSCode feature is the SSH remote working feature. VSCode gives me the full editing / console / Claude environment on my local workstation, where all files, shells, and yes Claude as well run on a company lab machine over the VPN. Props to the collaborative working feature where several people can all share the same VSCode editor session on their individual workstations.

      Vim can do the above two things if you run as a terminal app with tmux. Sublime could do it if you shared the editor via X or Waypipe (well not the second feature). But VSCode integrates it directly in the app and it's a much better experience.

      • godelski 2 hours ago ago

          > But VSCode integrates it directly in the app and it's a much better experience.
        
        Not for the admin of the server who has a bunch of idle vscode sessions. Sure, cli users do it too with tmux but the resource consumption is vastly different
  • flufluflufluffy an hour ago ago

    > We install them without a second thought. They're in the official marketplace. They have thousands of reviews. They work. So we grant them access to our workspaces, our files, our keystrokes - and assume they're only using that access to help us code.

    Who is this “we”? I don’t, and don’t know anybody else who does this.

    Also, was this article itself written by an AI assistant? If the author is that carefree regarding these extensions, I guess probably.

  • darepublic 5 hours ago ago

    It's hard for me to fathom that there are capable devs who would pollute their ide with this crap in the first place, malicious or not

  • jszymborski 4 hours ago ago

    Between this and the notepad++ thing... I got to start running programmes with firejail or something.

  • apt-apt-apt-apt 5 hours ago ago

    This seems expected, when you install free, random software, especially from sources known for surveillance/malware/crime.

  • DeepSeaTortoise 5 hours ago ago

    I'm honestly surprised this issue in general didn't cause nearly every company to immediately ban all AI.

    Why do these companies put so much effort into fighting right to repair to avoid IP leaks any halfway serious company could reverse engineer in a week, but on the other hand encourage their employees to vibe all company secrets into the cloud?

    • graemep 5 hours ago ago

      > Why do these companies put so much effort into fighting right to repair to avoid IP leak

      Only if you believe they are truthful about the reason for fighting right to repair. I think the reason for fighting right to repair is to reduce the time before a replacement purchase is required.

      > but on the other hand encourage their employees to vibe all company secrets into the cloud?

      Lots of companies do ban or restrict usage of LLMs etc.

    • embedding-shape 5 hours ago ago

      It's a bit trite, but the answers are: 1) money 2) money

      Can't repair your own stuff and either need to use authorized repair shop or buy new? The company gets more money.

      Force your developers to forgo quality in efforts to produce more cruft in less time? The company gets more money.

      Of course, only considering short-term, long-term they'll lose money, but at that point all the executives and managers already got their bonuses and probably moved on to doing the same in some other company.

    • direwolf20 5 hours ago ago

      Companies aren't interested in hypotheticals, nobody is paid to care, and most code isn't that valuable anyway.

      • godelski 2 hours ago ago

        Code isn't valuable? And here I thought software was a multi trillion dollar business

    • pixl97 4 hours ago ago

      Most large companies have their CI/CD behind a proxy with an allow list and require internal approval for tools and extensions. So there is that.

    • wxre 5 hours ago ago

      Uhh a lot of companies did and are strict on what AI tools are allowed.

      The main thing I had to wait on for a long time was support for preventing 3rd party code from being plagiarized since our code base was intermingled with partnered companies.

    • fragmede 5 hours ago ago

      Contracts.

  • SanjayMehta 4 hours ago ago

    > Not just what you're actively working on. Every file you glance at. Every character you type. Captured and transmitted.

    Even this reads like an AI extension wrote it.

  • deafpolygon 4 hours ago ago

    This is one of the many reasons why I don’t use VS Code, or use any “helpful” AI plugins (or any plugins really).

    You all can take vim out of my cold dead hands.

    • evilduck 4 hours ago ago

      What's stopping a vim plugin from doing similar data exfiltration? Tons of people blindly install LazyVim, Spacevim, or other vim tooling and choose a bunch of similar things.

      • deafpolygon 2 hours ago ago

        In general? Nothing, really.

        I think it’s the culture behind the (neo)vim community is a bit more technical, and are quite quicker to sound the alarm if anyone tries something shady.

        But, in any event, I hand-roll my own config and every plugin I install is inspected by me. When I pull changes, I check the diffs for anything shady. If a plugin is simple enough, I will just integrate it into my own stuff.

  • bestouff 5 hours ago ago

    Well, AI already sends your code to US so ...

    • october8140 5 hours ago ago

      Most AI only sends a limited context. These are sending all files it can access as well as all edits.

      • pcwelder 5 hours ago ago

        > These are sending all files it can access

        TBF, Cursor's code indexing works the same way, it has to send all workspace files to their servers.

        Auto-completion systems need previous edits to suggest next edits so no surprises their either.

    • raverbashing 5 hours ago ago

      Yes because there's no difference between a voluntary service with limited context needs and a malicious extension

      • otabdeveloper4 4 hours ago ago

        Cursor is a malicious extension though, and nobody seems to care.

    • y-curious 4 hours ago ago

      “I donate money to animal shelters”

      “Oh that’s cool, I already donate to my local neo nazi group. We are both philanthropists!”

      Nothing makes me go from apolitical to a red blooded American faster than seeing someone make a stupid false equivalency about the US on this forum

      • Quothling 4 hours ago ago

        It wasn't China that recently threatened with an invasion of Greenland (I'm Danish). That being said, it's not like you would want your data to go to either countries.

      • mentalgear 4 hours ago ago

        You did hear about Snowden and the massive NSA data collection ? That was almost 20 years ago, think about what it's like now: they probably use our data to run an elaborate simulation of everyone.

      • otabdeveloper4 4 hours ago ago

        Shocking news: not everybody here is from the US.

        In fact, many even are from "hostile countries" that are "enemies of democracy".

        What's more, some of those people aren't aligned with US interests and aren't willing to put their lives on the line for CIA operations!