Disrupting supply chain attacks on NPM and GitHub Actions

(github.blog)

88 points | by nyku 2 days ago ago

54 comments

  • alpineman a day ago ago

    >> high-impact npm accounts are now put into a read-only mode for 72 hours when they change their email or use a 2FA recovery code. This delay allows maintainers time to respond and recover the account before their account can be used to start an attack.

    'what time shall we put here?'

    'what's the longest hangover you ever had?'

    'let's put 72 hours'

    • Waterluvian a day ago ago

      A weekend plus a day to deal with things seems like a decent minimum duration when it’s somewhat arbitrary what the right number is.

      • datakan a day ago ago

        I don't think so at all. People go on vacation all the time. This should be 30 days not 3.

        • Waterluvian a day ago ago

          That's the usual runaway problem, right? Why not 60 then? People go on sabbaticals! etc etc.

          I feel like the only way to be wrong for this class of problem is to believe that there's a singular right answer. Just pick something reasonable (like how weekends are a fairly common thing, so don't make it shorter than 48 hours). Start there, then see how much of an issue persists. No matter what, at scale you'll find someone complaining that the number is too little, and people complaining that it's too much. Eventually you just have to tell the complainers to deal with it.

          • datakan a day ago ago

            There is a big difference between 3 days when someone may be sick or traveling and not looking and 30 which aligns better with 99% of scenarios this may be an issue with. 3 days is absurd. Even password apps with kill switches will default to 7 days.

            Cyber attacks will frequently happen on a Friday just for this scenario, they anticipate no ones looking over the weekend.

            • tredre3 a day ago ago

              And 365 days probably aligns with 100% of scenarios. It doesn't mean it's a good idea, because the longer the delay is the more downsides you also get.

              Blocking a maintainer from pushing updates for a full month after changing their e-mail would be obviously absurd. So Github would have to add a way of enabling maintainers to super-uper-duper confirm the change and cut the delay short. Which is likely not currently possible, hence starting with a shorter delay.

        • Normal_gaussian a day ago ago

          It sounds like we should plot 'count of compromised repos' against 'time taken for maintainer to initiate a resolution', and pick something just after the inflection point.

        • normie3000 a day ago ago

          It's often not the maintainer that removes a poisoned release, it's the npm security team/tools. So unless the whole team takes a month off, we're all safe.

    • lrvick a day ago ago

      I paid $8 for the recently expired email domain of the sole author of NPM package "foreach", so then control to ship any code I wanted to 70k companies was just a support ticket or password reset away. 72 hours would not make a difference here.

      Talked to NPM about this when it went viral, and once again all they could say was enabling package signing, even optionally, would discourage inexperienced people from contributing packages as they would be pressured to learn basic security and key management.

      Sorry, if you are unwilling to take 10 minutes to learn how to sign your code, you have no business maintaining software releases millions of people depend on. Full stop.

      But NPM will not even make it -optional- and rejects all PRs to implement this going back a decade. So no one feels pressured into basic security. Cool.

      • e40 16 hours ago ago

        > Sorry, if you are unwilling to take 10 minutes to learn how to sign your code, you have no business maintaining software releases millions of people depend on. Full stop.

        The crux of the problem is right here.

      • undefined a day ago ago
        [deleted]
      • inigyou 20 hours ago ago

        Who chose to depend on "foreach"? Millions of ill-advised people did. The author did not choose that.

        • lrvick 3 hours ago ago

          If an author is not willing to do responsible release engineering they should lose their publishing ability and have their project marked as unmaintained. This would force people to migrate to better maintained code, or find responsible volunteer maintainers to maintain it. Everyone wins.

          Even when you drop off food at a food pantry for consumption by strangers there is a basic expectation it is not spoiled or dangerous.

  • summarybot a day ago ago

    Github owns NPM. Github has unprecedented access to code analysis tools. Github can run static analysis on nearly everything. Introducing a cooldown period seems like the lowest tech solution to a technological problem I have seen in a longwhile.

    • insanitybit a day ago ago

      Cooldowns have the benefit of being straightforward. It doesn't rely on updating rules, scanning, keeping up with obfuscation techniques, etc. It just stops installs for a period of time after publish.

      I think that cooldowns are mostly dumb but they're very cheap to deploy and they do give other approaches (like scanners) more time to adapt/ do their job.

      • summarybot a day ago ago

        In my opinion, things like scanners should not need to exist if npm's parent company, GitHub, were to take responsibility for what was published to npm. If Walmart sells you an item that ends up being a handgrenade there would be hell to pay. In this example, Walmart cannot simply say that Walmart is not in charge of what goes into the item and therefore "buyer beware." Yet, this is precisely what GitHub does with every npm package.

        One reasonable approach is to increase the scrutiny for new releases based on how many active downloads of the package there are. So for new packages, maybe there will be not a ton of scrutiny, but for things that are downloaded by millions of users, like Axios, there would be significantly more scrutiny per release.

        The epidemic of "not my job" and "[we] don't get paid to do that" is a large contributor to why the world is so messed up. I don't know what to call this phenomenon. In the Hawaiian language there is a word Kuleana which means "sacred responsibility" or "sacred task." If more people took on their Kuleana joyfully we would be inviting the ideal [realm] more rapidly.

        • insanitybit a day ago ago

          Walmart and Github are so different. Walmart doesn't let anyone come in and set up a booth to sell things, Github does. Thinking that Github should take responsibility for what's hosted on it is... well, radical, to say the least.

          What you're suggesting is a radical departure from how free software has been distributed historically. That goes well beyond Github and NPM.

          • doubled112 14 hours ago ago

            Would the metaphor have worked better if it was Amazon?

            Walmart does allow other stores to sell through their website.

            • insanitybit 12 hours ago ago

              I would just drop the metaphor entirely since I don't think they're helpful. If you want to advocate for liability, I'd do so directly

          • summarybot 10 hours ago ago

            You say "free software" but npm is no longer a small open-source project with a federated rulership, it is owned and operated by a for-profit technological giant. There was a radical departure already, when Github acquired npm. It sounds like your stance is "if something is given away for free, consumer protections are not necessary." But if I get poisoned by free samples given away at a grocery store, who is on the hook? Buyer beware? The onus is on the organization that distributes the food, in the case of the grocery store. It is unreasonable for every person to become a food chemist to be able to ascertain the safety of every single food product. Rather, we as a society agree to put that onus on the distribution. This is no different.

            • insanitybit 9 hours ago ago

              This isn't about npm, it's about who publishes to npm. This isn't a grocery story.

        • inigyou 14 hours ago ago

          You're suggesting that only stuff billionaires like should be allowed on GitHub or npm. No Monero libraries, for example, or YouTube downloaders.

    • lazyasciiart a day ago ago

      It's certainly the lowest tech piece of their announcements over the last day, as far as I can tell.

      https://github.blog/changelog/2026-07-28-npm-publish-time-ma...

    • lrvick a day ago ago

      That is because Github is not a thing anymore. We are talking about Microsoft.

  • zzo38computer a day ago ago

    My only use of GitHub Actions specifies permissions that only allow it to write to the issue tracker, so if the program it executes (which is the "gh" program) is malicious then the only damage it can cause is to the issue tracker and not to the repository itself (unless GitHub Actions itself is compromised). Furthermore, there are no private files (I do not host private files on remote services such as GitHub), so they can't steal my files either.

    I think one thing that would help is to support mutual TLS, with X.509 certificate chains for authentication. This is more secure than personal access tokens and allows you to issue them to yourself and others without needing to login first, including to specify whatever permissions you want to set (as a subset of the permissions of the issuer) and to make them expire. (You could also store the private keys of one certificate on a separate computer not connected to the internet, and use that to issue another one to yourself to use that one instead, and can use a passworded private key as well (an alternative of 2FA), so you can recover even if your certificate you are using does become compromised, or something deletes it (intentionally or not), and can revoke the ones that are compromised.)

    Mutual TLS (and even ordinary TLS) won't solve everything, but in combination with other things, including ability to restrict access to what is needed, and avoiding needing too many dependencies (since, programs that need too many dependencies is also one thing that can cause many problems), and many other things, it can be an additional step to help with security.

    • acdha 14 hours ago ago

      > I think one thing that would help is to support mutual TLS, with X.509 certificate chains for authentication. This is more secure than personal access tokens

      This adds an large amount of overhead and complexity (i.e. you now need to manage a revocation list) but you didn’t really explain any benefit which isn’t already present except for the possibility of restricting access to the private key. That can have some benefits — e.g. it’d be nice if you could say that releases can only be signed by a particular key stored on a Yubikey with a physical presence check – but I think most of the benefits come not from using x.509 but rather from fine-grained permissions.

      Most of the attacks we’re seeing are escalation from an initial minor compromise where an attacker compromised something which had far greater permissions than needed for the task the attacker first exploited, as well as the broken design of GitHub Actions with mutable tags. I don’t think TLS gives us enough to take effort away from working on those.

      • zzo38computer 5 hours ago ago

        I agree that benefits come from fine-grained permissions; however, for the fine permissions of user authentication (and API authentication), I think it would help to use X.509 certificate chains. The certificate can include a extension for specifying the permissions that it grants (a permission will only be granted if all certificates in the chain grant that permission). This avoids needing to login or use 2FA in order to set up new API keys or access tokens, and prevents problems with misauthentication with the wrong server (personal access tokens don't help much compared with passwords, but X.509 does help). Merely using X.509 won't cause it to have fine-grained permissions, but they can be combined in the way that I described. (This use of X.509 can also be used to allow someone else to operate on your behalf with limited permissions; in this case, a client would issue a certificate to a server, and that server would then also be a client to GitHub.)

        If there is a broken design of GitHub Actions, that is another issue, and neither TLS nor X.509 is the issue with that. (Also, GitHub Actions could hopefully be made to allow finer permissions if the existing permissions are not fine enough; that is not related to X.509 at all.)

        Avoiding too many dependencies, is another thing to do.

        And, signed releases is another thing to do, and is helpful for additional reasons (unrelated to authentication with GitHub, but useful for knowing that you published it rather than someone else (without having to trust GitHub with it)). The key used to sign the release might or might not match that in the X.509 certificate used to authenticate with TLS; there are some benefits if it does (such as if you have published your certificates that someone else can check, or your mention of Yubikey), but it doesn't have to (e.g. because you have multiple certificates, because multiple people are publishing releases, because you have some other reason to not use the same certificate for the other purpose, etc).

  • skipants a day ago ago

    One thing I don't quite get is how trusted publishing is supposed to be more secure. It still allows publishing if they pwn your workflow.

    Is it purely more secure because they can't exfiltrate your secret keys to publish again?

    I feel like if your workflow gets pwned you'd be rotating your keys anyways, so I'm not sure if the vendor lock-in is worth it.

    • pimterry a day ago ago

      Trusted staged publishing helps a lot: you have to independently pwn the workflow _and_ then complete a separate 2FA flow as a maintainer. The workflow never sees any keys that can publish independently.

    • flyingshelf a day ago ago

      Keys can be reused from anywhere. Trusted publishing means the attacker must trigger the specific workflow on GitHub, which is more difficult and leaves trace of actions on GitHub itself.

    • insanitybit a day ago ago

      Trusted Publishing pins the publish to the workflow. The next step is to isolate "run untrusted code (`npm build`)" from "publish" in your workflows. Now you have two properties - the publish can only happen from one place and the publish is isolated from untrusted code execution.

  • MSkill1 a day ago ago

    I literally am reading this after setting GitHub actions five minutes ago to generate the APK version of my cross platform project for the first time. Great :\

  • undefined a day ago ago
    [deleted]
  • hncsiocp9x a day ago ago

    Been quietly thinking this for years

  • effnorwood a day ago ago

    Hey Siri, what is a local repo?

  • pluto_modadic a day ago ago

    the bare minimum award, for the only language and only registry where this regularly happens.

    • hinkley a day ago ago

      At this rate I'm going to have "npm is by far the worst thing about NodeJS" printed on my gravestone.

      • TacticalCoder a day ago ago

        > "npm is by far the worst thing about NodeJS"

        I'll come and add: "after JavaScript".

  • lrvick a day ago ago

    It is fascinating how far people will go to do anything else at all other than having authors cryptographically sign their packages like every Linux package manager that matters since 1996.

    Linux package managers like debian/apt also ship hundreds of signed NPM packages. In other words the unpaid Linux community has once again solved a problem Microsoft fundamentally failed to. Some things never change.

    • acdha 14 hours ago ago

      What keeps Linux distributions safer is that fewer people are able to push updates and there’s a time delay. The compromises are because people exploit release pipelines and if your build infrastructure is compromised it’ll merrily sign the compromised packages as well.

      • lrvick 3 hours ago ago

        Which is why automated signatures in release pipelines are usually meaningless. Authors must sign their code on their own hardware before it is published so we can tell when the signing key changes and pay extra special attention to that, and maybe a longer delay.

  • torment-nexus a day ago ago

    [flagged]

  • receiptincar 17 hours ago ago

    [dead]

  • DiabloD3 2 days ago ago

    [flagged]

    • theF00l 2 days ago ago

      The job can require you to

      • DiabloD3 2 days ago ago

        [flagged]

        • UqWBcuFx6NV4r a day ago ago

          These comments should be removed r from HN, because they in no way actually add to the conversation. They aren’t intelligent, they aren’t insightful, they aren’t actionable, and they don’t invite a genuine reply. All you’re saying is that you happen to not use these technologies yourself – something that I’m sure is only by happenstance – and that you feel superior for it.

          This is a blog post by GitHub. what are you suggesting that these employees do? Simply ignore that they exist? Regardless of whether or not you use them, they still exist.

          • DiabloD3 a day ago ago

            It very much is actionable.

            You might not know this, but the world of software development existed before NPM and Github, and it continues to exist after them. You are not beholden to either of them.

          • anon48293 a day ago ago

            > This is a blog post by GitHub. what are you suggesting that these employees do? Simply ignore that they exist?

            The buggy, insecure feature that is GitHub actions? Yes, preferably so.

          • sieabahlpark a day ago ago

            [dead]

        • x86a 2 days ago ago

          This is such a myopic take

        • baby_souffle a day ago ago

          GitHub actions doesn’t really make you a js shop

    • rho138 2 days ago ago

      > Opting out of toxic ecosystems is a valid option

      Quick, everyone break out the pitchforks for a valid analysis of a game! /s