GitHub Down Again?

(github.com)

116 points | by ridowk 8 hours ago ago

122 comments

  • mat0 7 hours ago ago

    Yes. It is incredibly frustrating that everything is hosted on GitHub nowadays. I wanted to install Bazzite (a Linux distro!) and I cannot download their ISO, I cannot download their recommended partition tool, I cannot even download Rufus to start formatting some drives. And then you go and check their status page and you get the "all good nothing wrong here". So annoying

    • Flere-Imsaho 6 hours ago ago

      It's frustrating that engineers, who should know better, design systems in such a way as to have single points of failure. We have the technology and knowledge to have reliable systems, but here we are.

      • isolay 6 hours ago ago

        Fashion and religion are well-known overrides for reason. And people who call themselves engineers are not immune.

      • txrx0000 4 hours ago ago

        Alternatives are being built. Consider migrating to Nostr repos and host a relay if you can: https://ngit.dev/ https://gitworkshop.dev/danconwaydev.com/ngit

        The real underlying problem is hardware manufacturing (explained further here: https://news.ycombinator.com/item?id=49690354), which exerts centralizing pressure all the way up the stack. We should have pursued computer manufacturing in a print shops and printers kind of way instead of building large fabs, but that would have required foresight decades prior.

      • teekert 5 hours ago ago

        Is it a single point of failure though? Or just a convenient end point?

    • Gander5739 7 hours ago ago
    • ashishb 6 hours ago ago

      The real lock-in is stars, and not reliability

      https://ashishb.net/tech/github-stars/

      • serf 5 hours ago ago

        as someone who has been stuck in github for a hundred years and pays essentially zero attention to stars, eh.

        for me the lock in is cheap data reliability and low chance of loss. downtime problems blow, but if they lost any of my data for any reason other than a eula/tos violation or some such then i'd run.

        • jareklupinski 5 hours ago ago

          personally i just like how if someone questions my release, theyre welcome to check out my pipeline

        • ashishb 5 hours ago ago

          Everybody claims to not care about GitHub stars even though there are tons of websites for buying them.

    • fock 7 hours ago ago

      "well, who would not have the internet!"

      (the anduril-type folks using nix for high-assurance setups!)

  • renehsz 7 hours ago ago

    I'm surprised they even reach 1 nine of uptime (according to [1]), it certainly feels worse than that.

    Maybe it would be better to measure uptime not as what percentage of time the site is up, but instead as how likely a randomly selected visitor is to be able to access the site.

    That would adjust for the fact that the window of peak demand, e.g. the US workday, which is when Github tends to go down, is also when people need the service the most.

    [1]: https://mrshu.github.io/github-statuses/

    • proxyscore 5 hours ago ago

      Your first argument is borderline unintelligent and selective and emotional and definitely off, your second argument is genius, though, I was wondering when anyone here would bring that up, so many of the status sites are a selective one way street to escaped sla obligations

  • mkagenius 8 hours ago ago

    Why doesn't https://www.githubstatus.com reflect this accurately?

    I usually check this and then doubt my network might be at fault if they don't show outages.

    • skissane 8 hours ago ago

      I think that page only displays outages if some manager at GitHub decides it is "sufficiently major" to display there.

      GitHub has outages all the time without that page showing anything.

      Another comment pointed to a third party alternative which is more accurate – https://mrshu.github.io/github-statuses/ – but that alternative isn't showing an outage right now either, since I believe it is being fed by GitHub.com's incident data.

      • mrshu 7 hours ago ago

        > I believe it is being fed by GitHub.com's incident data.

        This is indeed correct.

        • skissane 7 hours ago ago

          Now https://www.githubstatus.com/ is declaring an incident but your page doesn’t have it

          Maybe I spoke too soon in labelling your page “more accurate”?

          • mrshu 3 hours ago ago

            It is only being updated once per day. In an ongoing incident it is indeed not necessarily more accurate.

    • adrian_b 7 hours ago ago

      Yes githubstatus shows all OK for Europe, where I am, but when trying to open the Web page of a repository I receive:

        503 Service Unavailable
      
        No server is available to handle this request.
      • bheztio 6 hours ago ago

        They updated the status page now, it says they are "investigating degraded performance". Like, their web is 100 % down. But no, have to call it "degraded performance for some users, some services" etc.

      • hurfdurf 7 hours ago ago

        Same here. What's the point of a status page at this stage? "Copilot Ali"?

    • exploraz 7 hours ago ago
    • egorfine 7 hours ago ago

      It never does.

  • shimms 8 hours ago ago

    They seem to be having an issue with release downloads (https://github.com/orgs/community/discussions/212407).

    Our CI is broken at the moment, unable to download tools used in various checks (gitleaks, etc).

    • kmfrk 2 hours ago ago

      Release download speeds felt particularly slow over the last week on my setup for whatever reason.

    • lifeisloving 7 hours ago ago

      Why dont companies host their own CI anyways.. Its really not that complicated. I see companies spending 20k a month on github actions.. It makes no sense to me.

      • shimms 6 hours ago ago

        We migrated to GitLab a while back, and run Buildkite for CI.

        Our public mirrors for open source stuff that results in built binaries that we ship is built on GitHub Actions from the public GitHub mirror so we have build attestation from the source.

        Sometimes you're just dependent on things.

      • clivedup 7 hours ago ago

        The opportunity cost is higher.

  • esFran 8 hours ago ago

    I'd recommend the free (as in freedom) Git alternative to everyone: Codeberg. Super lightweight, without that vicious circle of knowing your code is being used to train AI. Free for hobby projects, and reasonably viable for professional ones.

    • dutchbart 8 hours ago ago

      No, not free as in freedom. LLM-assisted coding is prohibited and everything MUST be open-source. So not viable for all (hobby) projects.

      • slaymaker1907 3 hours ago ago

        Do they really prohibit all use? My reading of their policy is that they mainly block vibe coding. It doesn't seem like LLM auto-complete or using it to review your code would run afoul of it.

      • teekert 5 hours ago ago

        Hmm I have my Hugo website there, closed repo and build, in part, with LLMs. Never realized while setting thing up that both are not allowed tbh. I just wanted to not choose GH.

      • vrighter 7 hours ago ago

        free as in freedom for humans. machines neither want nor need freedom

        • _heimdall 6 hours ago ago

          That's like saying you're free to build your own shed, as long as you don't use hammers. Hammers don't need nor want freedom.

        • philipallstar 7 hours ago ago

          Machines are tools used by humans. Codeberg has banned a certain class of tool used by humans.

          • vrighter 7 hours ago ago

            as is their right, and the wish of a very significant chunk of their users. you're still free to use other platforms.

            • r_lee 6 hours ago ago

              "free to use other platforms" doesn't mean that Codeberg is a "free" platform.

              might as well say Google is a free Git alternative because you're free to use another one since you can't use it for git

        • Shish2k 7 hours ago ago

          I wonder how much internet-argument bandwidth would be saved if we used different words for "freedom from" vs "freedom to", instead of putting both things on a single scale and then arguing over which is "more free"...

          • 1718627440 7 hours ago ago

            It's a matter of perspective and what you perceive to be the default. Freedom from prevention of X is Freedom to not do X.

        • bheztio 6 hours ago ago

          So if I'm not free to use a hammer, that's actually not a constriction of my freedom because "the hammer neither wants nor needs freedom"?

        • aabhay 7 hours ago ago

          Speak for yourself

          • vrighter 7 hours ago ago

            i spoke for nobody. neither myself nor you or anyone else. and machines can't speak or hold opinions, so i can't speak for them either

      • otabdeveloper4 7 hours ago ago

        Okay, what's the downside tho?

        • Hendrikto 7 hours ago ago

          Restricting user freedom.

        • solyin 6 hours ago ago

          Having to wade through piles of human slop.

          • pixel_popping 5 hours ago ago

            Insecure human slop on top of that. Projects not checked in-depth by AI is recklessness at this point, prove me wrong with an example.

            I've found 2 serious vulnerabilities in Librewolf (can crash computer from HTML page, from an iframe) as an example this week, as per their rule, agent isn't allowed to report it, guess we will just have to leave those critical around or wait around that a human discover them while attackers already exploit them in the wild :/

            Codeberg doesn't want the world to be secure, quite shady, unless they care to explain what's the downside of discovering critical vulnerabilities with AI, especially automatically? Thinking that humans will write more secure programs than machines is just wrong and shows an outdated and dangerous mindset.

            Humans+AI > Humans.

            • tavavex an hour ago ago

              Their ToS only forbids projects that 'mostly consist of code' created with genAI, it says nothing of code that is reviewed through it or projects where it only makes up a small portion of the whole package. The reasons for that are outlined and quite clear - their organization focuses on projects with a certain license and LLM code is in limbo regarding its copyright status.

              Basically, unlike your stance, they are rejecting people who are outright saying that AI alone > humans. Which seems to be a popular opinion on HN. Given how angry you are about this, I don't think there's as much separating your opinion from the vibe coders' view as you think there is. I mean, look at this, we're only a few years into LLMs being viable code generators and we already have people accuse them of malice, recklessness and embracing 'human slop' because they dare to have a softball policy on not using this brand new tech absolutely everywhere and for everything. What will the average opinion be like in 2 years? Humans are just bad and can't be trusted to do anything manual at a computer, leave it to the agent?

      • foolserrandboy 5 hours ago ago

        So they will never have these frequent downtime issues because they will never be bombarded by agents building millions of personal tools with no license applied. Clever.

      • shiomiru 5 hours ago ago

        Free as in freedom has had a rather specific meaning in software for the past 40+ years, which is "software that matches the Free Software definition." Codeberg runs on Forgejo, whose authors grant you the rights to run, study, modify, and distribute it - unlike GitHub, which is proprietary. (Indeed, there are many Forgejo instances besides Codeberg, it's not just a theoretical benefit.)

        It is a ridiculous misinterpretation to assume it means "somebody else will host anything I upload, free of charge", which is of course not the case - not on GitHub either, by the way, their ToS just happens to be more friendly to slop.

        (For understandable reasons, too; while M$ is making a killing from selling LLM access & Free Software as training data, the main "benefit" Codeberg sees is getting DDoSed by LLM scrapers 24/7 and vibecoders using up disproportionate amounts of resources.)

    • raincole 7 hours ago ago

      > that vicious circle of knowing your code is being used to train AI

      There is no such circle. If you put your work in public, it has been used to train AI. That's it.

      • notrealyme123 6 hours ago ago

        And if you put it in private we will never know, but pinky promise. They would never steal training data

    • peanut-walrus 6 hours ago ago

      Why exactly do you think public code on Codeberg is not used to train AI? I would assume the opposite...it's unlikely to be Codeberg themselves doing it, but for sure your code is going to be used as training data somewhere.

    • aurareturn 8 hours ago ago

      People who migrate away from Github, what makes you think they're more reliable? All Git services are prone to massive surges in commit volume due to AI agents. At least if Github goes down, your boss is not going to blame you.

      I'd rather stay in Github and maybe use another service purely as backup.

      • Lapel2742 7 hours ago ago

        > All Git services are prone to massive surges in commit volume due to AI agents.

        You are probably wrong on that.

        > You must not share projects that mostly consist of code written by "generative AI"-tools (including services such as Claude, OpenAI Codex). Such projects having an unclear copyright status (see requirements § 2 (1) 1 and § 2 (1) 3) and furthermore have little safeguards to ensure that they do not include harmful code (c.f. § 2 (1) 5).

        https://codeberg.org/codeberg/org/src/branch/main/TermsOfUse...

        • aurareturn 6 hours ago ago

            You are probably wrong on that.
          
          I don't think I'm. If another Git service gets popular, they too will experience surges and go down often.

          If Github truly becomes unreliable and people migrate to this niche service en mass, they'll ignore their no AI coding policy.

          • chrismorgan 6 hours ago ago

            You missed the rest of the comment, which was to the effect that such AI agents are not allowed on Codeberg. Any abuse at scale will be easily detected and banned.

            • aurareturn 6 hours ago ago

              How would they know?

              Are they going to ban your account based on an algorithm? Are they going to personally check your code? Are they going to scan your code base for AI generated code? How do they define “mostly”?

              These seem like insane policies to have a critical business dependent on.

              Not only that, their uptimes are still worse than GitHub.

              • chrismorgan an hour ago ago

                I said any abuse at scale. Very good heuristics are quite straightforward, you can then manually check those that are less reliable.

                • aurareturn 23 minutes ago ago

                  Manually check? What? You want their staff to check your company's secret code and see if any line was written by Claude?

                  Come on man.

                  Heuristics are never 100%. Even a 0.001% error rate is simply unacceptable to businesses.

                  Imagine being that business that gets banned because their heuristics mistaken you for using AI coding agents.

                  PS. AI writing code does not have to be in scale. It's not easy to detect.

      • atiedebee 7 hours ago ago

        Codeberg specifically has a policy against projects that are maintained heavily with AI tooling, so I don't think they would have that problem.

      • Shorel 7 hours ago ago

        Their terms of service forbid AI agents or even AI created code to be submitted there.

        • aurareturn 5 hours ago ago

          How would they know?

          Detect it using an algorithm and automatically ban your business account?

          Do people not see how insane that policy is?

      • hagbard_c 7 hours ago ago

        > I'd rather stay in Github and maybe use another service purely as backup.

        Swap those two and you're on the right track. I think the best way to use GH is as a mirror for a self-hosted git instance, with or without web frontend. It gives you the reach of GH with the, dare I say it, reliability of self-hosted. It also gives you the chance to keep things off GH which you don't want to be there. Keep the self-hosted instance private for most things, use GH links for public access. When GH is down again you can always give people (temporary) access to your self-hosted instance if the need arrives.

    • alpaca128 7 hours ago ago

      TOS are not protection against scraping. Is there any reason to not assume by default that Codeberg gets used for training just like every other online collection of content?

    • r_lee 6 hours ago ago

      see e.g.: https://codeberg.org/ziglang/zig/commits/branch/master

      "This endpoint is seeing a high influx of requests for this repository, to preserve the availability of Codeberg your request will not be processed. Sorry for the inconvenience and please try again later."

    • _heimdall 6 hours ago ago

      I've had issues recently with CI jobs in codeberg taking quite a bit of time for a runner to pick it up.

      This is not a complaint, its a free offering and I appreciate alternatives, but just yesterday I moved one of my open source projects from codeberg to github after waiting 30+ minutes multiple times for CI to start.

    • roenxi 7 hours ago ago

      Hosted services can't really be "free as in freedom". We can't know for sure what software they're actually running and users don't have any freedom in using their service since that is entirely at their pleasure. For something to be free the thing has to be able to move beyond its present controller.

      • 1718627440 7 hours ago ago

        Isn't that the case fro AGPL? You have the control over the source code and can change it, even if the current instance is not in your control.

    • matkoniecz 7 hours ago ago

      Why you think that code from Codeberg is not used to train LLMs?

      (You are not required to agree for such use in tos, but it will be used anyway)

      (Not by Codeberg operators, to be clear)

    • Muhammad523 8 hours ago ago

      You're now my friend just because you said "free as in freedom"

      • PalmPilotProMax 7 hours ago ago

        Join us now and share the software;

        You'll be free, hackers, you'll be free~~~

    • mejutoco 7 hours ago ago

      git --bare init

      and some discipline can go a long way.

    • serf 5 hours ago ago

      i'll never defend microsoft, but it irks me to hear the phrase 'free as in freedom' used specifically to describe a service that , as an advertised benefit, restricts user freedom.

      no thanks.

      I don't have to be pro or against AI to avoid signing a deal with (another) devil.

      p.s. : "my code won't be training AI if im on codeberg" is just straight delusion, regardless of their policies.

      • 1718627440 2 hours ago ago

        "'free as in freedom'" only narrows it down to not refer to the price. It doesn't actually state what you are free from or to.

    • BoingBoomTschak 7 hours ago ago

      Codeberg has its own issues with anti-scraping measures often impacting (not just annoying) users or some deep ideological bias; and I'm not talking about LLM/open source, which I agree with.

    • dizhn 7 hours ago ago

      I love and use Forgejo which they are both running and now developing within their ecosystem but development is so slow and such basic things are missing that one cannot help but think perhaps they should get some LLM assistance. At this rate they are very likely to end up a historical footnote.

      • lexicality 7 hours ago ago

        What's missing?

        • dizhn 7 hours ago ago

          One example Projects (boards). I would not necessarily call it missing for my own use case but it might as well not exist in its current state.

          Label management is a mess too with almost no ease of use features. Lots of clicking when prior art already exists elsewhere.

          Federation is another thing that's supposed to happen but isn't.

          Action runners I like actually but their web UI is really confusing.

          API is missing a lot of endpoints too. (Pretty straight forward for AI agents to work on as "good first issue")

          No machine users or management of the same via Administrator. If you want to have a bot (not necessarily AI) you need to create a user with a hard to guess password and manage it by actually logging in as that user.

          Another user will probably have their own set of things that matter to them.

          As I've already said above, I do use it and love it but sort of half hoped Gitea would have proceeded farther since Forgejo forked away. It hasn't.

          OneDev looks much more capable on surface to me today but I have not used it extensively yet and it has some features not included in the self hosted version. Not as much as open core projects but some.

  • JanneVee 8 hours ago ago

    504 from Firefox not logged in and works from Safari which is logged in. CDN error perhaps?

  • Kholin 8 hours ago ago

    > 504 Gateway Time-out

    > The server didn't respond in time.

    • alfiedotwtf 8 hours ago ago

      504 for the past hour, status says all green

      • Kholin 8 hours ago ago

        Still got 504 if I'm not logged in

    • fragmede 8 hours ago ago

      Interesting to see the black text on white background page.

  • pluc 4 hours ago ago

    At this point it's on you if you're still using them

  • faangguyindia 8 hours ago ago

    Github being host on Azure, this is really bad PR for their cloud and technical prowess.

    • cube00 8 hours ago ago

      > this is really bad PR

      No such thing as bad PR these days, 24 hours later and we're on to the next thing, that's assuming people even saw it in their doom scroll feed. Gone are the days when everyone watches the same TV news or reads the same newspaper.

      • bonesss 7 hours ago ago

        I'm just confused how we can be in an industry where "<shrug> what's the alternative, just buy MS and live with it" is making a strong comeback simultaneously as all the big tech giants, MS included, are telling us we can generate infinite code and cloud configuration for a reasonable per-token price.

        Amazon og Google or OpenAI having a one-click port of Githubs daily experience and Githubs CI infrastructure seems like it would be a pretty straightforward proof-of-AI-concept for a critical audience.

        • bagels 7 hours ago ago

          Every Github outage is one more shove in the direction of me taking my org off of github.

        • pluc 7 hours ago ago

          That's the cult of Microsoft for you. They have always been an average to below average company but have benefited from incredible openness from its customers because for the longest time they were perceived as the only option. Their real product is Microsoft

          Microsoft of today isn't a leader in anything they're not the only ones in.

          • cube00 4 hours ago ago

            > They have always been an average to below average company

            I think about this every time I'd hit a corrupted Access database, corrupted Exchange store or OneDrive messing up a sync. Was it actually the hardware or just sloppy programming filled with race conditions?

            I was going through some old emails in an @hotmail.com account only find some had their headers but completely blank bodies so I guess Microsoft's own servers get impacted by it too.

  • solyin 7 hours ago ago

    Oh man it's really struggling.

  • exploraz 7 hours ago ago

    504 gateway timeout only when logged-out. Authenticated access seems to be normal at my side.

  • chilli_axe 8 hours ago ago

    Seems to be back now.

    • skissane 7 hours ago ago

      I just tried again, my GitHub actions jobs are still getting 503 errors from GitHub API

  • gnegggh 7 hours ago ago

    currently trying to get out of my GitHub "enterprise" plan. Waiting patiently for the required support ticket to see if that is possible

  • ed_chambers 6 hours ago ago

    Yep, was down 2h ago in EU, maybe still down. Didn't bother to check. Just see my nightlies failing with some 504s.

  • ridowk 8 hours ago ago

    Their status page (lol) says it's working (https://www.githubstatus.com/) but i get a 504 time-out when trying to access repositories

    • mrshu 8 hours ago ago

      A big part of the reason why https://mrshu.github.io/github-statuses/ happened

      • wartijn_ 7 hours ago ago

        That page isn't more useful right now, it isn't showing any outage right now, but I can't view the source code, because of an active GitHub outage.

      • alpaca128 7 hours ago ago

        That page being hosted on github seems like an interesting choice.

  • czarek_pl 7 hours ago ago

    I read recently that number of commits on github platform increased 5 times over the year - their infrastructure is under heavy load. It's due to AI revolution. Stay tuned to see 5x more features in your favorite apps. Great times are coming :-)

    • r_lee 6 hours ago ago

      > Stay tuned to see 5x more features in your favorite apps.

      who is asking for this?

  • bagels 8 hours ago ago

    It's been down for >30 minutes. Status page of course says everything is fine. 504 when downloading releases, so CI and Terraform don't work.

  • AyanamiKaine 8 hours ago ago

    Man, it happens more and more that I can’t work because nix can’t fetch something from git or my elixir build fails because it needs to fetch a release from GitHub.

    Even if I change my CI to something else, all the dependencies depend on GitHub (Not like there exists a cheap/free alternative where nix packages could be hosted ¯\_(ツ)_/¯)

    • r_lee 6 hours ago ago

      I guess running a local cache would work, at least for nix

  • borisgolovnev 7 hours ago ago

    Regular reminder that if you just need a remote to collaborate with others and don't need stars commit graphs and all that social coding aspects, you can just host your git repo yourself on any cheap VPS.

  • geniium 7 hours ago ago

    funny yo post a github down again (?) with a link pointing to github.com

  • countWSS 8 hours ago ago

    Can't see commit pages.

  • jhgkblv 8 hours ago ago

    seriously, what do you guys need to finally get off github? being on github is a huge liability these days.

    • ridowk 8 hours ago ago

      The issue isn't about us, but there are a lot of projects that are happening on github. I wanted to take a look at the sources of a random lib when i realized it was down, it wasn't about hosting some of my stuff

    • Lucasoato 8 hours ago ago

      Solid open source alternatives?

      • JoshStrobl 7 hours ago ago

        Forgejo is a solid open source alternative.

    • matkoniecz 7 hours ago ago

      Why it is liability?

  • gib444 7 hours ago ago

    My algorithm has not needed an update for a while:

      def is_github_down(day):
          return day[-1] == "y"
    
    
    It does have a flaw though where if you are shouting, it returns some false hope.
  • dom96 8 hours ago ago

    Works for me

  • ipsum2 8 hours ago ago

    I have an agent automatically commit code hundreds of times to a private repo every day. sorry for contributing to the ddos.

  • BoingBoomTschak 8 hours ago ago

    Just another day in "Azure code", I guess.

  • DishyDev 8 hours ago ago

    Yep same, looks like another outage

  • Brajeshwar 8 hours ago ago

    We seem to be seeing `Github Down` pretty regularly but that seems to be mostly the public facing ones. In the last one year that I have been using Enterprise Github regularly, I don't remember it being down anytime.

    • beberlei 8 hours ago ago

      Hosted or even cloud?