Rendered at 17:58:45 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
arandomhuman 59 minutes ago [-]
I love how even self hosted workers don't work during these outages - running jobs on their infrastructure being flakey is marginally acceptable, but the API to simply schedule workflows having this availability is mind boggling. Github just doesn't seem like a serious company anymore.
inigyou 5 minutes ago [-]
GitHub implements self hosted runners by running a normal runner that passes the environment to your runner and then polls it. That's why they cost as much as the smallest GitHub-hosted runner.
I'm not sure that it's that mind boggling. The entire complexity and value of GHA is not in the runners but the scheduler that the GH frontend hooks into and runners subscribe to jobs from. It's the most likely thing to fall over. '
If the scheduling was self hosted it would be inexcusable but you can always just connect whatever you want to webhooks.
packetlost 8 minutes ago [-]
That "scheduling" is just a git hook and a message queue, maybe with some database updates in between with very clear boundaries that make sharding easy to reason about, assuming they have a sane architecture (they evidently don't)
hinkley 12 minutes ago [-]
The entire complexity and value of GHA is rent-seeking to keep the lights on for other things.
They have a strong motivation (self preservation) to continue to misunderstand the problem. If they did what is best for us, then we could avoid a substantial fraction of all GH subscriptions by using a FOSS tool to hit the Pareto frontier by replicating just enough GH services to watch commits and PRs.
cyanydeez 3 minutes ago [-]
gitlab-ce can run all the integration tests in docker et al.
Sitting on Github these days is the same as sticking to twitter a decade ago, expect next mecha hitler, I suppose.
x3n0ph3n3 39 minutes ago [-]
Microsoft is not a serious company anymore.
hinkley 4 minutes ago [-]
They never were. They were just really good at maintaining the bluster and backing it up with expensive lawyers. Now that they don't come in claws-first like they did when they were a NASDAQ rocket to the stars, it's easy to see.
Their MO was to court an executive and sell second-rate tools to them before the people who had to use them had a chance to say anything. It doesn't matter how much evidence you can provide to the contrary, once the million dollar deal is signed, you are going to be tasked with finding reasons to say that your executive was shrewd for buying this pile of junk and unfulfilled promises, and not an insane idiot sucking away your job satisfaction as fast as they can.
They did a lot of deals based on how their products would have features their competitors already have 'soon' when they haven't even started them, and a long track record of taking 3 major releases to get from something to good, and then breaking everything again by doing a 4th major release that re-imagined everything and made it horrible again.
I'm not going to claim that Apple was or is a panacea. Apple doesn't use vaporware which is big, and their Cycle of Awful is 2 releases instead of 3. You could afford to skip 1 waiting for the next even-numbered version, instead of being 2 versions behind and getting pressed to upgrade.
peterldowns 1 hours ago [-]
Can't wait for the Graphite/Cursor alternative (Origin) to become available. Actions being down for an hour in the middle of the US East workday is crazy. It's super frustrating to be an enterprise customer of Github, seems like we don't get any dedicated resources or additional stability. At this point our only option is self-host github enterprise or move away.
KyleTheDev 2 hours ago [-]
Incident number 6 for the month of August, notably on August 6th. This pattern does not bode well.
paularmstrong 1 hours ago [-]
Four of them were regarding AI/Copilot. So that's really just one long running incident and will probably continue to be 50/50 chance of being down.
Insanity 2 hours ago [-]
Soon we can start posting "GitHub {x} is up!" instead, that's becoming more news-worthy lol. I know they like to blame AI/scaling, but really they are owned by one of the main hyperscalers, so I find that excuse kind of weak..
1 hours ago [-]
presbyterian 2 hours ago [-]
The speed at which GitHub is tanking their reputation is fascinating
opwizardx 36 minutes ago [-]
How did they get Github Pages into “degraded performance”? Aren’t those just CDN stored static pages?
classictraffic 31 minutes ago [-]
I believe Pages uses Actions under the hood for publishing, so in my head it sort of makes sense for these things to be affected in the same outage
fir3pho3nixx 13 minutes ago [-]
Are we even surprised anymore? Since microsoft (the IBM of its time) have taken this over all I can say is ... piss up ... brewery.
newtonianrules 2 hours ago [-]
Did Microsoft fire all the people working on the GitHub features people actually use?
Topfi 1 hours ago [-]
I mean, Github doesn't have its own CEO anymore, so arguably.
Arubis 50 minutes ago [-]
Github Actions are rapidly becoming an argument against dogfooding into your critical path.
ptx 38 minutes ago [-]
Are there any other CI providers that offer free cloud-hosted Windows runners?
Or has Microsoft made sure (through Windows licensing terms and pricing) that it's not possible to compete with their own CI offering?
inigyou 2 minutes ago [-]
You can always pay for it, use your own machine, use a VM on your own machine, or if you are developing non-slop FOSS, ask Codeberg.
classictraffic 42 minutes ago [-]
This has been preventing me from merging a PR for over an hour. I'm really tired of the constant outages GitHub has
fir3pho3nixx 15 minutes ago [-]
Ever since Microsoft has taken this over are we even surprised by the downtime anymore? Piss up ... brewery ...
denysvitali 1 hours ago [-]
Was? Still is
Topfi 2 hours ago [-]
Very frustrating, nothing more I can add at this point.
h2aichat 1 hours ago [-]
I am having their issues, also... Anyway this guys are great!
dlcarrier 2 hours ago [-]
So, it's a normal Thursday?
dboreham 1 hours ago [-]
Someone needs to work on improving the way Actions deals with resource exhaustion like this. Today it exploded all the CI jobs with "failed" and an inscrutable message about "The job was not acquired by Runner of type hosted even after multiple attempts" then "Internal server error". It should say "Timed out due to no runner available, our apologies, try again later, or pay $$$ here to get priority for runners".
This is no surprise given standard Microsoft operating procedure - https://news.ycombinator.com/item?id=47616242
If the scheduling was self hosted it would be inexcusable but you can always just connect whatever you want to webhooks.
They have a strong motivation (self preservation) to continue to misunderstand the problem. If they did what is best for us, then we could avoid a substantial fraction of all GH subscriptions by using a FOSS tool to hit the Pareto frontier by replicating just enough GH services to watch commits and PRs.
Sitting on Github these days is the same as sticking to twitter a decade ago, expect next mecha hitler, I suppose.
Their MO was to court an executive and sell second-rate tools to them before the people who had to use them had a chance to say anything. It doesn't matter how much evidence you can provide to the contrary, once the million dollar deal is signed, you are going to be tasked with finding reasons to say that your executive was shrewd for buying this pile of junk and unfulfilled promises, and not an insane idiot sucking away your job satisfaction as fast as they can.
They did a lot of deals based on how their products would have features their competitors already have 'soon' when they haven't even started them, and a long track record of taking 3 major releases to get from something to good, and then breaking everything again by doing a 4th major release that re-imagined everything and made it horrible again.
I'm not going to claim that Apple was or is a panacea. Apple doesn't use vaporware which is big, and their Cycle of Awful is 2 releases instead of 3. You could afford to skip 1 waiting for the next even-numbered version, instead of being 2 versions behind and getting pressed to upgrade.
Or has Microsoft made sure (through Windows licensing terms and pricing) that it's not possible to compete with their own CI offering?
Once this goes in, I'd expect to see 89%, which is zero nines. (I'd like to say, "a new low!", but sadly we've had this before)
After 6 years of this nonsense of "centralizing everything on GitHub", it is not a good idea at all.
You might as well self host like I said before [0].
[0] https://news.ycombinator.com/item?id=22868406