Someone really fucked up the SL forum.

[WolfEyes]

Just now:

 

[Kalel]

i 'm getting an apache error page now... maybe they are in the middle of an upgrade... and you got lucky?

 

[WolfEyes]

Getting 404s now. Both Edge and Chrome.

 

[Noodles]

I would think that a company as large as LL would not do this live. It would be more like... Roll it to another server then point the domain there maybe?

 

[CronoCloud Creeggan]

I would think that a company as large as LL would not do this live.

You'd think...but this large company is LL. Oopsies are what they do.
Getting a 503 Service Unavailable error myself.

 

[Free]

Forum seems to be up and running again. Not seeing errors as I wander around LL's web properties, but some of it is a bit slow.

 

[WolfEyes]

Yeah, it's back and not a word about the outage. lol

 

[Casey Pelous]

Someone really fucked up the SL forum.

Gee whillickers, who could have done such a thing?

- Nancy Drew, The Secret Of The Bumblefuck Corporation​

 

[Lyndka Cochrane]

Someone tried to do an upgrade/update without reading the install instructions - I maintain a PHPbb forum and it isn't as straightforward as pointing at an installer (though it's got a lot better since I first started with it). I don't know about Invision but I bet theres some arcane procedure involved & the forums back now because someone restored a backup

 

[Madi Perth]

I would think that a company as large as LL would not do this live. It would be more like... Roll it to another server then point the domain there maybe?

As some one that does this for a living its not really that simple. There is always a test bed that is run where all this is tested out and make sure it runs correctly. But at some point you have to switch over to the production servers. This may sound like a simple task but it isn't always the case. Each system is different and there is always the chance that something will go wrong in a unexpected manner. This is why any company during changes overs calls these "outage windows" and not just "maintenance windows." No matter how well planned there is always a chance that something will stick its head up its ass and the whole thing will head south quickly.

 

[Andi]

This is a managed hosted service by invision boards, they do the upgrades and such, all LL is doing is pointing their url to invisions hosted services.

 

[Argent Stonecutter]

As some one that does this for a living its not really that simple. There is always a test bed that is run where all this is tested out and make sure it runs correctly. But at some point you have to switch over to the production servers. This may sound like a simple task but it isn't always the case. Each system is different and there is always the chance that something will go wrong in a unexpected manner. This is why any company during changes overs calls these "outage windows" and not just "maintenance windows." No matter how well planned there is always a chance that something will stick its head up its ass and the whole thing will head south quickly.

We do a release 4 times a week, rolling over one webserver at a time out of rotation so end users never see a server in transition. The new software has been in staging, actively used by all employees, 24 hours before release. Taking down the whole site for an upgrade is so 2005.

Devotion to Duty

xkcd.com

 

[Madi Perth]

We do a release 4 times a week, rolling over one webserver at a time out of rotation so end users never see a server in transition. The new software has been in staging, actively used by all employees, 24 hours before release. Taking down the whole site for an upgrade is so 2005.

Devotion to Duty

xkcd.com

An in that time you have never had one thing go wrong and take the whole site down or severely impacted the service in some way? I'm not saying they took the whole site down for maintance. I'm saying "shit happens."

 

[Argent Stonecutter]

An in that time you have never had one thing go wrong and take the whole site down or severely impacted the service in some way? I'm not saying they took the whole site down for maintance. I'm saying "shit happens."

I can't recall the last time we had to post an outage window or exposed a broken webserver to the customers.

Our main product had 100.0% uptime in 2021. That's no measurable downtime in a year.

Invision has a much simpler job to do, they have no excuse for the page at the top of this thread.

 

[WolfEyes]

This is a managed hosted service by invision boards, they do the upgrades and such, all LL is doing is pointing their url to invisions hosted services.

Hence the "someone" in the thread title.

 

[bubblesort]

Between this and the AWS outage the other day... I'm thinking every now and then a network engineer just snaps, and decides the internet was a bad idea, and tries to unplug it.

 

[Free]

Between this and the AWS outage the other day... I'm thinking every now and then a network engineer just snaps, and decides the internet was a bad idea, and tries to unplug it.

They usually require help.