OT(ish): Move MgoBlog to the Public Cloud?
I want to thank our fearless leader for his effort in this site; as we sit here in the after-Harbaugh /pre-Gary glow I think its important to realize that none of this would have likely come to pass if it wasn't for him and his efforts. I don't think this is hyperbole; his email outing was the final nail in the Brandon coffin.
That said, I feel like the site infrastructure lets us down on days like today. We basically punted the server load to twitter to avoid the Gary rush. In this day and age, it feels like elastic computing has evolved to the point where we shouldn't have to do this. A comination of pre-packaged dupal (Cloud Formation/Cloud Laucher on AWS/GCP) and elastic load balancing/autoscaling should enable our leader to a)lower is bill and b)support #chaos. Sooooo: are there any mgocloud arcitechts out there who would love to chip in a reccomend a solution to keep the site performant at all times?
February 3rd, 2016 at 4:24 PM ^
Sent from MGoBlog HD for iPhone & iPad
February 3rd, 2016 at 4:27 PM ^
Sure. I'm a API/app developer by trade, but do some stuff in AWS. I'd be willing to help
February 3rd, 2016 at 5:10 PM ^
Didn't Brian post an appeal for developers recently, to help with site redesign? You could just email him.
February 3rd, 2016 at 8:13 PM ^
That was for the Drupal upgrade IIRC. I did respond saying I'd be interested in working on the apps, but I would be of little practical use for the Drupal upgrade.
February 3rd, 2016 at 4:28 PM ^
Yeah. I don't want to sound like an ungrateful jerk for all the free content but the site is impossible to get into on days like this and even on other random days I get booted out. I'd contribute money to a new server.
February 3rd, 2016 at 4:28 PM ^
co-signed. I don't know much about web hosting specifically, but from personal experience, AWS provides a great, mature platform of cloud capabilities.
February 3rd, 2016 at 4:28 PM ^
I know (or highly suspect) that you were writing in English, but I have no idea what you just wrote...is a "Cloud Formation" a variation on the single wing offense? Maybe it's derivative of the 46 defense of the '85 Bears?
February 3rd, 2016 at 4:29 PM ^
Sent from MGoBlog HD for iPhone & iPad
February 3rd, 2016 at 4:29 PM ^
Not sure why you believe AWS would be cheaper. Can they handled the traffic? Sure, but you will pay pretty dearly for that previlege.
February 3rd, 2016 at 7:03 PM ^
February 3rd, 2016 at 4:30 PM ^
What a bunch of ungrateful dickwads on this board... ;-)
February 3rd, 2016 at 4:31 PM ^
You guys are addicts.
February 3rd, 2016 at 4:34 PM ^
You might want to retitle this thread "Meta"
February 3rd, 2016 at 5:52 PM ^
A meta policeman or a grammar nazi? Just wondering, since I happen to be both.
February 3rd, 2016 at 6:56 PM ^
"We're an anarcho-syndicalist commune. We take it in turns to act as a sort of executive officer for the week."
This week I'm living in the meta-policeman castle.
February 3rd, 2016 at 9:58 PM ^
Sent from MGoBlog HD for iPhone & iPad
February 3rd, 2016 at 4:35 PM ^
are you nerds talking about?
February 3rd, 2016 at 4:42 PM ^
I'm not 100% sure either, but I think they are talking about making a quantum leap into your mom's pants...
February 3rd, 2016 at 4:52 PM ^
Based on Quantum Mechanics, we are already in his mom's pants OR we are not already in his mom's pants. They are equally as likely. You won't know which until you walk in on us.
February 3rd, 2016 at 5:39 PM ^
Spoken like a guy who is not, um, currently taking measurements in his mom's pants, ifyouknowwhatImean.
February 3rd, 2016 at 6:57 PM ^
February 3rd, 2016 at 8:44 PM ^
February 3rd, 2016 at 4:37 PM ^
But I am in agreement, with the requisite qualifier "I love the site, but..."
It seems to me that this fine website being on such an unreliable infrastructure is like Al Borges putting Denard Robinson under center despite ovewhelming evidence to it's ineffectiveness.
And no, I don't even know if "unreliable infrastructure" is the correct syntax, but you get my point.
February 3rd, 2016 at 4:39 PM ^
It looks like the site already uses Cloudflare as a CDN for statically served content. As for what has been configured and deemed to be static vs dynamic only the admins can say, but it's possible that it could be optimized.
February 3rd, 2016 at 4:58 PM ^
Yes but given the real "meat" of this site are the boards and comments to recent threads which are not static content. So using the CDN only gives you minimal improvements - the banners might load better, the CSS is there, then... nada.
February 3rd, 2016 at 7:00 PM ^
February 3rd, 2016 at 4:42 PM ^
I've been a web developer for 16 years and this site has taught me that Drupal is to be avoided at all costs. There is no way that any website the size of this blog should be getting 503 errors like they do. I'd be more than willing to look at what they are using and make cost effective recommendations. I've seen one server crash because of traffic and if this site is getting that kind of traffic. They don't need to worry about money.
February 3rd, 2016 at 8:48 PM ^
I'm sure many of us would be eager to chip in a few bucks to alleviate some of the frustrations inherent in using these boards.
If there's any concern that it would take away from beveled guilt, fuck it, make it that you have to make a matching donation to beveled guilt. That would probably end up getting people to donate who haven't donated in the past.
February 3rd, 2016 at 4:42 PM ^
Hosting Drupal on Azure is pretty simple to setup, and there's definitely some elastic scaling available as well.
February 3rd, 2016 at 4:44 PM ^
Server upgrades, drupal upgrades, CSS upgrades to allow better ad placement, fixing voting, it has all been talked about here and either not happened or taken much longer to implement than it should. I see Brian tried Cloudflare today, but it didn't get the job done. Perhaps MGoBlog Drupal needs to go the way of the Haloscan variant. Drupal itself is never that pleasant and perhaps this site just has so many customizations it is time to dump it.
We could use the upgrade though, MGo used to be one of the better sites in terms of layout, stability, etc. Now compared to Shaggy, /r/cfb in reddit, etc it is lagging behind.
February 3rd, 2016 at 4:46 PM ^
. . . Conceptually what is the Reader's Digest version of the "Cloud"
February 3rd, 2016 at 4:55 PM ^
Sent from MGoBlog HD for iPhone & iPad
February 3rd, 2016 at 6:12 PM ^
. . . is it just a marketing term for what used to be called a server farm or something else ?
February 3rd, 2016 at 8:31 PM ^
Its main benefit is you are outsorcing a lot of the management of the system, companies guarentee uptime (99.9999%), and you arent tied to the physical hardware as cloud companies generally allow you to delete/add new instances at will. Basically instead of physically running your own server (either at home or hosted in a datacenter) or renting out a single actual box in a server farm, you are renting out a virtual machine which can be running on whatever hardware.
Cloud companines are basically renting out space from their own server farms and slapping a self-service interface on top for adding/deleting vms and some other customer service benefits
February 3rd, 2016 at 4:53 PM ^
From having worked with heavy and spikey web traffic I speculate that the MGoBlog site is suffering from Drupal(PHP)/MySQL-itis in that one server can only handle roughly 600-1000 simultaneous requests. I don't know what the configuration of this site is like but to make it work for, let's say 10,000 simultaneous requests, they'd have to do some type of load balancing, buy more servers/nodes, which equals more $$$. However, this site deserves to be accessible when there's news, and it's let us down on those occasions.
It can get very expensive to try to work around the limitations of PHP/MySQL. A better idea is to switch to a lower overhead and less blocking tech.
However, I'd like to help! Please let me know if I can chip in. I've successfully transitioned PHP/MySQL applications to Node/Mongo and seen the vast improvements in load and response times.
February 3rd, 2016 at 5:11 PM ^
February 3rd, 2016 at 5:52 PM ^
I'd volunteer but it's up to Brian. As someone else posted, he's investing in Drupal 8.
Also, while I may be a nerd, I'm not a dude. You'd be surprised how many women are on these boards.
February 3rd, 2016 at 5:59 PM ^
February 3rd, 2016 at 6:58 PM ^
February 3rd, 2016 at 9:17 PM ^
lol. Hey "Rob", prove that you're a dude.
February 4th, 2016 at 2:12 AM ^
February 3rd, 2016 at 6:57 PM ^
February 3rd, 2016 at 11:06 PM ^
February 3rd, 2016 at 4:51 PM ^
I did not experience a lot of problems today relative to some of the other things I've heard from folks, but about an hour ago I was kicked out and then unable to log back in for about 20-30 minutes as the site was intermittently up and down for a while. One thing that I did notice today is that the 524 errors (connection timed out) started again during peak traffic after a long absence - they were prevalent during the height of football season. If you open multiple tabs for threads, as I do when modding, some were opening and some were not so bandwidth was definitely an issue.
February 3rd, 2016 at 5:02 PM ^
It's only been during the last 15 minutes that I've managed to get on without getting kicked off into another 502. I've gotten some 521s too, whatever those mean.
Problems the same on laptop and mobile.
February 3rd, 2016 at 5:16 PM ^
521 errors are pretty much the exact same thing as 503 errors.
February 3rd, 2016 at 6:13 PM ^
I couldn't get on this morning, and purposefully avoided the site all day (until now). Instead I followed on the livestream, twitter, and victors boards, fyi.
February 3rd, 2016 at 6:54 PM ^
This site is basically unavailable on gamedays, announcements, other events etc.
The exact time it should be readily available. Not saying anything new here, just adding that "it's getting better" is not correct.
I hate sending my traffic and dollars elsewhere. Please make it easier to give you my traffic and revenue.
February 3rd, 2016 at 5:00 PM ^
"Put it in the cloud" is easy to say but much harder than you would expect. Just hosting the site on cloud products would be easy but the elastic part requires a lot of engineering and staff. It's not nearly as automated as you would think. In particular, scaling backend services is much more difficult than front end.
At any rate, I recall a few months back Brian had a front page post looking for a developer with Drupal experience, so I expect he's got some plans around this. It's going to be a long-term process.
February 3rd, 2016 at 5:00 PM ^
Wouldn't a caching solution would be cheaper for the (relatively short) surges in traffic that the site sees? Switching to a full AWS cloud architecture with properly sized instances and auto scaling groups would be fairly expensive*, require more attention to maintain, and a little overkill for the normal traffic patterns and days. A caching layer or CDN may not give you up to millisecond accurate data or posts but it would allow the site to continue to server traffic without breaking the bank
* I say this with zero knowledge of the existing infrastructure, cost, or budget.