MGoBlog 3.1: We’ve Migrated Comment Count

Seth February 22nd, 2022 at 11:09 AM

So far only those of you trying to reply in the message board have noticed, but our big back-end project finally launched yesterday. This post was supposed to come with it to fill a lazy late February Monday.

The two main changes are we moved the server to a much larger and more stable bank, and we upgraded the site from Drupal 8 to Drupal 9. Major events might still overload our capacity, but 80,000 people trying to refresh for basketball muppets at the same time shouldn’t bring us down anymore.

We also pushed a lot of the smaller and medium problems into this project, since fixing them earlier would likely be undone by the move. Some things, like custom code to get UFR to load on modern light browsers and our home-brewed points system, had to be written in-house by our coding hero at Human Element, Hillary Lewandowski. Getting this done also means we get to move on to the other big projects that I've been hinting at in meta threads the last few months.

You are invited to help us test it.

Of course, in the process we broke a few things. We had a long bug testing phase and found plenty, which means there are going to be more of them. I’ve started a new bug thread for this upgrade that the HUEL people are going to be keeping an eye on (not indefinitely). Please keep general complaints, feature demands, and anything related to the ads out of that thread so we’re not blowing dev hours on things they can’t fix.

Since the nerds like to know what we did, I’ve included the details after the jump.

[After THE JUMP: No you’re not supposed to understand all of this. I just want you to know it was hard.]

------------------------------------

Details on the update:

Via Hillary, the site has the same bones as when it was on Drupal 8, but we basically did an "organ transplant" to upgrade it to Drupal 9. All other updates and upgrades (to Drupal 10 and beyond) should be smooth and painless because of all of this work. In order to update to Drupal 9 we needed to upgrade the database to a newer version, which we couldn't do without the server upgrade.

Once we had the server hardware and software in place, all of the custom modules had to be updated to replace deprecated functions (pretty painless). What wasn’t painless was contributed modules needed to be upgraded to their most recent version (compatible with Drupal 9). This worked just fine for all but 3 modules:

These 3 modules needed small, but doable, updates to their code to mark them as compatible with D9 and update deprecated code. Points needed a lot more detailed bug fixes to get it up to snuff—you don’t talk about MGoPoints, but behind the scenes oh lordy did we have to talk about points. All updates to these contributed modules were provided back to the Drupal community to help get these modules back in active status, and allow other sites to upgrade to Drupal 9.

The general bug fixes we saved for the update included custom code to get UFRs to load on mobile, and a few features that weren’t working correctly, like the “Hide comments” function, and our custom template and theme code that was sometimes breaking the site’s layout. The crashing UFRs were an issue having to do with the way mobile browsers have been optimized for short attention spans, to the point that the browser would run out of memory and crash. Our articles are huge, with a lot of HTML elements, embedded images, and embedded videos/gifs close together. Hillary wrote a lot of behind-the-scenes code to resize the images, lazy-load the elements, and defer loading of javascript files so that the browser can paint earlier.

The jump code fix worked in Drupal 8 so we already implemented that. If you’re interested, gone is the <!-- break --> tag that Cloudflare wouldn’t get along with, so the jump is now a <div id="read-more">[After the JUMP: …]</div>.

Details on the server migration:

It’s all new server hardware. The old server was so old that our server host (Nexcess) refused to keep maintaining or updating it. The new server bank has a ton more RAM so it should work faster and run more concurrent operations. It’s a RAID Protected SSD setup, so it’s super-fast with redundant copies. Our internal tests had the site running about 8x faster for the normal things we do.

We also moved to a new server software, from CentOS 7 to CentOS 8, which was necessary for Drupal 9 database requirements. The new OS supports newer, faster software, a more robust site, and is way, way, way more secure.
    
The new version of Drupal updates from MySQL 5.6 to MariaDB 10.3, upgrade from PHP 7.3.31 to 7.3.32, and updates Redis from 3.2.8 to 6.2.6. The core was updated to 9.3.5, which included some bug fixes from last deployed version, but everything above was necessary to get to this bleeding edge point. Drupal 9 also came with updated system requirements and components for longer security support. Drupal 9 was not a reinvention—it was mostly built in Drupal 8 through deprecating APIs and updated dependencies.

Comments

1VaBlue1

February 22nd, 2022 at 12:43 PM ^

It looks pretty nice on the quick view, and it sounds a lot better!  I do have to say, though, that I'm quite surprised you're running this on RHEL (CentOS is a discontinued free flavor of RHEL).  I thought you were running a Windows shop, what with the crashes and all.  But seeing how old everything was, it kinda makes sense...

So you guys want the RHEL 8 STIG version, just for fun?  (LOL?  Blah, I'm going back to my Oracle STIG checklist now...)

bronxblue

February 22nd, 2022 at 1:27 PM ^

<nerd mode>

I'm interested to see how performance improves with upgrades to Redis and the move to MariaDB.  I've never used MariaDB in production but I've read that's it's more performant especially when coupled with Redis.  

</nerd mode>

befuggled

February 22nd, 2022 at 4:57 PM ^

Eh, I'd want to see the numbers first; it doesn't make sense for every customer.

I'm not convinced autoscaling is necessarily a great fit for a site like this one. Sure, it would increase performance under heavy load--but would it increase revenue enough to make the additional cost worthwhile? They're not losing orders if people have trouble getting onto the site.