6th June: Further update at the bottom of the post
Here at geegeez.co.uk we pride ourselves on being able to offer uninterrupted access to the editorial, racecards and form tools that we know you've come to rely on to find your picks. We have an outstanding record of 'uptime' - time the site is available with all services running - but this past week has been a challenge, to say the least; in what follows I'll share what has happened and is happening, and why.
Tuesday last week was a scheduled maintenance day - techie speak for something that has to be upgraded in order to ensure the future smooth running of the show. In this case, we were upgrading our server, an activity that involves migrating all files, code and databases (as well as email) from the old 'box' to the new one.
Servers have lots of settings, dials and levers and a change in just one of those can result in problems. Given that the software you know as Geegeez Gold runs to hundreds of thousands of lines of code across multiple languages and is deployed on four different servers, problems are unwelcome visitors.
As it transpired, the first issue we faced - last Tuesday - was with our own firewall (software that defends us against nefarious would-be interlopers who try to hack the site). Because our IP address (the way computers and servers identify each other on the internet) had changed, the firewall thought we were attacking ourselves! We spotted that immediately but, because the software is very good at its job, it still took a while to reconfigure things.
And that's when the 'fun' really began...
What became apparent was that the full results files - which we receive from our data supplier, Racing Post, and with which we then perform a million downstream magic tricks - were not processing; or, rather, they were processing on the separate server where that stuff happens, but that machine could not connect to the database on the new server to write them into the data store.
This, it turns out, is a problem. A big problem. Those results drive large parts of the Gold ecosystem, including Query Tool, Bet Finder, Bet Tracker and, crucially, the reports.
Previously, last weekend, I'd had the very great fortune to be in Bratislava with Carole (formerly known as Mrs Matt) while our son had a two-day sleepover at granny's. We were there to see the peerless Depeche Mode in concert on the Sunday and had got in the mood by attending a 'warm up' night in a downtown nightclub on Saturday. Loud music, lots of grog, and two hundred Eurogoth types of a similar vintage strutting, heaving and sweating in a basement dungeon. A marvellous throwback to the days of all of our youths.
By Wednesday, it was apparent that all was not well not only with the website but also with non-techie conducting a growing orchestra of disparate technicians in search of a fix. For the server problem, I mean.
Yes, I had managed to bring Covid back through the 'Nothing to Declare' channel on the way home, and it was now taking staunch residence in my respiratory system. Thankfully, I'm reasonably fit (for my age, at least) and fully vaccinated but, in spite of those pluses, Wednesday to Friday particularly - and still dragging its heels leaving me now - were not good health days.
I offer that merely as a little added context to what was already a debilitating situation: perhaps it reflects well on family health and the like that very little in life stresses me out more than site downtime, and here I was in a pickle.
At this point I want to thank all of the key people who have helped so far, and who are continuing to work through the final obstacles. The main man has been Dave - Database Dave as I very affectionately call him. He's an absolutely brilliant engineer, a lovely bloke, and he now works for a bank full time. In spite of that, Dave put in three near full-time shifts diagnosing and liaising with our hosting company's (excellent) support team to find the problem.
Heading up the support team was one of the co-owners of the hosting company, Dom, himself a database architect out of the very top drawer and a fellow of the open source group that support and maintain MariaDB, one of the world's most popular database technologies and the one we use to store and retrieve horse-y info here at geegeez.co.uk. These are the calibre of brains that were stymied.
Without getting too technical and boring - far too late, I hear at least one reader cry - the issue was nothing to do with the code. Rather, it was to do with a character set incompatibility which had been introduced by MySQL, the bossy older step-brother of MariaDB. Long and short, we needed a different database connector in order to mitigate for a cock up by the team at MySQL, the preeminent database technology in use today.
It was Friday evening, a few hours after Soul Sister's and Frankie's Oaks and a few before Auguste Rodin's and Ryan/Aidan/the lads' Derby, when we finally got all the jigsaw pieces in place. Saturday morning involved updating the connector, running a series of tests and, when they came back positive, processing the backlog of results ahead of the start of Derby day.
The sense of relief here was genuinely palpable. Imagine decrepit Grandpa Joe from Charlie and the Chocolate Factory lying in his bed just before the tale's title hero barrels in clutching the golden ticket. That's not a dissimilar scene to the one that played out in this sickly Hackney homestead!
The relief. Sometimes the burden of this gig is a bit too much. Not very often at all, to be truthful. But sometimes. This past week has been one of those times.
Anyway, we're in far better shape but we're not out of the woods yet. Neither the website nor I are clear of the impediments placed upon us early last week. The dreaded 'rona is taking its time to move on, in spite of a false dawn yesterday where I was feeling much better, and today's Morningsong was accompanied by a thumping beat inside the noggin which is yet to meaningfully subside.
Of more concern to you will be the fact that Query Tool Angles, a hugely popular part of the Gold provision, is not showing. The timing of this suggests that it's related to the fixes we applied for the results processing issue; and so we'll focus on getting that sorted today. Additionally, we're having some problems with email deliverability - on messages sent out from here - regarding our @geegeez.co.uk email addresses.
So, still some stuff to manage through, and I thank you very sincerely for your patience and understanding while we get to the bottom of them. If you're feeling charitable, you might reflect on how different life is using other data sources to find interesting horses; if not, I understand completely, and please know we're on the case.
In a day or three, this will all be a fading image in the rear view mirror as we speed forwards to the next set of racing puzzles but, for now, there remain some smaller conundrums to unravel behind the scenes.
I hope the above has offered at least a little flavour of the past week and, more than that because this is supposed to be an entertainment website, that it wasn't too boring a retrospective.
Keep well, and thank you again,
Matt
[6th June 10am]
Just when I thought it was safe to go back in the water... the main challenges of last week had managed to mask a further issue, with the Query Tool server. That machine was refusing to speak to the new live server, in spite of all introductions having been made (firewall permissions having been initiated). Again, we were eventually able to get to where we needed to be and, late last night, Query Tool began catching up on a week's worth of data updates. It is now fully up to date. Hallelujah.
QT Angles, however, was refusing to play ball. Again, this issue was masked by the QT server issue, which in turn was masked by the Live server results issue. More masks than a secret policeman's ball! Anyhoo, long and short, this was a bug. The only one in the entire week-long docudrama, and we were able to trap it, squish it, and re-run the QT Angles code this morning. All is now as it should be. Sing hosannah!
Thanks again for your patience and understanding during this torrid time. I'm off for a very long lie down...
Matt