To Scout users and subscribers,
The forums experienced a major hardware and database failure on Sunday. They're up and running again, but recovery required using a backup from February 14, and therefore all posts between then and Sunday morning have been lost, except for private messages. We are very sorry for this and will compensate our subscribers by adding a week to all subscriptions proactively. This will be done automatically, you need not call or take action to receive it.
As most of you know, Scout was recently purchased from a prior owner who eliminated all internal engineering and made no significant hardware investments since 2005. We have been upgrading this aging infrastructure to reduce and eventually eliminate the chances of failures like Sunday's. To minimize potential disruption, we had planned to wait until after the current NCAA basketball season to perform major upgrades, but will obviously accelerate those plans now. We have also been working hard on a major site redesign, including mobile-friendly sites, improved video, new categories and other features, and look forward to delivering those in the next few months.
Again, we apologize for the inconvenience, will strive to do better and appreciate your patience and support.
To our users and customers,
We apologize for the forums issues this past week that are currently affecting Scout forums. They arise from having to replace obsolete hardware that had not been updated under previous ownership. These updates had originally been planned for after the NCAA basketball season to minimize intrusiveness, but we obviously now will be accelerating this work.
We will do whatever it takes to provide stability, speed and capacity, but will err on the side of getting things right for the long haul as opposed to a Band-Aid approach.
We will continue to provide updates as the forums issues continue to be addressed.
Scout Engineering Team
MONDAY AFTERNOON UPDATE (3:45 pm) - From Scout CTO Bill Sornsin: I'm sorry to report we do not have imminent good news. Our repeated attempts to restore the forums database cluster from backups have all failed. We have 7 people working this...and have been in contact with all relevant vendors, and have tried a variety of DB recovery techniques, but the most recent restore failed about 30 minutes ago. We have simultaneously experienced problems, as mentioned earlier, with the storage media underlying the databases, one reason this has become unusually difficult to fix.
We are currently hand-carrying additional hardware into the datacenter so we can run multiple restoration processes in parallel. We worked through the night last night, and will continue working as long as necessary to resolve the problem.
The latest news stories and info from the Associated…