Stellar Attraction Discussion Forum
http://www.stellar-attraction.com/cgi-bin/yabb2/YaBB.pl
Site Information >> News >> Scheduled Maintenance - 24th - 25th September
http://www.stellar-attraction.com/cgi-bin/yabb2/YaBB.pl?num=1316820444

Message started by ProgMaster on 24th Sep, 2011 at 12:27am

Title: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 24th Sep, 2011 at 12:27am
All,

As mentioned on the S.A. news page, I am performing essential maintenance to the source, SAM server over the weekend of the 24th and 25th of September.

This is an essential, annual 'service' to SAM and will involve (amongst other things), backing-up the entire file structure and database, upgrading all of the core applications to the very latest versions and cleaning / replacing a number of the fans.

During this time, there will be intermittent outage of the stream(s). Please do not panic - I will be working continuously and will endeavour to have this essential work completed as soon as possible. I shall also post within this thread, updating as progress is made and confirming once complete.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 25th Sep, 2011 at 5:50am
Folks,

All is proceeding as planned and without any problems.

The music database back-up has begun and will take some time ... I'll post again once this has finished.

D

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 25th Sep, 2011 at 6:37pm
All,

SAM's RAID drives are currently being formatted. Once complete, I will be in a position to re-install all of the software and data.

I'll post again when I have more information.

Again, the real time consuming element of this task these days is the sheer size of the music database and the length of time it takes to copy, back-up and restore.

D

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 26th Sep, 2011 at 5:54pm
SAM's SQL server has been re-formatted and the very latest version of MySQL server - 5.5.16 - has been installed.

Once this has been configured, I'll be in a position to install the SAM software itself and will then start the final configuration.

I'll post again once this has been completed.

Apologies for the downtime, folks.

D

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 26th Sep, 2011 at 10:55pm
Well, the power outage this evening hasn't really helped in expediting things ...

I've had to re-install the Firewall software and associated definitions.

Realistically, it's going to be tomorrow before everything is functioning as normal again.

As usual, I'll post within this thread when I have further info.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 27th Sep, 2011 at 8:12pm
All,

I'm still having some issues with SAM's MySQL server. Hopefully I shall have these ironed-out by the end of this evening and everything will be back online again.

I'll post more in this thread.

Thanks for your patience !

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 27th Sep, 2011 at 10:26pm
Folks,

Well, I'm delighted to say that all is running as normal again. [smiley=thumbsup.gif]

Sincere apologies for the extended downtime, however, I worked as hard and for as long as I could to complete this essential work. SAM has had three fans replaced, an unhealthily large amount of dust removed and an extra Gig of RAM added for good measure.

Also, amongst numerous other software and application upgrades, the MySQL server is now running on the very latest version.

Continue ... to enjoy.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by David Z on 28th Sep, 2011 at 1:04am
Darran -- Thank you for your continued efforts... The music on SA is well worth the wait!

:-)   8-)

- David Z.

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by zappa on 28th Sep, 2011 at 8:18am
Congratulations, Darren, hard work you have done, I know what I'm talking about, just restored an SAP server with 3TB database for 3000 users,,,

btw... Guernsey is a beautiful Island. my next vacation is planned.
https://buchung.casamundo.de/houseinfo.php?SearchId=288355&Duration=7

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by fish_head on 28th Sep, 2011 at 8:55am
Thx D!!   8-) 8-)

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 28th Sep, 2011 at 10:44pm
All,

Due to yet another power cut this evening, both SAM and the MySQL server were unexpectedly switched-off during some essential updates to their respective application(s).

I'm obviously extremely both disappointed and frustrated at this but will resolve to fix the situation and bring things back to normal as soon as possible. Please bear in mind, I have to go to work tomorrow, so it's entirely possible that S.A. will not be brought back-up until tomorrow evening.

Again, apologies for this (totally unexpected) downtime but I'll work as hard and as quickly as possible to have everything back to normal.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by fish_head on 29th Sep, 2011 at 12:19am
Don't stress D!!  well wait.  Just have to listen to some cds for awhile  :)

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 29th Sep, 2011 at 8:22pm
Folks,

As mentioned above, due to yet another power outage yesterday evening, SAM (and the MySQL server) are in more than a bad state.

As I hope you can all already imagine, I have back-ups of everything that is important - music, databases, applications, license keys - the lot.

So, I'm now in the process of restoring everything back to it's former glory but please bear with me, I'm working as quickly and for as long as possible to achieve this.

Essentially, yesterday's power-cut occurred in the middle of core-application updates. As most of you are no doubt aware already, computers don't exactly appreciate power-cuts at the best of times but whilst in the middle of server-wide application upgrades, it really was a death blow, I'm afraid to say.

Please be patient, I shall post again in this thread when I have more information.

Many thanks.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by Andrew P on 30th Sep, 2011 at 1:25am
Argh! Don't you hate it when that happens?
We're hanging in there with you, D.
(I've got rehearsal CDs I should be listening to, anyway...)

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by JaneEyre on 30th Sep, 2011 at 5:52pm
:(

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 30th Sep, 2011 at 11:23pm
All,

The MySQL server has been rebuilt and is running swimmingly. I'm now about half the way through re-building SAM from scratch.

I'll post a full report once the work is complete but I hope to have everything back to normal very soon - working through the night if necessary.

Thanks again for all of your patience.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by pno74 on 2nd Oct, 2011 at 8:32pm
:)

Its up again  [smiley=bath.gif]

Thanks Dee :)

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 2nd Oct, 2011 at 9:30pm
Thanks Patrice. :)

I shall post a full update a little later on.

Oh, the nightmare ... :'(

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 2nd Oct, 2011 at 10:53pm
All,

As already posted, around the time of the annual maintenance that I performed last weekend there were two island-wide power cuts. The first lasting just under an hour and the second for around ten minutes. I'm certain that if these had occurred during the course of regular operations, it would have been no more than an inconvenience and I would have simply re-started things once power had been restored. As both of these power outages occurred during critical installations and updates to core applications, both SAM and the local SQL server were affected very seriously.

I initially tried to manually restore the failed installations and updates on both occasions but as the 'damage' involved (amongst other things) corrupt Windows registries to both machines, both machines were in a serious state of non-operation. If this had occurred on my own, personal PC I would have been less concerned but as SAM and the SQL server have to be running 24/7/365, it is essential that both are as stable as possible and won't be in a precarious situation which would lead to faults, crashes and so on at some point in the future.

So, on Thursday of last week I took the decision to completely re-format and install everything 'from scratch' on both machines. Due to the obvious precarious situation both machines were in and not being confident that even if I were to have both of them up-and-running again, they would not be online for very long and would no doubt be prone to instability.

Having worked constantly Thursday and Friday evening, I had both SAM and the SQL server back to a 'fresh state' and ready to run again. It was during the last stages of this fresh install that I noticed some instability on SAM. This was mostly due to a constantly failing connection - both to the local SQL server and to the internet itself. After a lot of investigation, it transpired that the Norton anti-virus software that I was using had become all but obsolete. Although still receiving regular updates to both the firewall and virus definitions it had trouble working well with both Windows XP service pack 3 (although there was an online fix for this on the Norton website) and the .NET framework which was installed.

Frantically searching online for solutions to both of these apparent (and potentially fatal situations, the longer SAM was left running in this state), I kept coming across the same answer - basically, time to upgrade both the firewall and anti-virus software running on SAM - something that I wasn't planning to do both money-wise and as a new software installation.

Anyway, I finally decided on the very latest version of Norton Internet Security and dutifully purchased a copy from their official website. Having bought software online numerous times in the past, I was (I hope understandably) very surprised to receive an e-mail from Norton, not containing a download link or license key but stating that their ordering system was currently offline and wouldn't be available for another four to eight hours. So, I took this window of time to get some much needed sleep and would continue the next day.

Frustration was heightened even more (bear in mind that the station had now been offline for several days) when I checked my e-mail account to discover that I hadn't received anything whatsoever from Norton. I signed-into my online account area which was also empty of any content - download links for the new version I had just bought etc. I checked my credit card balance online and sure enough, they had taken the money from my account, so I gave their 'help desk' a call. Having tried several times before receiving an answer, I was told that their ordering system was experiencing 'issues' and these would be resolved shortly. Not exactly news to me (I knew this already) and I pressed from an ETA as to when this would be - 'as soon as possible', was the answer.

Anyway, it wasn't until today (Sunday) that I finally received a response from Norton and installed this new software package. Once this was complete, I was finally in a position to put S.A. back online again.

Hopefully you'll all appreciate, this was a far from an ideal situation - two unexpected total power outages, corrupt servers, incompatible Internet Security software and a ridiculously long delay in processing an order for a new version. Anyway, things are back to normal and apologies once more for the downtime.

Enjoy.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by fish_head on 3rd Oct, 2011 at 12:36am
Thx for the work   :)   good to have SA 8-) 8-) back!!

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by Thomas S on 3rd Oct, 2011 at 2:46am
Many Thanks for your ongoing dedication.  So glad the music is back and became the focus again...

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by zappa on 3rd Oct, 2011 at 5:32pm
great work, and thx for detailed information [smiley=tekst-toppie.gif] [smiley=tekst-toppie.gif] [smiley=tekst-toppie.gif]

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by zappa on 3rd Oct, 2011 at 8:38pm
is there anybody aout ther ???? :( [smiley=DJ.gif]

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by ProgMaster on 3rd Oct, 2011 at 9:49pm
Thanks for all of the comments, guys. Appreciated. [smiley=dankk2.gif]

I have to confess to having had better weeks ... :'(

Anyway, all back to normal.

Darran

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by zappa on 3rd Oct, 2011 at 9:51pm
:D :D :D :D :D :D :D :D :D :D :D :D :D :D

Title: Re: Scheduled Maintenance - 24th - 25th September
Post by Yes Phreak on 3rd Oct, 2011 at 10:15pm
Thanks for your work Darren.      We all appreciated it. 8-) 8-) 8-)

Stellar Attraction Discussion Forum » Powered by YaBB 2.5 AE!
YaBB Forum Software © 2000-2010. All Rights Reserved.