ARIES server crashed a few hours ago and prompted a File System Check (FSCK) upon reboot. The FSCK completed but has cause some problems, namely some corruption of the critical /usr directory that hold many OS files and libraries. We are attempting to restore this partition from a rescue CD. Will update you when we know more.

UPDATE: Most services have been restored and the main two things left are:

1) Email – working to get this back up right now

2) PHP – Some modules need to be compiled in. This will be done after mail is fixed.

[b]UPDATE:[/b] Aries should be 95% back to normal. Websites are up, mail is flowing and Plesk is working. We still have a few things to install (few perl/PHP modules) but these things should affect most people. If you notice something wrong with any sites or email please let us know immediately by emailing support@hostnexus.com or submitting a ticket via the online Helpdesk.

Many thanks for your patience!

At 3am tonight 17th November EST we will be performing a memory upgrade on the VPS2 node. Please allow 30 minutes downtime for this upgrade.

Cancer server is reporting a failed drive in its 4 drive RAID 10 array. Although this server has a hot-swappable chasis and ability to change a drive without downtime we want to perform some deeper maintenance on th RAID controller to make sure everything is okay (as the server is a few years old and has had a few failed drives in the past). Therefore we are scheduling a 30 minute downetime window to take the server down and open it up to inspect the RAID array. This will happen at 3am Wednesday 19th October (early hours of Wednesday morning US EST).

Incredible as though it may seem Proton has crashed again. All efforts to boot the machine have been unsuccessful. All hardware was swapped, server was swapped, chdisk ran on the RAID array. The server was completely clean, full malware and anti-virus scan just completed, Windows was up to date, yet the server just blue-screened every time – the sign of a corrupt OS.

We are restoring once more. When the server comes back up we should probably migrate everyone to other servers and through Proton out the window.

UPDATE: We are in the last stages of the Proton restore. We belive we have found a quicker method to complete this kind of issue and if everything went well the server should be running all services in a few hours.

UPDATE: As of about 12 hours ago all services have been restored on Proton. If you find any domains with problems it will probably be a simple permissions issue so please contact support immediately and we’ll fix for you.

Thanks to all for your patience. If you’d like to be moved to a different server please let us know and we’ll get that done.

The data center’s utility company will be shutting down one of the two feeds that power the building on Tuesday, October 4th from 17:00 EST until Wednesday morning. During
this time the power to the building will be on a single feed and this will not affect any service. In the event of the remaining feed failure the UPS and emergency generators will continue to supply uninterrupted power to the facility.

We apologize for the short notice as the data center has just informed us and we have no control/choice in the matter. But it does look as though the data center will function normally during the maintenance window.

We thank you for your continued support.

Parallels finally released a new 9.5 version that supports Coldfusion 9! We will be upgrading Magnon first this Saturday 2am 27th August 2011. Plesk will need to be upgraded first and then CF will be done. We estimate about an hour downtime for most services and 2-3 hours downtime for Coldfusion while this upgrade takes place. As always please allow 4 hours in case there are complications.

UPDATE: We ran into a few unexpected issues and decided to postpone the upgrade for one week. This week gives us time to research the errors we found and discuss internally how to proceed with the upgrade. Rescheduling this to Saturday 2am 3rd September.

On Monday 15th August 4pm EST we will be upgrading CF on Atom to 9.0.1. This should be a painless upgrade with very little downtime but please allow 30-60 minutes in case there are unforseen complications.

Electron server crashed about an hour ago and all efforts to boot the server have failed. We’ve tried replacing all the hardware yet it still fails to boot. At this stage all we can do is copy data to an external device, reload the OS and then copy the data back.

From experience this can take 24 hours (or longer). My apologies for this downtime event.

UPDATE: So after 12 hours we have the new server ready and data has cloned to a USB device. As this is a RAID server the data has been be cloned in this way and we can’t just copy direct from old RAID array to new RAID array as the data needs to be on one device (not 4). Yes, we do have backups on a remote server but even on a private 1000mps network it would take too long to transfer the data. It is quicker to clone the data then connect the USB enclosure direct to the motherboard and restore. This restore is now taking place and should be done within 12 hours. Whilst the cloning took place we reloaded the OS and installed Plesk and Coldfusion so we’ve been moving as quickly as possible.

Thanks for your patience thus far.

UPDATE: The server is almost restored. All data is now successfully copied over and we are now completing the final tasks to get everything up. We estimate this should be done in 2 hours.

Many thanks to all for your patience. You’ve all been fantastic and very understanding.

UPDATE: We are running a tool to reconfigure all the permissions and this is taking longer than expected. We do expect it to finish within an hour though.

UPDATE: All domains are up now. We have an issue with Access based DSNs and are working to correct that. Please note that archived mail has not yet been copied over and that will start now. Mail is working however. Many thanks for your patience.