Resources

The outage has impacted approximately 9000 severs according to the official notification thread. A key issue is that EV1 servers DNS systems were impacted. As of time of this post, they are not 100% functional.

DNS Issues
We are starting to see other clients with servers not directly impacted by the outage start to have issues. The old EV1 DNS servers (ns5/ns6.ev1servers.net) are not operational. If you have a domain or nameserver domain depending on these servers, your sites or DNS may start to fail. This can potentially impact 1000's more servers as DNS expires.

Depending on your number of domains, there are multiple options for you to resolve this issue.

• Use your registrar's DNS services if provided.
• Use a third-party DNS service
• Use a secondary server you have (or purchase)

If you have a few domains, the fastest resolution may just be using your registrar's DNS. This will involve setting up domain records for your site and updating the nameservers at your registrar. Changes to nameservers get sent to the root servers very quickly for com/net domains. So these changes should take effect very quickly.

If you have many domains, you may want to consider a service like DNS Made Easy. The do have a template wizard which will allow you to create a template and then bulk-add domains. This could be a pretty quick solution for you. It does take sometime (30+ minutes) for your account and the domains to become active.

If you have a secondary server, then consider moving your DNS to that system. Make sure that the DNS is in no way connected to ThePlanet. For example, I would assure that the domain names you are using have their DNS hosted elsewhere.

From what I've read so far, they hope to have DNS fully operational by noon Sunday, June 1st.

Down Server
If you have a server impacted by these events, then there is little you can do without offsite backups. If you do not have offsite backups, then you must wait until ThePlanet restores operations. They are hoping for midday today but I expect it to take longer. This is a major issue and many people (fire, utility, building engineers, data center engineers) have to sign off before the power can be restored.

If you do have offsite backups, then I strongly urge you to look into secondary provider options now. You will want to consider the steps required to recover:

1. Find a provider that can handle your requests and can deploy rapidly.



2. What type of backups do you have? Make sure you match OS, Control Panel and other items as closely as possible.



3. How much data do you have? Consider that for large backups, just transferring the data can take 2-3 hours.



4. Once the data is restored, you will have to update your DNS. The way to do this will depend on your DNS setup.



5. There may be post-restore issues involving PHP settings, Apache settings, etc., so plan on having your support staff available to deal with these issues.

IP Changes
IP changes always have consequences. Consider that it will take some hours for the DNS changes to propagate. If your IPs change, also consider that 3rd party vendors that use IP information may need to be updated. Firewalls may require modification, etc. There could also be SEO consequences for changing your IPs.

Waiting Game
You should decided now how long you can wait before implementing a recover scheme. Consider the time it will take to provision, restore and switch up DNS. That typically is a 8-24 hour process depending on the size of the server, domain tld and other factors.

You will need to pick a time at which you initiate recovery or decide just to wait.

Also, I suggest you just pay attention to the official information. Posts on other areas can get way out of control and fueled only by speculation. I've been through several major data center outages and the rumors are often worse than the actual events.