Mar 12, 23:00:57 GMT+0
Investigating -
We are currently investigating packet loss affecting all services in LV01 (Las Vegas, NV).
Mar 13, 00:44:29 GMT+0
Monitoring -
We implemented a fix and are currently monitoring the result..
Mar 13, 01:00:00 GMT+0
Resolved -
This incident has been resolved..
Feb 28, 12:30:00 GMT+0
Identified -
We are planning for a scheduled maintenance on Wednesday, February 28th, between 7:30am and 10am EST. This maintenance will impact all customers on EPYCK01.TPA01\. We will be powering off this node in order to upgrade the memory and disk capacity. .
Feb 28, 12:30:01 GMT+0
Identified -
Maintenance is now in progress.
Feb 28, 14:01:20 GMT+0
Completed -
Maintenance has been completed, all instances are rebooting now..
Feb 15, 16:23:33 GMT+0
Resolved -
This incident has been resolved..
Feb 14, 15:18:44 GMT+0
Investigating -
We are working to resolve the partial network outage in Las Vegas. .
Feb 14, 20:23:47 GMT+0
Identified -
We have narrowed the issue down and are working with our upstream providers to restore BGP connectivity. .
Feb 14, 23:55:55 GMT+0
Identified -
We are continuing to work on a fix for this incident. We have narrowed the issue down to a potential problem area and are awaiting responses from upstream providers..
Feb 15, 00:55:41 GMT+0
Identified -
We have identified the issue and are waiting on our upstream provider to make the changes necessary to resolve this. We expect a solution within the next few hours..
Feb 15, 06:05:11 GMT+0
Monitoring -
We implemented a fix and are currently monitoring the result..
Feb 10, 01:51:28 GMT+0
Investigating -
We are performing maintenance to resolve the issue causing slow download speeds over IPv4.
Feb 10, 17:44:11 GMT+0
Investigating -
We are still investigating this incident. We appreciate your patience. We will be restarting some core networking devices in an attempt to debug this issue..
Feb 12, 18:34:13 GMT+0
Investigating -
We are continuing to investigate this incident and hope to achieve a solution as soon as possible. .
Feb 13, 01:02:51 GMT+0
Identified -
We have identified the issue to be a faulty piece of equipment with one of our upstream network providers. We are actively communicating with them about the issue and are waiting for them to replace it. We do not have an ETA on that replacement currently but we can expect an update in the next 24-48 hours.
We are currently adjusting routes in order to provide as much throughput as possible which is causing some intermittent downtime as we work towards a solution. .
Feb 13, 18:16:14 GMT+0
Monitoring -
We have re-routed traffic via HE to avoid the problematic upstream for now, until the failed equipment has been replaced. Download speeds are back to normal now..
Feb 14, 00:34:14 GMT+0
Resolved -
This incident has been resolved..
Jan 29, 21:48:02 GMT+0
Investigating -
We are currently investigating network issues in **DET01**. This may result in brief network outages as we debug this issue. .
Jan 29, 23:15:05 GMT+0
Monitoring -
We implemented a fix and are currently monitoring the result..
Jan 29, 23:36:28 GMT+0
Resolved -
This incident has been resolved..
Nov 14, 19:45:09 GMT+0
Identified -
We are shutting down and extracting the servers from Miami now. .
Nov 14, 18:00:00 GMT+0
Identified -
We are planning on migrating our Miami, FL presence to a new datacenter in Tampa, FL between Tuesday, November 14th and Wednesday, November 15th 2023.
This migration will involve a few hours of downtime as we physically power off the machines and drive them over to Tampa, and get them racked in our new datacenter space.
Please expect a total of approximately 6 hours of downtime during this time period.
This move will give us more space to expand and allow us to bring Hostodo Gen2 to the Florida region.
Thank you for your understanding and feel free to reach out to us with any questions..
Nov 14, 18:00:01 GMT+0
Identified -
Maintenance is now in progress.
Nov 14, 20:47:23 GMT+0
Identified -
We are transporting the nodes back to Tampa now. Expect to be online around 10PM EST. .
Nov 15, 02:50:06 GMT+0
Identified -
Servers will be back online momentarily..
Nov 17, 00:35:22 GMT+0
Completed -
Maintenance has completed successfully..
Oct 30, 19:00:00 GMT+0
Identified -
We are planning to perform upgrades to our core network routers in **LV01** Las Vegas, NV and **DET01** Detroit, MI on **Monday, October 30th **between **3:00PM to 6:00PM EST**. Please expect about 30 minutes of downtime during this period. .
Oct 30, 22:00:00 GMT+0
Identified -
Maintenance is still in progress.
Oct 30, 23:11:10 GMT+0
Completed -
Maintenance has completed successfully..
Oct 30, 19:00:01 GMT+0
Identified -
Maintenance is now in progress.
Oct 3, 18:02:08 GMT+0
Identified -
We are aware of broken IPv6 in Las Vegas, NV and are working on resolving this issue. .
Oct 4, 01:58:24 GMT+0
Identified -
We are continuing to work on a fix for this incident..
Oct 12, 18:02:00 GMT+0
Resolved -
This incident has been resolved..
Oct 1, 05:47:12 GMT+0
Completed -
K4.LV is now back online. .
Sep 30, 16:00:00 GMT+0
Identified -
We are planning for a scheduled maintenance during this time. This is a continuation of last nights maintenance period. We will be switching over to the new network on Saturday, Sep 30th 2023 between 9AM to 2PM Pacific
We will also be resolving an issue causing K4LV to be unresponsive..
Sep 30, 16:00:01 GMT+0
Identified -
Maintenance is now in progress.
Sep 30, 21:05:17 GMT+0
Identified -
Maintenance is still in progress. .
Sep 30, 23:23:05 GMT+0
Identified -
We have replaced the failed boot drive causing K4LV to become unresponsive. We are currently working on getting all VMs back online within the next hour..
Sep 29, 20:00:01 GMT+0
Identified -
Maintenance is now in progress.
Sep 30, 02:06:21 GMT+0
Completed -
Maintenance will continue tomorrow morning. We are aware of an issue with VPSs on K4LV, we are working on it..
Sep 29, 20:00:00 GMT+0
Identified -
We are planning for a scheduled maintenance on September 29th, 2023.
We will be upgrading our network equipment and begin using a new network transit blend which will dramatically improve latency.
During this maintenance period we will also be performing maintenance on our Las Vegas dedicated servers. Please expect your dedicated server to be shutdown and rebooted during this time.
We will be doing this work between 1:00PM to 6:00PM Pacific time (4:00 PM to 9:00PM EST)
.
Sep 30, 01:00:00 GMT+0
Completed -
Maintenance has completed successfully.
Jun 30, 13:00:00 GMT+0
Identified -
**When:** Friday, June 30, 2023 9AM Pacific -> Saturday July 1, 2023 11PM Pacific
**Where:** Las Vegas, NV
**What:** We will be performing a physical migration of our Las Vegas, NV infrastructure to a new rack within the same datacenter. There will also be some networking changes taking place during this maintenance period. We will be aiming for at most 2 hours of downtime.
**How Long:** Each node will be down for approximately 2 hours at some point during this maintenance period..
Jun 30, 13:00:01 GMT+0
Identified -
Maintenance is now in progress.
Jul 1, 19:06:40 GMT+0
Identified -
UPDATE Saturday July 1st
The maintenance was successful however the network change announced previously was not completed and will be completed at a different time.
k3.lv.hostodo.com has some extended downtime but will be fixed shortly (12PM PST)
All other nodes should be online..
Jul 2, 03:00:00 GMT+0
Completed -
Maintenance has completed successfully.
Dec 1, 05:02:05 GMT+0
Completed -
Maintenance has completed successfully..
Dec 1, 03:17:17 GMT+0
Identified -
Maintenance is now in progress. We have extended the maintenance period to midnight EST. .
Dec 1, 02:00:00 GMT+0
Identified -
We will be performing some maintenance on NVME-K3.LV and NVME-K10.LV between 9PM and 11PM Eastern time. Each node will go down for approximately 20 minutes..
Oct 17, 19:43:36 GMT+0
Resolved -
We just resolved the issue!.
Oct 17, 19:32:29 GMT+0
Investigating -
We received an alert that NVMe KVM node NVMEK5LV went down as of 3:30PM EST 10/17/2021
We are investigating this issue..
Oct 3, 20:05:13 GMT+0
Investigating -
We are currently investigating this incident..
Oct 3, 20:05:42 GMT+0
Identified -
LV network is down due to a failing line card on our router. We are dispatching a technician to fix the issue now..
Oct 4, 00:36:58 GMT+0
Resolved -
We just resolved the issue!.
Oct 3, 20:25:00 GMT+0
Monitoring -
We implemented a fix and currently monitoring the result..
Sep 24, 23:23:31 GMT+0
Resolved -
The DDoS attack has been mitigated. .
Sep 24, 15:06:19 GMT+0
Investigating -
We are currently investigating this incident..
Sep 24, 15:20:02 GMT+0
Monitoring -
We implemented a fix and currently monitoring the result..
Sep 24, 15:19:33 GMT+0
Identified -
We are being hit by a Denial of Service attack on the website. We are working on mitigation..
Sep 16, 13:36:25 GMT+0
Investigating -
We are currently investigating this incident..
Sep 16, 15:51:36 GMT+0
Monitoring -
We implemented a fix and currently monitoring the result..
Sep 16, 16:14:26 GMT+0
Resolved -
We just resolved the issue!.
Sep 5, 02:23:16 GMT+0
Completed -
Maintenance has completed successfully..
Sep 4, 19:00:00 GMT+0
Identified -
We will be performing datacenter maintenance in our Las Vegas, NV datacenter on Saturday September 4th between 1:00 PM and 5:00 PM PST.
We will be moving our nodes to a new rack now that we have used up all of the existing space in our existing rack.
We expect total downtime of ~ 30 minutes per node.
.
Sep 4, 22:08:09 GMT+0
Identified -
All nodes are back online! Maintenance still in progress, so there may be some intermittent downtime, but the core work has been completed..
Sep 4, 20:50:00 GMT+0
Identified -
Maintenance is now in progress.
.
Sep 3, 21:49:57 GMT+0
Investigating -
We are currently investigating this incident..
Sep 3, 22:30:00 GMT+0
Resolved -
We just resolved the issue!.
Aug 12, 00:34:00 GMT+0
Resolved -
We just resolved the issue!.
Aug 11, 22:05:40 GMT+0
Investigating -
We are currently investigating this incident..
Aug 11, 22:36:00 GMT+0
Monitoring -
We implemented a fix and currently monitoring the result..
Jul 28, 13:40:36 GMT+0
Investigating -
NVMe virtual machines hosted on NVME-K3.LV are currently inaccessible. We are currently investigating this incident. We will have this resolved today..
Jul 28, 18:26:30 GMT+0
Identified -
We have dispatched a technician to the datacenter to investigate. .
Jul 28, 22:21:52 GMT+0
Resolved -
We just resolved the issue..
Jul 21, 14:39:36 GMT+0
Investigating -
NVME-K7.LV has a read only boot drive. Most VMs are still online as long as they weren't rebuilt today.
We are working on a solution for this. .
Jul 22, 01:14:51 GMT+0
Identified -
Currently on location to replace the drive.
Node will be down temporarily..
Jul 22, 02:05:26 GMT+0
Resolved -
Boot drive replaced. User VM's are back online..
Jun 16, 00:33:00 GMT+0
Resolved -
We will be performing network maintenance in our Miami datacenter tomorrow 06/16/2021.
Please expect up 2 hours of downtime as we complete this maintenance.
This work involves replacing our current Linux based router with a Juniper MX204. This greatly increases our network capacity and also prepares us for the arrival of our additional bandwidth provider.
You may have noticed some packetloss in Miami lately. This new router will solve those issues.
.