<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
  <channel>
    <title>dec0de | 1406557 b.c. ltd. Status - Incident history</title>
    <link>https://1406557-bc-ltd.instatus.com</link>
    <description>dec0de | 1406557 b.c. ltd.</description>
    <pubDate>Sun, 21 Jan 2024 20:00:00 +0000</pubDate>
    
<item>
  <title>Hardware</title>
  <description>
    Type: Maintenance
    

    Affected Components: Kelowna, BC, Canada
    Jan 21, 20:00:00 GMT+0 - Identified - We are planning for a scheduled maintenance during that time. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    
    <p><strong>Affected Components:</strong> </p>
    &lt;p&gt;&lt;small&gt;Jan &lt;var data-var=&#039;date&#039;&gt; 21&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;20:00:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are planning for a scheduled maintenance during that time..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sun, 21 Jan 2024 20:00:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/clrn49k6i34300beoigm4llstu</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/clrn49k6i34300beoigm4llstu</guid>
</item>

<item>
  <title>Network Upgrade</title>
  <description>
    Type: Maintenance
    Duration: 4 hours

    Affected Components: Phone System, Portal / WHMCS, Kelowna, BC, Canada
    Dec 23, 20:15:00 GMT+0 - Identified - We are planning for a scheduled maintenance during that time. Dec 24, 00:15:00 GMT+0 - Completed - Maintenance has completed successfully Dec 23, 20:15:01 GMT+0 - Identified - Maintenance is now in progress 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> , , </p>
    &lt;p&gt;&lt;small&gt;Dec &lt;var data-var=&#039;date&#039;&gt; 23&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;20:15:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are planning for a scheduled maintenance during that time..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Dec &lt;var data-var=&#039;date&#039;&gt; 24&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;00:15:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance has completed successfully.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Dec &lt;var data-var=&#039;date&#039;&gt; 23&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;20:15:01&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress.&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sat, 23 Dec 2023 20:15:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/clqhi2ky8138224bkn4b362el5q</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/clqhi2ky8138224bkn4b362el5q</guid>
</item>

<item>
  <title>Storage Incident</title>
  <description>
    Type: Incident
    

    Affected Components: Phone System, Portal / WHMCS
    Nov 7, 02:46:04 GMT+0 - Investigating - We are currently investigating this incident. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    
    <p><strong>Affected Components:</strong> , </p>
    &lt;p&gt;&lt;small&gt;Nov &lt;var data-var=&#039;date&#039;&gt; 7&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;02:46:04&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Investigating&lt;/strong&gt; -
  We are currently investigating this incident..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Tue, 7 Nov 2023 02:46:04 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clonqbswn8010b3oqrimv8wlh</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clonqbswn8010b3oqrimv8wlh</guid>
</item>

<item>
  <title>New Storage Node</title>
  <description>
    Type: Maintenance
    Duration: 2 hours

    Affected Components: Phone System, Portal / WHMCS
    Oct 28, 01:00:01 GMT+0 - Identified - Maintenance is now in progress Oct 28, 03:00:00 GMT+0 - Completed - Maintenance has completed successfully Oct 28, 01:00:00 GMT+0 - Identified - We will be installing an additional storage node and configuring it. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours</p>
    <p><strong>Affected Components:</strong> , </p>
    &lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:00:01&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;03:00:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance has completed successfully.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:00:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We will be installing an additional storage node and configuring it..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sat, 28 Oct 2023 01:00:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/clo8qtbid55028bpojyol8i2iv</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/clo8qtbid55028bpojyol8i2iv</guid>
</item>

<item>
  <title>Cloud Maintenance</title>
  <description>
    Type: Maintenance
    

    
    Oct 25, 00:00:01 GMT+0 - Identified - Maintenance is now in progress Oct 25, 00:00:00 GMT+0 - Identified - We will be performing maintenance to our primary YLW cloud cluster. We are not expecting any downtime. Oct 25, 00:00:00 GMT+0 - Completed - Maintenance is now in progress. This was completed on the 24th at 10PM. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    
    
    &lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 25&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;00:00:01&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 25&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;00:00:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We will be performing maintenance to our primary YLW cloud cluster. We are not expecting any downtime..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 25&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;00:00:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance is now in progress. This was completed on the 24th at 10PM..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Wed, 25 Oct 2023 00:00:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/clo4pl59k6865b9ofofgwicbv</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/clo4pl59k6865b9ofofgwicbv</guid>
</item>

<item>
  <title>Network Maintenance</title>
  <description>
    Type: Maintenance
    Duration: 4 hours

    Affected Components: Phone System, Portal / WHMCS, Kelowna, BC, Canada, Seattle, WA, USA
    Oct 22, 01:30:01 GMT+0 - Identified - Maintenance is now in progress Oct 22, 01:30:00 GMT+0 - Identified - We are planning for a scheduled maintenance during that time. We are upgrading our core links to 10Gigabits per second.  Oct 22, 05:30:00 GMT+0 - Completed - Maintenance has completed successfully Oct 22, 04:29:27 GMT+0 - Identified - Maintenance is now in progress. We are making good progress with migrating to 10Gigabit backbone for main switches and routers. Thank you for your patience.  
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 4 hours</p>
    <p><strong>Affected Components:</strong> , , , </p>
    &lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 22&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:30:01&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 22&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are planning for a scheduled maintenance during that time. We are upgrading our core links to 10Gigabits per second. .&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 22&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;05:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance has completed successfully.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 22&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;04:29:27&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress. We are making good progress with migrating to 10Gigabit backbone for main switches and routers. Thank you for your patience. .&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sun, 22 Oct 2023 01:30:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/clo0psrh634271boocolm9zh1t</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/clo0psrh634271boocolm9zh1t</guid>
</item>

<item>
  <title>Phone System Operational</title>
  <description>
    Type: Incident
    

    Affected Components: Phone System
    Sep 20, 18:04:16 GMT+0 - Resolved - This incident has been resolved. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    
    <p><strong>Affected Components:</strong> </p>
    &lt;p&gt;&lt;small&gt;Sep &lt;var data-var=&#039;date&#039;&gt; 20&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;18:04:16&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Wed, 20 Sep 2023 18:04:16 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clms1zq5453291bnodbol2s9zs</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clms1zq5453291bnodbol2s9zs</guid>
</item>

<item>
  <title>Storage Maintenance</title>
  <description>
    Type: Incident
    Duration: 2 days, 21 hours and 1 minute

    
    Aug 21, 15:07:37 GMT+0 - Identified - Backups for game servers have temporarily been disabled while we re-work a solution for them. Please do not make any backups, as they will likely fail, and cause more backups as of this time. Oct 22, 00:11:09 GMT+0 - Resolved - This incident has been resolved. Aug 21, 03:10:36 GMT+0 - Identified - We are continuing to work on a fix for this incident. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 2 days, 21 hours and 1 minute</p>
    
    &lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 21&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;15:07:37&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Backups for game servers have temporarily been disabled while we re-work a solution for them. Please do not make any backups, as they will likely fail, and cause more backups as of this time..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Oct &lt;var data-var=&#039;date&#039;&gt; 22&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;00:11:09&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 21&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;03:10:36&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are continuing to work on a fix for this incident..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Mon, 21 Aug 2023 03:10:36 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/cllkauwyj13680baonainp5ovm</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/cllkauwyj13680baonainp5ovm</guid>
</item>

<item>
  <title>Wild Fires</title>
  <description>
    Type: Incident
    Duration: 11 days

    Affected Components: Portal / WHMCS, Phone System, Kelowna, BC, Canada
    Aug 20, 03:30:00 GMT+0 - Monitoring - We have restored normal power. We are currently monitoring the fire situation in the region. Aug 31, 01:03:00 GMT+0 - Resolved - This incident has been resolved. Aug 20, 01:03:08 GMT+0 - Identified - We are currently running on backup power, our datacenter is currently surrounded by wild fires in the region. More information can be found at https://wildfiresituation.nrs.gov.bc.ca/map regarding the Kelowna/West Kelowna fires. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 11 days</p>
    <p><strong>Affected Components:</strong> , , </p>
    &lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 20&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;03:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Monitoring&lt;/strong&gt; -
  We have restored normal power. We are currently monitoring the fire situation in the region..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 31&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:03:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 20&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:03:08&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are currently running on backup power, our datacenter is currently surrounded by wild fires in the region. More information can be found at https://wildfiresituation.nrs.gov.bc.ca/map regarding the Kelowna/West Kelowna fires..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sun, 20 Aug 2023 01:03:08 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clliqv4kl63684bbnb2qbf8ixo</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clliqv4kl63684bbnb2qbf8ixo</guid>
</item>

<item>
  <title>Power Failure / Outage</title>
  <description>
    Type: Incident
    Duration: 43 minutes

    Affected Components: Portal / WHMCS, Phone System, Kelowna, BC, Canada
    Aug 1, 18:26:57 GMT+0 - Investigating - We are currently investigating this incident. We had switched services to be on generator power for a power upgrade, however, the rack is currently unresponsive.  Aug 1, 19:10:00 GMT+0 - Resolved - This incident has been resolved. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 43 minutes</p>
    <p><strong>Affected Components:</strong> , , </p>
    &lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 1&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;18:26:57&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Investigating&lt;/strong&gt; -
  We are currently investigating this incident. We had switched services to be on generator power for a power upgrade, however, the rack is currently unresponsive. .&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Aug &lt;var data-var=&#039;date&#039;&gt; 1&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;19:10:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Tue, 1 Aug 2023 18:26:57 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clksmsbly9405lnoaxksdeaxt</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clksmsbly9405lnoaxksdeaxt</guid>
</item>

<item>
  <title>Urgent Security Upgrade</title>
  <description>
    Type: Incident
    Duration: 8 hours

    Affected Components: Portal / WHMCS
    Jul 30, 09:10:28 GMT+0 - Investigating - We are performing urgent security updates to an isolated server. Jul 30, 17:10:00 GMT+0 - Resolved - This incident has been resolved. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 8 hours</p>
    <p><strong>Affected Components:</strong> </p>
    &lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 30&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;09:10:28&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Investigating&lt;/strong&gt; -
  We are performing urgent security updates to an isolated server..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 30&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;17:10:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sun, 30 Jul 2023 09:10:28 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clkp80yg456813btlqodw89mmc</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clkp80yg456813btlqodw89mmc</guid>
</item>

<item>
  <title>Vultr Issue</title>
  <description>
    Type: Incident
    Duration: 3 days

    Affected Components: Seattle, WA, USA
    Jul 26, 04:38:00 GMT+0 - Identified - We are currently investigating this incident. It appears Vultr had lost our routes during the maintainance last night. We have opened a ticket with them... Jul 26, 19:33:39 GMT+0 - Monitoring - We have opened a ticket with our upstream regarding the timeouts and increased latency. It has been escalated to their network engineering team. We are awaiting a response.  Jul 26, 15:26:10 GMT+0 - Monitoring - Vultr implemented a fix and are currently monitoring the result. Jul 27, 15:09:57 GMT+0 - Monitoring - Our upstream implemented a fix which broke another portion of the network. We have reached out and they have repaired it. Currently continuing to monitor for further issues. Jul 29, 04:38:00 GMT+0 - Resolved - This incident has been resolved. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 3 days</p>
    <p><strong>Affected Components:</strong> </p>
    &lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 26&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;04:38:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are currently investigating this incident. It appears Vultr had lost our routes during the maintainance last night. We have opened a ticket with them....&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 26&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;19:33:39&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Monitoring&lt;/strong&gt; -
  We have opened a ticket with our upstream regarding the timeouts and increased latency. It has been escalated to their network engineering team. We are awaiting a response. .&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 26&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;15:26:10&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Monitoring&lt;/strong&gt; -
  Vultr implemented a fix and are currently monitoring the result..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 27&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;15:09:57&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Monitoring&lt;/strong&gt; -
  Our upstream implemented a fix which broke another portion of the network. We have reached out and they have repaired it. Currently continuing to monitor for further issues..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 29&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;04:38:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Wed, 26 Jul 2023 04:38:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clkjtyswa733642rcoku96ls584</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clkjtyswa733642rcoku96ls584</guid>
</item>

<item>
  <title>Quorum Lost</title>
  <description>
    Type: Incident
    

    
    Jul 15, 05:38:06 GMT+0 - Resolved - An intern made the cluster loose quorum. This has been resolved. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    
    
    &lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 15&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;05:38:06&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  An intern made the cluster loose quorum. This has been resolved..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sat, 15 Jul 2023 05:38:06 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clk3ku2ya123360bboac34irbvp</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clk3ku2ya123360bboac34irbvp</guid>
</item>

<item>
  <title>UPS Maintenance</title>
  <description>
    Type: Maintenance
    Duration: 2 hours

    Affected Components: Portal / WHMCS, Phone System
    Jul 12, 14:30:00 GMT+0 - Completed - Maintenance has completed successfully Jul 12, 12:30:01 GMT+0 - Identified - Maintenance is now in progress Jul 12, 12:30:00 GMT+0 - Identified - We are planning for a scheduled maintenance during that time. We will be replacing the batteries in several UPSes, as well as switching over some equipment to the newly replaced UPSes. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours</p>
    <p><strong>Affected Components:</strong> , </p>
    &lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 12&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;14:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance has completed successfully.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 12&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;12:30:01&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 12&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;12:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We are planning for a scheduled maintenance during that time. We will be replacing the batteries in several UPSes, as well as switching over some equipment to the newly replaced UPSes..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Wed, 12 Jul 2023 12:30:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/cljyjxivc135573allrkow1ctjh</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/cljyjxivc135573allrkow1ctjh</guid>
</item>

<item>
  <title>Game servers failing to start due to low storage</title>
  <description>
    Type: Incident
    

    
    Jul 4, 19:25:36 GMT+0 - Resolved - We implemented a fix and are currently monitoring the result. Our primary storage server had lost it&#039;s mount to the wings node. As a result, backups were made to the local disk, instead of the storage server, filling up the main root disk. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    
    
    &lt;p&gt;&lt;small&gt;Jul &lt;var data-var=&#039;date&#039;&gt; 4&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;19:25:36&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  We implemented a fix and are currently monitoring the result. Our primary storage server had lost it&#039;s mount to the wings node. As a result, backups were made to the local disk, instead of the storage server, filling up the main root disk..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Tue, 4 Jul 2023 19:25:36 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/cljoojvqp60734tgodhym9szq5</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/cljoojvqp60734tgodhym9szq5</guid>
</item>

<item>
  <title>Wings Migration | Node #1 -&gt; #3</title>
  <description>
    Type: Maintenance
    Duration: 3 hours and 20 minutes

    
    Jun 10, 23:58:44 GMT+0 - Identified - Maintenance is in progress. Expected to be completed before 6PM PST. Jun 11, 02:05:14 GMT+0 - Completed - Maintenance has completed successfully. Confirmed systems are operational again. Jun 10, 22:45:00 GMT+0 - Identified - We will be migration all game servers from Wings Node #1 -&gt; #3. This will cause both nodes to go offline as we will be moving physical storage media between the servers. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 3 hours and 20 minutes</p>
    
    &lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 10&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;23:58:44&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is in progress. Expected to be completed before 6PM PST..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 11&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;02:05:14&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance has completed successfully. Confirmed systems are operational again..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 10&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;22:45:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We will be migration all game servers from Wings Node #1 -&gt; #3. This will cause both nodes to go offline as we will be moving physical storage media between the servers..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sat, 10 Jun 2023 22:45:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/cliqitqxp302399wvncav2ogh6j</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/cliqitqxp302399wvncav2ogh6j</guid>
</item>

<item>
  <title>UPS &amp; Network Upgrade</title>
  <description>
    Type: Maintenance
    Duration: 2 hours and 4 minutes

    Affected Components: Portal / WHMCS, Seattle, WA, USA, Phone System
    Jun 10, 21:33:33 GMT+0 - Completed - Upgraded card in core router, and installed UPS. Jun 10, 19:30:00 GMT+0 - Identified - We will be performing maintenance to our core network infrastructure to move it onto a new power source. We will also be installing a 10Gig card for our core router in preparation.  We will also be replacing a failing drive in our the core storage. Jun 10, 19:30:00 GMT+0 - Identified - We will be performing maintenance to our core network infrastructure to move it onto a new power source. We will also be installing a 10Gig card for our core router in preparation.  We will also be replacing a failing drive in our the core storage. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours and 4 minutes</p>
    <p><strong>Affected Components:</strong> , , </p>
    &lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 10&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;21:33:33&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Upgraded card in core router, and installed UPS..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 10&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;19:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We will be performing maintenance to our core network infrastructure to move it onto a new power source. We will also be installing a 10Gig card for our core router in preparation.  We will also be replacing a failing drive in our the core storage..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 10&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;19:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  We will be performing maintenance to our core network infrastructure to move it onto a new power source. We will also be installing a 10Gig card for our core router in preparation.  We will also be replacing a failing drive in our the core storage..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sat, 10 Jun 2023 19:30:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/clilx3ufo325778bboer98goyyv</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/clilx3ufo325778bboer98goyyv</guid>
</item>

<item>
  <title>Network Issue</title>
  <description>
    Type: Incident
    Duration: 2 days, 20 hours and 32 minutes

    Affected Components: Portal / WHMCS, Seattle, WA, USA
    Jun 1, 01:41:52 GMT+0 - Investigating - We are currently investigating this incident. Jun 1, 02:41:00 GMT+0 - Monitoring - We are currently investigating this incident. Jun 1, 02:50:23 GMT+0 - Resolved - This incident has been resolved. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 2 days, 20 hours and 32 minutes</p>
    <p><strong>Affected Components:</strong> , </p>
    &lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 1&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:41:52&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Investigating&lt;/strong&gt; -
  We are currently investigating this incident..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 1&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;02:41:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Monitoring&lt;/strong&gt; -
  We are currently investigating this incident..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;Jun &lt;var data-var=&#039;date&#039;&gt; 1&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;02:50:23&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Thu, 1 Jun 2023 02:41:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clich0scx55773b2n2gl3f9y3h</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clich0scx55773b2n2gl3f9y3h</guid>
</item>

<item>
  <title>Router Failure</title>
  <description>
    Type: Incident
    Duration: 1 day, 15 hours and 10 minutes

    Affected Components: Seattle, WA, USA, Portal / WHMCS
    May 29, 13:38:43 GMT+0 - Identified - A router has presented with a drive failure. We are currently in the process of rebuilding the configuration on a replacement device with redundant drives and power.  May 29, 13:38:43 GMT+0 - Identified - A router has presented with a drive failure.  May 29, 14:38:00 GMT+0 - Monitoring - We implemented a fix and are currently monitoring the result. Most core services have been restored. Email is still having issues as we are currently migrating away from our OVH POP. May 31, 04:48:37 GMT+0 - Resolved - This incident has been resolved. Please open a ticket if you experience any further issues. 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    <p><strong>Duration:</strong> 1 day, 15 hours and 10 minutes</p>
    <p><strong>Affected Components:</strong> , </p>
    &lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 29&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;13:38:43&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  A router has presented with a drive failure. We are currently in the process of rebuilding the configuration on a replacement device with redundant drives and power. .&lt;/p&gt;
&lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 29&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;13:38:43&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  A router has presented with a drive failure. .&lt;/p&gt;
&lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 29&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;14:38:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Monitoring&lt;/strong&gt; -
  We implemented a fix and are currently monitoring the result. Most core services have been restored. Email is still having issues as we are currently migrating away from our OVH POP..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 31&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;04:48:37&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  This incident has been resolved. Please open a ticket if you experience any further issues..&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Mon, 29 May 2023 15:38:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/cli8wb4ef232212khofb2ggvizr</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/cli8wb4ef232212khofb2ggvizr</guid>
</item>

<item>
  <title>Network Upgrade #1</title>
  <description>
    Type: Maintenance
    Duration: 2 hours

    Affected Components: Seattle, WA, USA, Portal / WHMCS
    May 28, 02:30:01 GMT+0 - Identified - Maintenance is now in progress May 28, 02:30:00 GMT+0 - Identified - Network interruptions may be occur during this time while we clean up and prepare for network and equipment upgrades. May 28, 04:11:12 GMT+0 - Identified - Almost finished stage #1. Most services will be restored soon. Expect an update for when the rest of the maintenance will be scheduled. May 28, 04:30:00 GMT+0 - Completed - Maintenance has completed successfully 
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Maintenance</p>
    <p><strong>Duration:</strong> 2 hours</p>
    <p><strong>Affected Components:</strong> , </p>
    &lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;02:30:01&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Maintenance is now in progress.&lt;/p&gt;
&lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;02:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Network interruptions may be occur during this time while we clean up and prepare for network and equipment upgrades..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;04:11:12&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Identified&lt;/strong&gt; -
  Almost finished stage #1. Most services will be restored soon. Expect an update for when the rest of the maintenance will be scheduled..&lt;/p&gt;
&lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 28&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;04:30:00&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Completed&lt;/strong&gt; -
  Maintenance has completed successfully.&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Sun, 28 May 2023 02:30:00 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/maintenance/cli6rqekm848892b3mwb89pklxm</link>
  <guid>https://1406557-bc-ltd.instatus.com/maintenance/cli6rqekm848892b3mwb89pklxm</guid>
</item>

<item>
  <title>Power Surge</title>
  <description>
    Type: Incident
    

    
    May 19, 01:56:09 GMT+0 - Resolved - We have experienced a sudden power surge due to stormy weather which caused a failure in one of our UPSs. The affected servers have been migrated to an alternative power source while we repair the failing UPS.  
  </description>
  <content:encoded>
    <![CDATA[<p><strong>Type:</strong> Incident</p>
    
    
    &lt;p&gt;&lt;small&gt;May &lt;var data-var=&#039;date&#039;&gt; 19&lt;/var&gt;, &lt;var data-var=&#039;time&#039;&gt;01:56:09&lt;/var&gt; GMT+0&lt;/small&gt;&lt;br&gt;&lt;strong&gt;Resolved&lt;/strong&gt; -
  We have experienced a sudden power surge due to stormy weather which caused a failure in one of our UPSs. The affected servers have been migrated to an alternative power source while we repair the failing UPS. .&lt;/p&gt;
]]>
  </content:encoded>
  <pubDate>Fri, 19 May 2023 01:56:09 +0000</pubDate>
  <link>https://1406557-bc-ltd.instatus.com/incident/clhtwt34z83806beomp3gzs727</link>
  <guid>https://1406557-bc-ltd.instatus.com/incident/clhtwt34z83806beomp3gzs727</guid>
</item>

  </channel>
  </rss>