[SAC] [Hosting] Power issues this morning

[Lance Albertson]

All,

I got the following update from the data center manager which provides the
root cause for the issue yesterday:

"I had our UPS maintenance tech in last night and it seems that one of the
batteries in the battery string failed and that is what caused it not to
hold up on the switch over.  I have them on-site this morning replacing the
failed battery."

There is another planned outage tomorrow morning from 5-7am PST but I don't
expect any issues this time around.


On Thu, Dec 19, 2019 at 1:53 PM Lance Albertson <lance at osuosl.org> wrote:

> All,
>
> I wanted to send an update for the power issues from this morning. I
> investigated the hosts that lost power and discovered one of them had a bad
> PSU and lost all power when the other good PSU lost power (I replaced it
> this morning). There were other hosts that had their power not properly
> split between the two power feeds which I have since fixed. Other hosts
> that were affected are racked in a part of the datacenter that only has one
> power feed.
>
> I'm still waiting to hear why that particular power feed completely lost
> power instead of the UPS holding power.
>
> Thanks-
>
> On Thu, Dec 19, 2019 at 10:03 AM Lance Albertson <lance at osuosl.org> wrote:
>
>> All,
>>
>> This morning we experienced some power issues in the machine room where
>> most of our servers are hosted. There was a planned building power outage
>> however it was assumed all generator and UPS systems would work properly.
>> Unfortunately for some reason, some of the systems lost power and had to be
>> manually brought back online. I'm working with the data center manager to
>> figure out what the core issue was. In addition, I'm going to be on-site
>> later today to double check power connections to the hosts that did have a
>> problem.
>>
>> If you have any system or service that is still offline, please let me
>> know by sending an email to support at osuosl.org. I believe we have
>> resolved all of the outstanding issues.
>>
>> Thanks for your patience.
>>
>> --
>> Lance Albertson
>> Director
>> Oregon State University | Open Source Lab
>>
>
>
> --
> Lance Albertson
> Director
> Oregon State University | Open Source Lab
>


-- 
Lance Albertson
Director
Oregon State University | Open Source Lab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/sac/attachments/20191220/dbd58fce/attachment.html>
-------------- next part --------------
_______________________________________________
Hosting mailing list
Hosting at osuosl.org
https://lists.osuosl.org/mailman/listinfo/hosting