[SAC] [Hosting] mysql1-vip (mysql1/mysql2) 5.5 -> 5.6 Upgrade | April 20th, 2022 17-20 UTC

[Ethan Hampton]

All,

Due to Xtrabackup requiring a read lock for part of the backup process, 
there will be an additional outage at the beginning of the maintenance 
window so we can complete a final backup before the cluster upgrade. We 
will do our best to minimize the outage time.

Please let us know if you have any questions or concerns.


On 4/18/22 2:41 PM, Ethan Hampton wrote:
>
> All,
>
> We had an outage of the mysql1-vip cluster from 20:18 to 21:04 UTC 
> today (Apr 18th) related to the final backup and preparation for this 
> upgrade. Xrabackup issued a "FLUSH TABLES WITH READ LOCK" for the 
> final stage of the backup process. When combined with a long running 
> query, this exhausted the MySQL connection pool and resulted in a 
> read/write outage.
>
> We will take steps to mitigate this for the upgrade on the 20th. 
> Please let us know if you have any questions or concerns.
>
> Best-
>
> On 4/13/22 1:53 PM, Ethan Hampton wrote:
>> All,
>>
>> We will be upgrading one of our MySQL clusters (mysql1-vip 
>> (mysql1/mysql2)) next Wednesday, April 20th, 2022 between 17:00-20:00 
>> UTC (10am-1pm PDT).
>>
>> We are upgrading MySQL (Percona) from 5.5 to 5.6. This is the first 
>> step towards managing the cluster under Chef and upgrading to 
>> MySQL(Percona)5.7 at a future date. Expect more maintenance on this 
>> cluster in the near future.
>>
>> This will result in up to 20 minutes of downtime in the middle of the 
>> window as we fail-over the cluster. We estimate that the actual 
>> downtime will be far less than that bound.
>>
>> We have tested this upgrade in a separate environment and believe 
>> that it will have minimal impact to projects on the cluster. We will 
>> make a full backup of the cluster before the upgrade work begins.
>>
>> *If you are using the vip1 cluster, please make sure you are using 
>> the mysql1-vip.osuosl.org address instead of mysql1/2.osuosl.org 
>> directly or additional outages will occur.*
>>
>> If you have any questions or concerns, please let us know. We'll be 
>> in #osuosl on Libera Chat in case any one needs to contact us directly.
>>
>> Thanks!
>>
-- 
Ethan Hampton
Student Systems Engineer
Oregon State University | Open Source Lab
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/sac/attachments/20220420/1323bf98/attachment.html>
-------------- next part --------------
_______________________________________________
Hosting mailing list
Hosting at osuosl.org
https://lists.osuosl.org/mailman/listinfo/hosting