<div dir="ltr"><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Hi All,</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I'm not sure if you got my original email back in July but I'm finally ready to start scheduling this. I'd like to amend my plan below to the following:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Summary:</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><div class="gmail_default"><ol><li><b>Upgrade Ganeti from 2.6.2 to 2.15.2</b><br></li><li>Install CentOS 7 as the OS for all of the nodes<br></li><li>Switch to managing said nodes to Chef instead of Cfengine</li></ol></div></div><div class="gmail_extra"><div class="gmail_extra">Here's the actual steps I plan to do:</div><div class="gmail_extra"><ol><li><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline"><div class="gmail_default" style="display:inline">U</div><span style="font-family:arial,sans-serif">pgrade Ganeti to 2.15.2 on the current cluster from 2.6.2</span></div><br></li><li><div class="gmail_default" style="display:inline">Migrating high priority instances from plain to drbd using --no-wait-for-sync [1]<br></div></li><li><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">Failover instances on osgeo3 to osgeo4</div><br></li><li>Take osgeo3 down and reinstall it's OS with CentOS 7 and retain it's LVM data for VMs</li><li>Readd osgeo3 back into the cluster using its previous configuration and start all the VMs back up<br></li><li>Repeat the process of steps <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">3</div> <div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">through </div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif;display:inline">5</div> with osgeo4</li></ol><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">I'd like to go ahead with #1 and then schedule a time to do #2 after that's completed.</div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif"><br></div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">Let me know!</div></div><div><br></div><div><div class="gmail_default" style="font-family:arial,helvetica,sans-serif">[1] <i>The -t (--disk-template) option will change the disk template of the instance. Currently only conversions between the plain and drbd disk templates are supported, and the instance must be stopped before attempting the conversion. When changing from the plain to the drbd disk template, a new secondary node must be specified via the -n option. <b>The option --no-wait-for-sync can be used when converting to the drbd template in order to make the instance available for startup before DRBD has finished resyncing.</b></i></div></div><div><br></div></div></div><div class="gmail_extra"><div class="gmail_quote">On Thu, Jul 27, 2017 at 1:46 PM, Lance Albertson <span dir="ltr"><<a href="mailto:lance@osuosl.org" target="_blank">lance@osuosl.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div style="font-family:arial,helvetica,sans-serif"><div>OSGeo Admins,</div><div><br></div><div>I'd like to do several changes to your Ganeti cluster eventually to bring it up to a better supported platform and version of Ganeti as well. Unfortunately this is going to cause some downtime for each node but I'm pretty sure I can do it without losing data or downtime to certain VMs. Both of your nodes are currently running Gentoo which we haven't been maintaining other than for very important security issues that come up. Also, the version of Ganeti is currently 2.6.2 and the latest stable version is 2.15.2 which includes several improvements.</div><div><br></div><div>The summary of items I'd like to do are:</div><div><ol><li>Install CentOS 7 as the OS for all of the nodes<br></li><li>Switch to managing said nodes to Chef instead of Cfengine<br></li><li>Upgrade Ganeti from 2.6.2 to 2.15.2 (or whatever is stable at the point we get to this)</li></ol></div><div>This is going to need to be a multi-stage process unfortunately, but I'm hoping I only have to do one down time per node. I've tested this process in a Vagrant environment and it seems to work.</div><div><br></div><div>Here's the actual steps I plan to do:</div><div><ol><li>Take osgeo3 down and reinstall it's OS with CentOS 7 and retain it's LVM data for VMs<br></li><li>Install Ganeti 2.6.2 on osgeo3 using Chef so that the version stays the same throughout the whole cluster<br></li><li>Readd osgeo3 back into the cluster using its previous configuration and start all the VMs back up</li><li>Repeat the process of steps 1 through 3 with osgeo4</li><li>Upgrade Ganeti to 2.11.8 on all the nodes (I've found this to be safer than jumping from 2.6.2 directly to 2.15 as they made some major changes to the backend in those versions)</li><li>Finally upgrade Ganeti to 2.15.2 or whatever is latest stable at the time.</li></ol></div><div style="font-family:arial,sans-serif"><div style="font-family:arial,helvetica,sans-serif">So my questions to you are:</div><div><ol><li><font face="arial, helvetica, sans-serif">Should any of the instances below be migrated to another node during it's primary node downtime? If so and they're currently set to plain, we can convert them to DRBD, it will just take a short downtime (depending on how large the disk is) and move them over.</font></li><li><font face="arial, helvetica, sans-serif">When could we start doing this? I was hoping to start within the next month or so but it can certainly be adjusted.</font></li><li><font face="arial, helvetica, sans-serif">How should we communicate in real-time if we need to? Via #osuosl on IRC? Other means?</font></li></ol></div></div><div style="font-family:arial,sans-serif">
<table cellspacing="0" border="0">
<colgroup width="175"></colgroup>
<colgroup width="126"></colgroup>
<colgroup width="95"></colgroup>
<colgroup width="65"></colgroup>
<colgroup width="81"></colgroup>
<colgroup width="103"></colgroup>
<tbody><tr>
<td style="border-bottom:1px solid rgb(0,0,0)" height="17" align="center"><b>Instance</b></td>
<td style="border-bottom:1px solid rgb(0,0,0)" align="center"><b>Primary_node</b></td>
<td style="border-bottom:1px solid rgb(0,0,0)" align="center"><b>Status</b></td>
<td style="border-bottom:1px solid rgb(0,0,0)" align="center"><b>Memory</b></td>
<td style="border-bottom:1px solid rgb(0,0,0)" align="center"><b>DiskUsage</b></td>
<td style="border-bottom:1px solid rgb(0,0,0)" align="center"><b>Disk_template</b></td>
</tr>
<tr>
<td height="17" align="left"><a href="http://adhoc.osgeo.osuosl.org" target="_blank">adhoc.osgeo.osuosl.org</a></td>
<td align="center">osgeo4.osuosl.bak</td>
<td align="center">running</td>
<td align="right">4096</td>
<td align="right">65536</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://base.osgeo.osuosl.org" target="_blank">base.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">ADMIN_down</td>
<td align="right">-</td>
<td align="right">4096</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://download.osgeo.osuosl.org" target="_blank">download.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">running</td>
<td align="right">8192</td>
<td align="right">158720</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://mail.osgeo.osuosl.org" target="_blank">mail.osgeo.osuosl.org</a></td>
<td align="center">osgeo4.osuosl.bak</td>
<td align="center">running</td>
<td align="right">4096</td>
<td align="right">75776</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://projects.osgeo.osuosl.org" target="_blank">projects.osgeo.osuosl.org</a></td>
<td align="center">osgeo4.osuosl.bak</td>
<td align="center">running</td>
<td align="right">16384</td>
<td align="right">208896</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://qgis.osgeo.osuosl.org" target="_blank">qgis.osgeo.osuosl.org</a></td>
<td align="center">osgeo4.osuosl.bak</td>
<td align="center">running</td>
<td align="right">6144</td>
<td align="right">167936</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://secure.osgeo.osuosl.org" target="_blank">secure.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">running</td>
<td align="right">4096</td>
<td align="right">14464</td>
<td align="center">drbd</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://tracsvn2.osgeo.osuosl.org" target="_blank">tracsvn2.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">ADMIN_down</td>
<td align="right">-</td>
<td align="right">86016</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://tracsvn.osgeo.osuosl.org" target="_blank">tracsvn.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">running</td>
<td align="right">8192</td>
<td align="right">106496</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://web.osgeo.osuosl.org" target="_blank">web.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">running</td>
<td align="right">4096</td>
<td align="right">36864</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://webextra.osgeo.osuosl.org" target="_blank">webextra.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">running</td>
<td align="right">4096</td>
<td align="right">126976</td>
<td align="center">plain</td>
</tr>
<tr>
<td height="17" align="left"><a href="http://wiki.osgeo.osuosl.org" target="_blank">wiki.osgeo.osuosl.org</a></td>
<td align="center">osgeo3.osuosl.bak</td>
<td align="center">running</td>
<td align="right">4096</td>
<td align="right">20480</td>
<td align="center">plain</td>
</tr>
</tbody></table></div></div><div><div style="font-family:arial,helvetica,sans-serif"><br></div><div style="font-family:arial,helvetica,sans-serif">Thanks-</div><span class="gmail-HOEnZb"><font color="#888888"><br></font></span></div><span class="gmail-HOEnZb"><font color="#888888">-- <br><div class="gmail-m_-7172449932886513306m_-6993039583560125226m_-5560545199206064910m_1027486356967559232gmail_signature"><div dir="ltr"><font face="arial, helvetica, sans-serif">Lance Albertson</font><div><div><font face="arial, helvetica, sans-serif">Director</font></div><div><span style="font-family:arial,helvetica,sans-serif">Oregon State University | </span><span style="font-family:arial,helvetica,sans-serif">Open Source Lab </span></div></div></div></div>
</font></span></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><font face="arial, helvetica, sans-serif">Lance Albertson</font><div><div><font face="arial, helvetica, sans-serif">Director</font></div><div><span style="font-family:arial,helvetica,sans-serif">Oregon State University | </span><span style="font-family:arial,helvetica,sans-serif">Open Source Lab </span></div></div></div></div>
</div></div>