[SAC] New Hardware, can we purchase now

[Chris Giorgi]

 Hi Alex,
Answers inline below:
Take care,
   ~~~Chris~~~

On Mon, Mar 12, 2018 at 10:41 AM, Alex M <tech_dev at wildintellect.com> wrote:
> On 03/02/2018 12:25 PM, Regina Obe wrote:
>> I'm in IRC meeting with Chris and he recalls the only outstanding thing
>> before hardware purchase was the disk size
>>
>> [15:17] <TemptorSent> From my reply to the mailing list a while back, the
>> pricing for larger drives: (+$212 for 4x10he or +$540 for 4x12he)
>>  [15:19] <TemptorSent> That gives us practical double-redundant storage of
>> 12-16TB and 16-20TB respectively, depending how we use it.
>>
>>
>> If that is all, can we just get the bigger disk and move forward with the
>> hardware purchase.  Unless of course the purchase has already been made.
>>
>>
>> Thanks,
>> Regina
>>
>
> Apologies, I dropped the ball on many things while traveling for work...
>
> My take on this, I was unclear on if we really understood how we would
> utilize the hardware for the needs, since there are a few new
> technologies in discussion we haven't used before. Was also in favor of
> small savings as we're over the line item, and that money could be used
> for things like people hours or 3rd party hosting, spare parts, etc...
>
> So a few questions:
> 1. If we get the optane card, do we really need the SSDs? What would we
> put on the SSDs that would benefit from it, considering the Optane card?

The Optane is intended for caching frequently read data on very fast storage.
As a single unmirrored device, it is not recommended for write-caching of
important data, but will serve quite well for temporary scratch space.

Mirrored SSDs are required for write caching to prevent failure of a single
device causing data loss. The size of the write cache is very small by
comparison to the read cache, but the write-to-read ratio is much higher,
necessitating the larger total DWPD*size rating. The SSDs can also provide
the fast tablespace for databases as needed, which also have high write-
amplification. The total allocated space should probably be 40-60% of the
device size to ensure long-term endurance. The data stored on the SSDs
can be automatically backed up to the spinning rust on a regular basis for
improved redundancy.

> 2. What caching tool will we use with the Optane? Something like
> fscache/CacheFS that just does everything accessed, or something
> configured per site like varnish/memcache etc?

We can do both if desirable, allocating large cache for the fs (L2ARC in ZFS),
as well as providing an explicit cache where desirable. This configuration can
be modified at any time, as the system's operation is not dependent on the
caching device being active.

> 3. Our storage growth is modest, not that I don't consider the quoted 8
> or 10 TB to be reliable, but the 2 and 4 TB models have a lot more
> reliability data, and take significantly less time to rebuild in a Raid
> configuration. So how much storage do we really need for Downloads and
> Foss4g archives?

OSGeo-Live alone has a growth rate and retention policy that indicates needs for
on the order of 100GB-1TB over the next 5 years from my quick calculations, not
including any additional large datasets. Supporting the geodata project would
likely consume every bit of storage we throw at it and still be
thirsty for more in
short order, so I would propose serving only the warm data on the new server and
re-purposing one of the older machines for bulk cold storage and backups once
services have been migrated successfully.

Remember, the usable capacity will approximately equal the total capacity of a
single drive in a doubly redundant configuration with 4 drives  at
proper filesystem
fill ratios. We'll gain some due to compression, but also want to provision for
snapshots and backup of the SSD based storage, so 1x single drive size is a
good SWAG. Resliver times for ZFS are based on actual stored data, not disk
size, and can be done online with minimal degradation of service, so that's a
moot point I believe.

> 4. Do we know what we plan to put on the SSD drives vs the Spinning Disks?

See (1).

> I think with the answers to these we'll be able to vote this week and order.
>
> Thanks,
> Alex
> _______________________________________________
> Sac mailing list
> Sac at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/sac