[OSGeo Africa] Open-Source wins Big Data battle

[S Coetzer]

Morning,

Just sharing some techno news.......READ BELOW


AMD has migrated terabytes of information from an Oracle Database
installation to an Apache Hadoop stack, claiming Oracle's pricey software
was suffering from scaling issues.

The chip maker's chief information officer, Jake Dominguez, revealed
further details of the transfer in a chat with *The Reg*.

"Within the common Oracle platform we had we were struggling from a
performance and reliability perspective," Dominguez told your correspondent
in Atlanta just before the weekend. "One of the areas we were struggling
with was in our test and assembly manufacturing – large, large datasets."

The migration of 276TB of data, which was completed last year, was prompted
by "an environment outage that took weeks to recover," according to an
internal document seen by *El Reg*. This encouraged AMD to replace Oracle
for something else.

In the end, the processor giant settled on using Cloudera's Hadoop
distribution along additional open-source projects Apache Hive, ZooKeeper,
HBase, HDFS, httpfs, LZO compression, MapReduce and others.

According to AMD, the Hadoop software has an unlimited row limit for query
results compared to 100,000 rows on the chip giant's Oracle setup, and "99
per cent of all queries execute in 15 minutes or less, with a median
execution time of just 23 seconds."

What makes this shift so significant is that Oracle wants you to think AMD
is the sort of company that will always use Oracle kit.

Oracle is grappling with a shift in the data warehouse and analytics
market: its core business is being
squeezed<http://www.theregister.co.uk/2013/07/26/oracle_threatened_by_database_minnows/>by
free and open-source on-premises software, and its cloud wing is
facing
off with Amazon Web Services and the like.

Many organizations have sought to extricate themselves from Oracle's grip,
either by swapping out Oracle-owned open-source tech for other software, as
Google did with a vast MySQL to MariaDB
migration<http://www.theregister.co.uk/2013/09/12/google_mariadb_mysql_migration/>,
or by shifting away from the company's proprietary databases to open-source
ones, as the UK's National Health Service did with a major Riak
migration<http://www.theregister.co.uk/2013/10/10/nhs_drops_oracle_for_riak/>
.

One of the main open-source technologies commonly being deployed to
supplement or replace Oracle is Hadoop, a data storage and processing
framework that was first developed at Yahoo*!* in 2005 by engineers
attempting to replicate some advanced technologies invented at Google.

Today, software like Hadoop, and other distributed data storage and
management frameworks like Cassandra and Riak, are competing with software
from IBM, SAP, and most prominently Oracle.

For AMD, a sophisticated multinational manufacturing company, to launch a
major Oracle migration project is representative of a broader shift in IT
which benefits low-cost or free software at the expense of incumbents like
Oracle.

"We made the pivot to Hadoop [and] it not only increased our reliability
but [improved] our response time," Dominguez told us. "It's going to be an
integral part of our enterprise data warehouse concept." ®



Sarel Coetzer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/africa/attachments/20140325/d6e2e7e5/attachment.html>