Data warehousing 'on steroids'

Database News : Thursday December 06, 2007

BUSINESS INTELLIGENCE / NETEZZA OFFERS DISTRIBUTED APPROACH

Data warehousing 'on steroids'

DON SAMBANDARAKSA

A new data warehousing company is offering customers in Thailand a radically new distributed hardware and software architecture that promises to increase performance over traditional BI solutions by up to 400 times, while at the same time dramatically slashing power consumption and simplifying data centre configuration.

Allan Bagley, Netezza's managing director for South Asia and Michael Kearney, director of marketing, were recently in Bangkok to help explain how this distributed architecture works.

Rather than focus on building faster storage or databases, Netezza focusses on the customer need to analyse more and more data. Traditional data warehouses have been designed around the transaction and completing that transaction as fast as possible. Naturally, banks do not want to hold up their customers at the ATM. However, that architecture is not well suited when doing business intelligence.

That is because all the data has to be moved from the storage systems over fibre channel arrays and storage array networks across the data centre and loaded into the memory of powerful computers where it is then processed with an SQL database. Only after all this effort has been done, most of the data is then discarded.

Netezza uses a distributed architecture which puts computing power nearer to the storage. The solution consists of an array of blades, each with a 400GB hard drive, IBM Power CPU and a one million gate FPGA (field programmable grid array; a special CPU that can be reconfigured on the fly).

The Netezza approach merges database, storage and servers into one blended architecture. It integrates two basic SQL commands, select and where, right on the FPGA which itself also acts as the hard drive controller. This means that 99 percent of irrelevant data can be discarded at the hard drive controller cum SQL processor level and only the relevant data is passed on for further processing, thus saving enormous amounts of bandwidth and computing power, not to mention energy.

Of course, the other half of the solution is designing the actual business intelligence software that can run on this distributed architecture.

Charoen Fuangtharnthip, executive director of Netezza's partner in Thailand, Sahasedsiri International, has set his sights on the finance, telco and retail sectors with data warehouses of at least 1-2 terabytes and are feeling the pain of managing and powering their IT infrastructure.

He noted that in an executive seminar, 12 of the 15 invited CIOs showed up, which means that there is awareness on the importance of more efficient data warehousing and BI within the local market.

By : Bangkok Post