Column-Oriented Replication Management for Large-Volume Data Storages

Abstract

The column-based database repository is a highly advanced model for big data analysis systems with its superior I/O performance. In order to improve write operations, traditional database systems utilize a  block-oriented storage in which records of column attributes are placed continuously on the hard disk. However, for read-intensive data warehouse, column-oriented storage becomes a more appropriate model to exploit its excellent performance. In addition, flash SSDs using MLC memory have recently been recognized as a most suitable storage medium for high-speed data analysis systems.

This paper introduces the column-oriented model and proposes a new storage management scheme that utilizes cross-compression method for high-speed data warehouses. The proposed storage management scheme is implemented on two MLC SSDs and provides excellent performance and reliability even in high CPU and I/O workloads. The results of our performance evaluation show that the proposed storage management scheme is better than the conventional scheme in terms of column update throughput and response time.