Dr. Dobb's is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.

Channels ▼

Ken North

Dr. Dobb's Bloggers

100 GB to 30,000 GB: Size, Speed, and Benchmarks

June 21, 2012


The TPC gained its early fame publishing online transaction processing (OLTP) benchmarks, such as TPC-A and TPC-B. TPC benchmarks are defined to model actual business scenarios instead of being a suite of tests that isolate communications, disk I/Os, backup, and other factors that affect database application performance. TPC defines the specific benchmark functions but organizations are free to choose the mix of hardware and software to use in their implementation of a benchmark. It certifies auditors and requires a full disclosure report describing the hardware and software used for a benchmark.

When TPC-C was approved in 1992, it was a replacement for the older DebitCredit benchmarks. TPC-C is an OLTP benchmark that uses a more complex database and transaction types than its predecessors. It models an order processing application for a parts supplier with multiple sales districts and warehouses. The application processes incoming orders and maintains a stocking level of 100,000 items at the warehouses, while using a database with ACID properties (atomicity, consistency, isolation, durability).

TPC-C results show dramatic growth over the past decade in the number of transactions processed per second and a dramatic reduction in the price per transaction.

TPC-E is a more recent OLTP benchmark that's built around the model of a brokerage firm. Since 2010, vendors have submitted results for ten TPC-E benchmarks. The TPC provides a software tool named EGen to assist in TPC-E benchmarking. It provides project and make files and the capability of generating the data required for the benchmark.

A high volume of short-lived queries and a large number of concurrent users typically characterize large-scale OLTP applications. But decision support is a different kind of beast, often involving very large databases, sometimes fed by Big Data.

Decision Support

In recent years the TPC has recognized that an important class of database users runs analytics and business intelligence applications. Those users want optimal performance for decision support. Unlike an OLTP workload, decision support involves longer running queries, often against data warehouses, for a smaller number of users.

TPC responded to this community by developing benchmarks that model decision-support scenarios instead of processing transactions. The TPC-H benchmark runs a collection of ad hoc queries while concurrently performing updates to the data. TPC provides a tool, DBGEN, for generating the databases for TPC-H benchmarks. The published results for TPC-H are broken down by database size (100 GB, 300GB, 1000GB, 3000 GB, 10,000 GB, and 30,000 GB).

Recently, I received a briefing from two TPC representatives about an upcoming conference and a new decision support benchmark. Interest in benchmarking is a prime reason for a TPC conference. The TPC Technology Conference on Performance Evaluation and Benchmarking (TPC-TC) is an annual event. Raghunath Nambiar (Cisco) and Meikel Poess (Oracle) are the General Chairs of the 2012 conference that runs 27-31 August in Istanbul, Turkey. TPC-TC 2012 features a keynote by Dr. Michael Carey (UC Irvine), a well-known database researcher.

The new TPC-DS benchmark is a major improvement over TPC-H, which has been the industry's flagship decision support benchmark for a decade. Meikel Poess provided drill-down into details about TPC-DS. Because he worked on both TPC-H and TPC-DS, Meikel was able to highlight useful comparisons between the two.

We'll take a closer look at TPC-DS in my next blog post.

Related Reading

More Insights

Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.