Benchmarking Analytics Software
Recent blog postings about the extended set-theoretic data model and set-store architecture prompted an e-mail from Ralph Stout, the artist and one-time chief scientist at Information Builders.
Ralph found extended set theory provided a sizeable performance advantage after he ran benchmarks of IBM DB2, Oracle and David Childs' XSP software.The benchmarks that Ralph conducted were to compare database solutions for analytical processing, not transaction processing, CRM, social networking or other applications. During a seminar at Microsoft Research, David Childs discussed some of Ralph's findings about performance for TPC-H Query 9.
Video (duration 1:10:52) Using Extended Set Theory for High Performance Database Management
Ralph's e-mail said:
When I was at Information Builders, I ran a series of benchmarks designed to contrast the performance of Childs' XSP software with that of DB2 and Oracle -- the leading analytical DBMSs at that time. The results were stunning: XSP consistently outperformed these systems by a factor of 40 or more. While many of my colleagues dismissed these findings as simply too good to be true, others remained curious. The more we looked into the matter, the more puzzled we were about a demo that Childs had cobbled together to illustrate a point. We concluded that a fine-tuned implementation of the program might easily have achieved a two- or three-fold additional performance improvement. This led us to the further conclusion that the remarkable results that XSP had achieved had more to do with its rigorous mathematical underpinnings than anything else. It is this aspect of XSP that enables it, for example, to restructure data on the fly to suit the requirements of whatever query comes its way. No other DBMS, so far as I know, can do that.
Ralph's artwork is on display at the Carrie Haddad gallery in Hudson, New York.Ralph Stout found extended set theory provided a sizeable performance advantage after he ran benchmarks of IBM DB2, Oracle and Childs' XSP software.