Channels ▼

Ken North

Dr. Dobb's Bloggers

Standards, Stacks and Mergers: The SQL Backstory

June 25, 2010

In a movie plot, a backstory can help an actor understand the role he is playing and the audience to understand a charcter. Likewise the SQL backstory helps us understand why some corporations spent billions to acquire companies with a flagship SQL database platform. It also provides perspective as we look at the software eco-system and technology stack for building successive generations of services and applications.The SAP-Sybase and Oracle-Sun mergers provide an entree to discuss various topics about databases. One is a retrospective on SQL databases and how SQL vendor acquisitions provide a yardstick for measuring database technology. What's the reach of the SQL database today? Will it be included in the technology stack for the next generation of applications and services?

IBM, Oracle, Microsoft and SAP are betting the answer is 'Yes'. For example, recent acquisitions enabled SAP and Oracle to add a major SQL platform to their product portfolio. Both companies spent billions to broaden their product lines and the value of database platforms was undoubtedly a factor in the acquisition price. Oracle, IBM and Microsoft rank 1-2-3 in the database market, with each company having billions in revenue from its database products. 90% of enterprises have a mission-critical application running on a database server from IBM, Oracle or Microsoft.

Database Industry in the Wayback Machine The history of SQL since its development by Boyce and Chamberlin, the IBM System R incubator and the founding of Oracle spans several decades. SQL was developed to provide a declarative query language in an era when we were still working on expensive mainframe computers. But as computing and software have evolved, SQL database technology has proven to be resilient and adaptable.

During the SQL life span of almost 30 years, we've seen the arrival of networks, proliferation of personal computers, emergence of open source software, and software industry expansion and consolidation.

One reason for SQL's resilience is a commitment by database companies to standards. The first SQL standard arrived in 1986 and there have been several major updates. Nonetheless, periodic technology shifts usually bring out predictions the SQL database won't be with us much longer. But SQL vendors continue to evolve their platform, including Sybase. SAP would not invest billions to acquire a company whose primary revenue stream comes from technology that was soon to be obsolete.

When products such as DB2 and Oracle arrived on the database scene, the SQL database was a mainframe or minicomputer proposition. Proponents of hierarchical and CODASYL databases argued SQL performance was unacceptable, but SQL technology offered some advantages. It avoided painful, long-running database reorganizations required by schema changes and it offered a superior capability for ad hoc queries.

What ramped up SQL database sales was adoption of networks and client-server architecture, the infrastructure for distributed processing. The attraction of low-cost, de-centralized computing produced an explosion of PCs and networks in institutional settings. This resulted in a proliferation of ISAM software for desktop databases and SQL products for server databases. The database wars kicked into high gear. SQL became a buyer's market with more than 150 products and there was an update to the SQL standard in 1992.

Amidst growing adoption of SQL, object-oriented programming (OOP) became the rage. Some believed the new programming paradigm would prove to be an SQL killer.

After object-oriented programming (OOP) became the fashion, there arose an argument that an impedance mismatch between SQL and objects called for a new type of database. Some believed the Object Database Management Group (ODMG) standard and object databases would wipe out the franchise of database vendors such as Oracle, IBM, and Sybase. Instead, object databases remained a niche product. Some even added an SQL interface as a concession to the number of 'shrink-wrapped' applications that used SQL queries.

By 1997, it was suggested the World Wide Web would be the catalyst for abandonment of the SQL database. Subsequently it was argued that XML, and then open source databases, would undermine the maintream SQL vendors. Most recently some have argued that NoSQL databases will displace both commercial and open source SQL products. At times it seems the successive wave of 'SQL-killer' pronouncements is driven less by problems with the technology and more by a desire to displace vendors and open up markets.

Is Database Dead? So how accurate was the 'database is dead' mantra? Not very. The database market grew every year from 2000 to 2009, with SQL products holding the lion's share.

To better understand the importance of the SQL database to today's technology stack, we can look at the acquisition of several companies with a mainstream SQL DBMS in their product portfolio.

Forrester Research recently estimated the total database market (licenses, support, consulting) would grow from $27 billion in 2009 to $32 billion by 2012. SQL technology is entrenched in many organizations and across millions of web sites. Perhaps that explains why, during the past decade, IBM, Oracle, Sun and SAP made billion-dollar investments in a dead' technology.

During the Dot Com Boom, investors were in love with 'new tech' companies. Some argued the Web was going to change everything and make existing software technology obsolete. Some industry pundits suggested the database industry was going into a death spiral. But they based that conclusion on the wrong indicators. What they were observing was the thinning of the herd, not the demise of the entire database industry. There was a trend towards industry consolidation and the bigger vendors gained market share while evolving their SQL products along the Swiss Army Knife model. Today IBM, Oracle and Microsoft hold more than 80% of the database market.

Informix and IBM The "database is dead" rumors were an enabler for a major IBM acquisition of database technology and customers. IBM acquired Informix, a company that was a leader in object-relational technology. Informix had an early solution to integrating XML data into SQL databases and was the only company with a solution for embedding both Java classes and OLE components in SQL databases. IBM paid $1 billion for a company with $840 million in annual revenue, a price that was 1.2 times annual revenue.

MySQL, Sun and Oracle The Internet and XML databases did not eradicate commercial SQL products, but some believed the open source DBMS would do the job. Enthusiasm for open source software and widespread adoption of the LAMP stack produced a perfect storm in which Sun Microsystems acquired MySQL AB for $1 billion. For the same price IBM paid for Informix, Sun picked up a company approaching $100 million in annual revenue. The price tag to Sun for forging an identity as a champion of open source databases was roughly 10 times annual revenue.

Sun CEO Jonathan Schwartz said

MySQL's employees and culture, along with its near ubiquity across the Web, make it an ideal fit with Sun's open approach to network innovation. And most importantly, this announcement boosts our investments into the communities at the heart of innovation on the Internet and of enterprises that rely on technology as a competitive weapon.

At the time of the MySQL acquisition, I wrote of one favorable outcome. Sun could produce a unique new generation of database servers by tight integration of Java and MySQL with the transactional memory of the UltraSPARC (Rock) processor. But Sun did not put the Rock processor into production and Oracle has no plan to do so.

During the era when Digital Equipment Corporation (DEC) was failing, Oracle acquired DEC's relational and CODASYL model database products. It still offers the CODASYL product. Oracle has done some serious shopping for companies, buying 67 of them since 2005. It acquired Sleepycat Software, maker of the Berkeley DB product, and TimesTen, maker of a leading in-memory SQL database product. With the acquisition of Sun Microsystems for $7.3 billion, Oracle has added hardware and yet another DBMS to its product portfolio.

Oracle has niche database products and a major presence in applications suites, analytics and enterprise computing. With the acquisition of Sun, Oracle expands its portfolio with the Java platform, server hardware and the most popular SQL database for powering web sites. The Sun acquisition contributed $1.23 billion in hardware sales and $400 million in profits to Oracle's fourth-quarter earnings report.

SAP and Sybase SAP offered $5.6 billion for a company that recently experienced double-digit growth in database licensing revenue. For 2009, Sybase reported annual revenue of $1.17 billion. For the first quarter of 2010, it reported 10% year-over-year revenue growth to $294 million. SAP paid approximately 4.78 times annual revenue for Sybase. The Informix and MySQL deals were for database companies. But in Sybase, SAP sees the growth potential of other products in addition to the database platforms.

Technology Stacks, APIs Over the lifetime of the SQL database, we've seen the emergence of various software and computing paradigms. Early SQL developers were programming mainframe computers and creating monolithic executables, using embedded SQL (ESQL) and subroutine libraries. SQL developers were migrating to smaller computers (minicomputers, servers, PCs), client-server architecture programming, Windows and dynamic linking. ESQL became an industry standard in 1989. The C language gained a following and there were soon C call-level interfaces for accessing SQL databases. The Open Database Connectivity (ODBC) API was refined to produce the SQL Call-Level Interface (SQL CLI) standard in 1995. Java followed, along with the JDBC API.

The popularity of the Internet technology stack encouraged development of Web database interfaces. XML-based protocols encouraged the development of web services and collaborative applications, followed soon by emphasis on service-oriented architecture (SOA). The interest in Web-Oriented Architecture (WOA) put a heavy emphasis on solutions to enable JavaScript, PHP, Perl and scripting languages to access SQL data. Regardless of the evolution of client, middle-tier and web development technologies, SQL at the back end has remained a constant. Today ODBC and JDBC are still the predominant APIs for accessing databases, although service provider interfaces and XML-based data services are often part of a system architecture. The SQL language standard and standard APIs are undoubtedly reasons companies have decided having an SQL DBMS in the product portfolio is important. Proponents of some newer technologies, such as NoSQL databases and cloud computing, are still sorting out issues of not having standard APIs. In a recent webinar, the CTO of a leading cloud computing company indicated that proliferation of APIs for the cloud could slow the growth of cloud computing. The SQL database industry learned that lesson long ago and subsequently saw that growth followed adoption of standards.During the SQL life span of almost 30 years, we've seen the arrival of networks, proliferation of personal computers, emergence of open source software, and software industry expansion and consolidation.

One reason for SQL's resilience is a commitment by database companies to standards.

Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 


Dr. Dobb's TV