New SQL: An Alternative to NoSQL and Old SQL for New OLTP Apps An Article by Mike StoneBraker June 16, 2011, http://cacm.acm.org/ Group 18 Asmaa ElBadrawy Sundaram T R Motivation • Old OLTP requirements: ▫ Historically, OLTP was performed by customers submitting traditional transactions to a relational DBMS. ▫ Enterprises used ETL products to convert OLTP data to a common format and load into data warehouse for performing business analysis. ▫ Data warehouse activity rarely shared machine resources with OLTP because of lock contention in the DBMS and because business intelligence (BI) queries were so resource-heavy that they got in the way of timely responses to transactions. Motivation • New OLTP requirements: ▫ The transactions are increasing day by day. Good throughput has to be maintained even with the increase in transactions (e.g web and smart phones) ▫ Need for real-time analytics. (e.g a Web property wants to know the no. of current users playing its game) Picture from VoltDB.com New OLTP Deployment • Deployment Options: ▫ Three different Query languages: Traditional SQL NoSQL NewSQL New OLTP Deployment (cont’d) • Traditional SQL: ▫ The workload experienced by New OLTP may exceed the capabilities of Old SQL solutions. ▫ Data warehouses are typically stale by tens of minutes to hours. So real-time analytics very difficult with old SQL. Not ideal for New OLTP requirements. New OLTP Deployment (cont’d) • NoSQL: ▫ Overcomes workload problems in old SQL ▫ Provides Scalability and high performance Achieved through relaxing or eliminating transaction support and moving back to a low-level DBMS interface. ▫ Downside: Pushes ACID properties to applications where they are far harder to solve. The absence of SQL makes queries a lot of work. New OLTP Deployment (cont’d) • NewSQL: ▫ SQL-like, but not SQL. ▫ Implemented such that it is easy to use. ▫ Not so complex as SQL SQL not standardized or simplified. It is considered that object databases are not the future. ▫ Available Converters to migrate SQL based applications to NewSQL. So old applications based on SQL are not affected. ▫ Has java-like data types which are easier for developers. ▫ Supports advanced data types like arrays. New OLTP Deployment (cont’d) • NewSQL: ▫ Compared to the earlier options: Preserves SQL features Offers high performance and scalability Preserves the traditional ACID properties for transactions. ▫ Capabilities these systems should support: Should be equally capable of high throughput as the NoSQL solutions, without the need for application-level consistency code. Should preserve the high-level language query capabilities of SQL. A Comparison of old and NewSQL SQL NewSQL 'Jdb' NewSQL 'S2' CREATE TABLE TEST( ID INT PRIMARY KEY, NAME VARCHAR(255) ) test=new table( int id, string name, key(id) ) create table test( id int, name string, primary key(id) ) INSERT INTO TEST VALUES(1,'Hello') test.add(1,"Hello") insert test (1,'Hello') SELECT * FROM TEST test.get() select test SELECT T1.ID,T2.NAME FROM TEST T1, TEST T2 WHERE T.ID=T2.ID t1=test; t2=test; t1.join(t2[t1.id==t2.id]).get(t1.id,t2.na me) select t1:test join t2:test on t1.id==t2.id get t1.id, t2.name UPDATE TEST SET NAME='Hi' WHERE ID=1 test[id==1].set(name="Hi") update test set id=1 where name=='Hi' DELETE FROM TEST WHERE ID=1 test[id==1].delete() delete test where id==1 DROP TABLE TEST test.drop() drop test NewSQL Commercial Use • Clustrix ▫ Distributed systems-based database solutions. • NimbusDB ▫ Provides very fast transactional database solutions • VoltDB ▫ Provides a “blazingly fast” relational database system. real-time feeds, sensor-driven data streams, micro-transactions, low-latency trading systems Relevance to course • Chapter 21: ▫ OLTP • Chapter 29: ▫ Data warehouses References • “New SQL: An Alternative to NoSQL and Old SQL for New OLTP Apps”, Michael Stonebraker, June 16, 2011, http://cacm.acm.org/browse-by-subject/data-storage-andretrieval/109710-new-sql-an-alternative-to-nosql-and-oldsql-for-new-oltp-apps/fulltext . • “NewSQL Project”, Source Forge, http://newsql.sourceforge.net/ • “'NewSQL' Could Combine the Best of SQL and NoSQL”, Joab Jackson, IDG News, PCWorld, http://www.pcworld.com/businesscenter/article/238728/ne wsql_could_combine_the_best_of_sql_and_nosql.html • “The NewSQL Database You'll Never Outgrow”, VoltDB, http://voltdb.com/our-story/about-voltdb