HANA Performance

Efficient Speed and Scale-out for Real-time BI

HANA Performance: Efficient Speed and Scale-out for Real-time BI

Introduction
SAP HANA enables organizations to optimize their business operations by analyzing large amounts of data in real time. HANA runs on inexpensive, commodity hardware and requires no proprietary add-on components. It achieves very high performance without requiring any tuning. A 100TB performance test was developed to demonstrate that HANA is extremely efficient and scalable and can very simply deliver break-through analytic performance for real-time business intelligence (BI) on a very large database that is representative of the data that businesses use to analyze their operations. A 100TB1 data set was generated in a format as would be extracted from an SAP ERP system (e.g., data records with multiple fields) for analysis in an SAP NetWeaver BW system.2 This paper will describe the test environment and present and analyze the test results.

The Test Environment

The performance test environment was developed to represent a Sales & Distribution (SD) BI query environment that supports a broad range of SQL queries and business users.

Database Schema
The star-schema design in Figure 1 shows the SD test data environment and each tables cardinality. No manual tuning structures were used in this design; there were no indexes, materialized views, summary tables or other redundant structures added for the purposes of achieving faster query performance.

100TB = raw data set size before compression. These tests did not involve BW.

Figure 1 - SD Schema

Test Data
The test data consists of one large fact table and several smaller dimension tables as seen in Figure 1. There are 100 billion records in the fact table alone, representing 5 years worth of SD data. The data is hash-partitioned equally across the 16 nodes using Customer_ID. Within a node, the data is further partitioned into one-month intervals. This results in 60 partitions per node and approximately 104 million records per partition. (See also Figure 2 and Loading.)

System Configuration
The test system configuration (Figure 2) is a 16-node cluster of IBM X5 servers with 8TB

of total RAM priced at approximately $500,000. Each server has: 4 CPUs with 10 cores and 2 hyper-threads per core, totaling - 40 cores - 80 hyper-threads 512 GB of RAM 3.3 TB of disk storage

Figure 2 - Test Configuration

Setup
These performance tests did not use pre-caching of results or manual tuning structures of any kind and therefore validated HANAs load-then-query ability. A design that is completely free of tuning structures (internal or external) is important for building a sustainable real-time BI environment; it not only speeds implementation, but it also provides ongoing flexibility for ad hoc querying, while eliminating the maintenance cost that tuning structures require.

Loading
The loading was done in parallel using HANAs IMPORT Command which is a single SQL statement that names the file to load. Loading is automatically parallelized across all of the nodes and uses the distribution and partitioning scheme defined for each table3 (in this case, hash distribution and monthly range partitions). The load rate was measured at 16 million records per minute, or one million records per minute per node. This load performance is sufficient to load 100 million records (representing one business days activity) in just six minutes.

Compression
Data compression occurs during the data loading process. HANA demonstrated a greater than 20X compression rate4; the 100TB SD data set was reduced to a trim
3

As defined with the CREATE TABLE command.

Compression rates depend heavily on characteristics of the actual data to be compressed. Individual results may vary.

3.78TB HANA database, consuming only 236GBs of RAM on each node in the cluster (see Figure 2). The large fact table accounts for 85 of the 100TBs in the data set, and was compressed to occupy less than half of the available RAM per node.

Queries
The query suite totals 20 distinct SQL queries including 11 base queries plus a number of variations for time interval (month, quarter, etc.). The queries were chosen to represent a mixed workload environment running on data that is representative of its native form (i.e., not indexed, tuned or otherwise de-normalized to avoid joins, as would be customary in a conventional database system). They cover a range of BI activities from departmental to enterprise-level including general reporting, iterative querying (drill downs), ranking, and year-over-year analysis, The resulting queries range from moderately complex to very complex, including SQL constructs such as multiple joins, in-list, fact-to-fact joins, sub queries, correlated sub queries (CSQ), and union all. The queries are grouped into three general categories of BI usage: Reporting calculations on business performance measures for ranges of materials and/or customers for a time period Drilldown iterative user-initiated queries to gather details for a given individual or group of materials and/or customers Analytical periodic deep historical analysis across customers and/or materials Table 1 documents each query in terms of its business description, SQL constructs, ranges and qualifiers, and time period variations. If a query is described with multiple time periods, each time period is run as a distinct query; for instance, query R1 is run as four distinct queries, covering monthly, quarterly, semi-annual, and annual time periods, to show its performance relative to changes in data volume (annual data is12X the volume of monthly data). Items in bold denote query complexity factors.

BI Query

Business Description

SQL Constructs

Ranges & Qualifiers

Time Period(s)

R1

Sales and distribution report by month for: a) given range of materials b) given range of customers Sales and distribution report by month for given list of customers

Star-schema query with multiple joins to dimensions, with group by and order by

Range of materials Range of customers

Four variations: 1. One month; 2. Three months; 3. Six months; 4. Twelve months One month

R2

Star-schema query with multiple joins to dimensions, with group by and order by Star-schema query with multiple joins to dimensions, group by and order by Star-schema query with multiple joins to dimensions, group by and order by

Range of customers defined using in list

R3

Sales and distribution report by month for: a) given range of materials b) given customer country Top 100 customers report for: a) one or several materials groups b) one or several customer countries

Range of materials Customer country

One month

R4

One or several material groups One or several customer countries

One month of data for two variations: 1. One country and one product group; 2. Three countries and two product groups Two variations: 1. Three months; 2. Six months One month

R5

Top 100 customers report for: a) given material group b) given customer country Sales and distribution report for: a) single customer; b) given material group Sales and distribution report for: a) single customer; b) single material Sales and distribution report for: a) single material b) given customer country Sales and distribution report for: a) single customer; b) given range of materials Year Over Year (YOY) Top 100 customers analytical report: a) Top 100 customers identified for a current time period b) Business measures calculated for previously identified top 100 customers in a given historical period c) A final result set calculated that combines current and historical business measures for top 100 customers YOY Trending Report for Top 100 customers: a) Top 100 customers calculated for a given time period of current year b) Top 100 customers calculated for identical time period of previous year (or multiple years) c) A final result set calculated that combines business measures for top 100 customers over several years

Star-schema query with multiple joins to dimensions, group by and order by Star-schema query with multiple joins to dimensions, group by and order by Star-schema query with multiple joins to dimensions, group by and order by Star-schema query with multiple joins to dimensions, group by and order by Star-schema query with multiple joins to dimensions, group by and order by Star-schema query with fact-tofact join as a correlated subquery that includes multiple joins to dimensions, and then group by and order by

Material group Customer country

D1

Single customer Material group

D2

Single material Single customer

One month

D3

Single material Customer country

One month

D4

Range of materials Single customer

One month

Range of materials; Customer country

A1

Three variations: 1. One month with YOY comparison 2. Three months with YOY comparison 3. Six months with YOY comparison

A2

Several star-schema subqueries each with multiple joins, group by and order by, which are combined by a union all operator

Material group; Customer country

Three variations: 1. Three months over two years 2. Six months over two years 3. Three months over five years

Table 1 - Query Descriptions

Test Results
The test measured both query response time and query throughput per hour. The queries were run first in a single stream to measure baseline query times, then in multiple streams of 10, 20, and 25 to measure query throughput (calculated as queries per hour) and query response time in the context of different workloads5. The multistream tests randomized the sequence of query submissions across streams6 and inserted 10 milliseconds of think time between each query to simulate a multi-user ad hoc BI querying environment. Individual runtimes are listed in the Appendix.

Baseline Test
As you can see from the baseline results in Figure 3, the majority of queries delivered sub-second response times and even the most complex query, which involved the entire 5-year span of data, ran in under 4 seconds.

Figure 3 - Baseline Performance, in Seconds

The Reporting and Drill-down queries (267 milliseconds to 1.041 seconds) demonstrate HANAs excellent ability to aggregate data. For instance, the longest-running of these, R1-4, ran in just over a second (1.041) but took only 2.8X longer to crunch through 12X more data than its monthly equivalent (R1-1).
5

A single query stream represents an application connection that supports potentially hundreds of concurrent users.
6

This is done to prevent a query from being submitted by different streams at exactly the same time and to ensure a practical workload mix (e.g., drill downs happen more frequently than quarterly reports).

The Drill-down queries (276 to 483 milliseconds) demonstrate HANAs aggressive support for ad hoc joins and, therefore, to provide unrestricted ability for users to slice and dice without having to first involve the technical staff to provide indexes to support it (as would be the case with a conventional database). The Analytic queries (677 milliseconds to 3.8 seconds) efficiently performed sophisticated joins (fact-to-fact, sub queries, CSQ, union all) and analysis across a sliding time window (year-over-year). The queries with one- to six-month date ranges (A1-1 through A2-2) ran in 2 seconds or less. Query A2-3, which analyzed the entire 5year date range, ran in under four seconds. All analytic query times are well within timeframes to support iterative speed-of-thought analysis. Across the board, the baseline tests show that HANA scaled efficiently (better-thanlinear) as the data volumes increased for a given query.

Throughput
The throughput tests are summarized in Table 2 and show that in the face of increasing and mixed BI workloads, HANA scales very well.

Test Case 1 stream 10 streams 20 streams 25 streams

Throughput (Queries per Hour) 6,282 36,600 48,770 52,212

Table 2 - Throughput Tests

At 25 streams, the average query response time was less than three seconds and is only 2.9X higher than at baseline (see, Appendix), an indicator of HANAs excellent internal efficiencies and ability to manage concurrency and mixed workloads. A rough estimate of BI user concurrency can be derived by dividing total queries per hour by an estimated average number of queries per user per hour. For instance, the 25-streams 52,212 queries per hour divided by 20 (a zesty per-user rate of one query every three minutes) provides a reasonable estimate of 2610 concurrent BI users across a mixture of reporting, drill-down and analytic query types.

Ad Hoc Historical Queries

The performance tests were intended primarily to demonstrate HANAs performance in customary real-time BI query environments that focus on analyzing business operations. However, ad hoc historical queries that analyze the entire volume of data that is stored may occasionally need to be run (and they should not impose a severe penalty). In addition to Query A2-3 (included in the stream tests), Queries R1-1, R4-2 and R5-1 were deemed potentially relevant for a 5-year time window and were resubmitted, this time for the entire 5-year date range. Table 3 shows that HANA demonstrated better-than-linear scalability running against 5X more data with a 3X increase in response time, or less.
Time Period 1 year 2 years 3 years 4 years 5 years R1-1 1.1 secs 1.8 secs 2.6 secs 3.3 secs 4.1 secs R4-2 1.4 secs 2.4 secs 3.4 secs 4.4 secs 5.4 secs R5-1 1.1 secs 1.8 secs 2.4 secs 3.0 secs 3.6 secs

Table 3- Ad Hoc Historical Queries

Overall, these queries all ran in a few seconds and further highlight HANAs ability to support real-time BI on massive amounts of data without tuning and without massive hardware or proprietary hardware components.

Results Summary
In these performance tests, HANA demonstrated ability to deliver real-time BI-query performance as workloads increase in terms of capacity (up to 100TBs of raw data), complexity (queries with complex join constructs and significant intermediate results run in less than 2 seconds), and concurrency (25-stream throughput representing about 2600 active users). HANA performance is based on comprehensive, well-thought design elements, including, for example, an in-memory design smart internal data structures (e.g., native columnar, advanced compression and powerful partitioning) a clever, cost-based optimizer that can meld these smart data structures into an efficient query plan

efficient query execution that smartly executes the query plan to take advantage of internal components (e.g., advanced algorithms, multi-level caching optimization in the CPU and hyper-threading).

Much has been written on HANAs design and it is not necessary to rewrite it here. For more information on HANA technical features, please refer to SAP HANA Technical Overview and SAP HANA for Next-Generation Business Applications and Real-Time Analytics.

Real-world Experiences
These tests were run in a simulated environment to isolate HANA performance7; however, you can expect your own BI Query performance to be significantly better on HANA than on your existing conventional DBMS, potentially thousands of times better. Most importantly, SAP customer testimonials confirm HANAs performance. Here are a few examples. We have seen massive system speed improvements and increased ability to analyze the most detailed levels of customers and products. Colgate Palmolive replacing our enterprise-wide Oracle data mart and resulting in over 20,000X speed improvement processing our most complex freight transportation cost calculation. ...our stand-alone mobile applications that were previously running on Oracle are now running on HANA. Our 2000 local sales representatives can now interact with real-time data instead, and have the ability to make on-the-fly promotion decisions to improve sales. Nongfu Spring our internal technical comparison demonstrated that SAP HANA outperforms traditional disk-based systems by a factor of 408,000...Mitsui Knowledge Industry Co., Ltd. With SAP HANA, we see a tremendous opportunity to dramatically improve our enterprise data warehouse solutions, drastically reducing data latency and improving speed when we can return query results in 45 seconds from BW on HANA vs. waiting up to 20 minutes for empty results from BW on a traditional disk-based database. Shanghai Volkswagen

These performance tests were designed to isolate HANA system level performance independent of the variety of applications that may be used in SAP environments. Therefore, these test results should be used only as a general guideline, and results will vary from implementation to implementation due to variability of platforms, applications and data.

10

Conclusion
Real-time BI query environments should be able to support new queries as soon as users formulate them. This speed-of-thought capability is only possible if consistently excellent performance is freely available in the underlying database platform. These rigorous performance tests and their results presented here confirm that, without tuning and without massive hardware or proprietary hardware components, HANA delivers leading performance and scale-out ability and enables real-time BI for businesses that must support a range of analytic workloads, massive volumes of data and thousands of concurrent users.

11

Appendix Query Runtimes

*10-, 20-, 25-stream runs reflect 10ms of think time between queries.
Query Name R1-1 TESTCASE* 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams Average Runtime (seconds) 0.374301625 0.7104476375 1.11287894375 1.263543115 0.5193995 0.8852610125 1.2852265875 1.55048987 0.70110375 1.098921025 1.6092578375 1.85689055 1.0411995 1.5377812 2.24272095 2.53941444 0.2671005 0.4740847375 0.765801125 0.94418651 0.3772025 0.687762475 1.03632401875 1.28864399 0.67338575 1.15797815 1.7928188625 2.081943225 0.61149025 1.0192868125 1.48342026875 1.751548475 0.784670625 1.3914058 2.017145075 2.37001971 0.9343955 1.6150219

R1-2

R1-3

R1-4

R2

R3

R4-1

R4-2

R5-1

R5-2

12

D1

D2

D3

D4

A1-1

A1-2

A1-3

A2-1

A2-2

A2-3

20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams 1 stream 10 streams 20 streams 25 streams

2.326104475 2.70342093 0.4254266875 0.67727600625 0.9883220375 1.1591933925 0.2762340625 0.49984061875 0.7925614875 0.9211584 0.3439448125 0.6137608375 0.969213509375 1.1464113225 0.48321675 0.88516171875 1.345513390625 1.56276527 0.67762625 1.0925558 1.77672375 2.03975081 1.3362615 2.0676603 3.0319986125 3.62353857 2.0605925 3.15405455 4.469254975 5.01461024 1.5460765 2.7397760625 3.99316446875 4.90314213157894 1.838777 3.256667375 4.5519178125 5.32557834210526 3.816785 6.371394 9.88785475 11.9014413333333

13

14