Smart analytic optimizer how it works

Smart Analytics Optimizer for DB2 for z/OS:
How Does it Work?

Willie Favero – Data Warehouse on System z Swat Team (DB2 SME)
wfavero@us.ibm.com
(713) 940-1132
Copyright © 2010 IBM Corporation

Notices
This information was developed for products and services offered in the U.S.A.

Note to U.S. Government Users Restricted Rights — Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

IBM may not offer the products, services, or features discussed in this document in other countries. Consult your local IBM representative for information on the products
and services currently available in your area. Any reference to an IBM product, program, or service is not intended to state or imply that only that IBM product,
program, or service may be used. Any functionally equivalent product, program, or service that does not infringe any IBM intellectual property right may be used
instead. However, it is the user's responsibility to evaluate and verify the operation of any non-IBM product, program, or service.
IBM may have patents or pending patent applications covering subject matter described in this document. The furnishing of this document does not give you any license to
these patents. You can send license inquiries, in writing, to: IBM Director of Licensing, IBM Corporation, North Castle Drive Armonk, NY 10504-1785 U.S.A.
The following paragraph does not apply to the United Kingdom or any other country where such provisions are inconsistent with local law: INTERNATIONAL BUSINESS
MACHINES CORPORATION PROVIDES THIS PUBLICATION "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT
NOT LIMITED TO, THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Some states do
not allow disclaimer of express or implied warranties in certain transactions, therefore, this statement may not apply to you.
This information could include technical inaccuracies or typographical errors. Changes are periodically made to the information herein; these changes will be incorporated
in new editions of the publication. IBM may make improvements and/or changes in the product(s) and/or the program(s) described in this publication at any time
without notice.

Any references in this information to non-IBM Web sites are provided for convenience only and do not in any manner serve as an endorsement of those Web sites. The
materials at those Web sites are not part of the materials for this IBM product and use of those Web sites is at your own risk.

IBM may use or distribute any of the information you supply in any way it believes appropriate without incurring any obligation to you.

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has
not tested those products and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the
capabilities of non-IBM products should be addressed to the suppliers of those products.

This information contains examples of data and reports used in daily business operations. To illustrate them as completely as possible, the examples include the names of
individuals, companies, brands, and products. All of these names are fictitious and any similarity to the names and addresses used by an actual business enterprise is
entirely coincidental.

COPYRIGHT LICENSE:
This information contains sample application programs in source language, which illustrates programming techniques on various operating platforms. You may copy,
modify, and distribute these sample programs in any form without payment to IBM, for the purposes of developing, using, marketing or distributing application
programs conforming to the application programming interface for the operating platform for which the sample programs are written. These examples have not been
thoroughly tested under all conditions. IBM, therefore, cannot guarantee or imply reliability, serviceability, or function of these programs. You may copy, modify, and
distribute these sample programs in any form without payment to IBM for the purposes of developing, using, marketing, or distributing application programs
conforming to IBM's application programming interfaces.

Smart Analytics Optimizer for DB2 for z/OS: How Does it Work Slide 2 of 59
All rights reserved

Trademarks

This presentation contains trade-marked IBM products and technologies. Refer to the
following Web site:

http://www.ibm.com/legal/copytrade.shtml

All rights reserved

Topics

The IBM Smart Analytics Optimizer
–The basics
–Beta customer experience
–Under the hood
–Setting it up

Is it right for me?

All rights reserved

Business Challenges and Technology Trends

Changing business requirements
– BI/DW becoming mission critical and requires OLTP-like QoS
• reliability, continuous availability, security, mixed workload management, …
• orders of magnitude faster execution of complex, ad hoc queries
• predictable query performance
– Shift towards dynamic DW and operational BI
• Combining OLTP and OLAP workloads
Traditional performance tuning tools of the trade such as indexing, prebuilt
aggregates and MQTs struggling to keep the pace
– Require top DBA expertise and sophisticated tools
– Even then not good enough due to ad-hoc, unpredictable nature of the
workload
Technology trends
– Very large number of processor sockets and cores
– Massive amounts of real memory
– Specialized physical data designs: row-store vs. column-store

All rights reserved

The IBM Data Warehouse Solution on System z
Ultimate Consolidation Opportunity
z/OS LPAR
z/OS LPAR z/OS LPAR
z/OS LPAR Consolidation of mission-critical data
Operational
Operational Enterprise
Enterprise
System (OLTP)
System (OLTP) Data Warehouse
Data Warehouse on System z
Ability to leverage existing
environment, high availability, backup
Data
Data
Sharing
Sharing DB2 for z/OS
DB2 for z/OS ELT DB2 for z/OS
DB2 for z/OS
and governance procedures as well
ELT
as skills
Group
Group

Efficient data movement within a
data sharing group (no network)
z/VM LPAR
z/VM LPAR
Performance and TCO
Linux on System z
Information
Linux on System z
InfoSphere
Linux on System z
Cognos improvements through cubing
Server Warehouse BI Server services (data marts) and DB2
Replication
Server enhancements
Complex transformations and data
quality driven from Linux on System
z with Information Server
All rights reserved

IBM zEnterprise System –
Best in Class Systems and Software Technologies
A system of systems that unifies IT for predictable service delivery
Unified management for a smarter system:
Unified management for a smarter system:
zEnterprise Unified Resource Manager
zEnterprise Unified Resource Manager
Part of the IBM System Director family,
provides platform, hardware and workload Scale out to a trillion
Scale out to a trillion
The world’s fastest and
The world’s fastest and instructions per second:
instructions per second:
management
most scalable system:
most scalable system: IBM zEnterprise
IBM zEnterprise
Unifies management of resources,
IBM zEnterprise™ 196
IBM zEnterprise™ 196 extending IBM System z® qualities of BladeCenter® Extension
BladeCenter® Extension
(z196)
(z196) service across the infrastructure (zBX)
(zBX)
Ideal for large scale Selected IBM POWER7™ blades
and IBM x86 blades1 for tens of
data and transaction thousands of AIX® and Linux
serving and mission applications
critical applications High performance optimizers and
Most efficient platform appliances to accelerate time to
for Large-scale Linux® insight and reduce cost
consolidation Dedicated high performance private
network
Leveraging a large
portfolio of z/OS® and
Linux on System z
applications
Capable of massive
scale up, over 50 Billion HMC
Instructions per Second
(BIPS)
1 All statements regarding IBM future direction and intent are subject to change or withdrawal without notice,
and represents goals and objectives only.

All rights reserved

IBM Smart Analytics Optimizer – a Virtual DB2 Component
Applications DBA Tools, z/OS Console, ...
Application Interfaces Operation Interfaces
(standard SQL dialects) (e.g. DB2 Commands)

DB2

Data Buffer ... Log
IRLM
Manager Manager Manager

Superior availability
reliability, security,
workload management ...

z/OS on System z
10‘s of processors
100‘s GB of memory

All rights reserved

IBM Smart Analytics Optimizer – a Virtual DB2 Component
Applications DBA Tools, z/OS Console, ...
Application Interfaces Operation Interfaces
(standard SQL dialects) (e.g. DB2 Commands)

DB2
IBM
Data Buffer Log Smart
... IRLM
Manager Manager Manager Analytics
Optimizer

Abundance
of resources
Superior availability
reliability, security,
workload management ...

z/OS on System z Linux on blades
10‘s of processors 100‘s of processors
100‘s GB of memory 1000‘s GB of memory

All rights reserved

Characteristics of the IBM Smart Analytics Optimizer

A special purpose, network attached blades system
No changes to the applications
– Applications continue to attach to DB2
– When applicable query needs to be executed DB2 exploits the accelerator
transparently to the applications
– Full fencing and protection of DB2 against possible accelerator failures
Order of magnitude performance improvement
Reducing need for tedious tuning of DB2 (MQTs, indexes, etc.)
Appliance-like form-factor
– Hands free operations
Significantly improved price/performance and TCO as a combined effect of:
– Offloading very CPU intensive operations
– Orders of magnitude performance improvement for accelerated queries
– Reduced DBA effort for tuning accelerated queries
Hybrid technology
– Enabling Dynamic DW and Operational BI
– Preserving traditional DB2 quality of service
– Having transactional and analytical workload being handled by DB2
All rights reserved

Defining, which data to accelerate

A MART is a logical collection of tables which are related to each other. For
example all tables of a single star schema would belong to the same MART.
The administrator uses a rich client interface to define the tables which belong to a
MART together with the information about their relationships.
DB2 for z/OS creates definitions for these MARTs in the own catalog. The related
data is read from the DB2 tables and transferred to the Smart Analytics Optimizer.
The accelerator transforms the data into a highly compressed, scan optimized
format which is kept locally (in memory) on the Accelerator.

Define Transform

All rights reserved

Smart Analytics Optimizer: Mart Definition and Deployment
IBM Data Studio Client DB2 for z/OS IBM Smart Analytics
Optimizer
Smart Analytics
Optimizer Smart Analytics
Administrative Optimizer Catalog
Stored Procedures
Smart Analytics
Optimizer Studio DB2 Catalog

• Smart Analytics Optimizer marts need to be defined and deployed to Smart Analytics Optimizer before
data is loaded and queries sent to Smart Analytics Optimizer for processing.
- Definition: identifying tables and relations that make up marts.
- Deployment: making marts known to DB2, i.e. storing mart meta data in the DB2 and Smart Analytics Optimizer
catalog.
• Smart Analytics Optimizer Studio guides you through the process of defining and deploying marts, as well
as invoking other administrative tasks.
• Smart Analytics Optimizer Stored Procedures implement and execute various administrative operations
such as mart deployment, load and update, and serve as the primary administrative interface to Smart
Analytics Optimizer from the outside world including Smart Analytics Optimizer Studio.

All rights reserved

Browse the Mart With Table Relations

All rights reserved

Zoom In: Size Estimates and Fact table property

All rights reserved

Data View: Marts – Redundant Sets of Memory Resident Tables
Smart
Analytics
DB2 for z/OS Optimizer
memory memory
Mart 1
disk

l es
t ab
disk

x es
n de Mart 2
i
Ts
Q
M
• DB2 continues to own and manage all data • Smart Analytics Optimizer contains fully memory resident,
• Access performance is influenced by traditional compressed copies of performance critical tables grouped into
logically connected marts, bound ideally by star schema
marts
tuning mechanisms such as indexing, MQTs, constraints
aggregates, ... • Similar but much broader than MQTs: no column projections, no
• Typical usage: Enterprise Data Warehouse, large row restrictions, no row aggregations
Data Marts • Typical usage: Data Marts, MQTs consolidation and replacement

All rights reserved

Smart Analytics Optimizer Mart Load

DB2 for z/OS Smart Analytics Optimizer
Smart Analytics Optimizer Administrative Stored Procedures
Disk
Backup
Table A
Table B
Worker 1
Part 1 Unload USS Pipe

Table C
Worker 2

Smart
Part 2 Unload USS Pipe Worker 3
Analytics

Coordinator
Table D
Optimizer .
Part 1
Studio . . . .
. . . .
Part 2 . . .

Worker n-1
Part 3 Unload USS Pipe
Part m
Worker n

All rights reserved

Smart Analytics Optimizer Mart Update

IDUG study on DW update frequency Typical DW update operations:
– LOAD RESUME and REPLACE
– ADD and ROTATE PARTITION
– SQL INSERT, UPDATE, DELETE
– Delete complete partition or table
– TRUNCATE TABLE

Smart Analytics Optimizer will over time
phase-in support for all the typical operations
in this order
1. Full table reload
2. Updated partition reload
3. Individual row change

The marts update is initiated and controlled
through Smart Analytics Optimizer Studio

Queries off-loaded to Smart Analytics
Optimizer before the marts are refreshed can
In 90% of cases DW is updated once a day or return different result set as compared to not
being off-loaded
even less frequently – In case this is not acceptable use
SET CURRENT REFRESH AGE = 0

All rights reserved

Pause for
Questions

All rights reserved

IBM Smart Analytics Optimizer Node Overview

Query parsing and matching.

DB2 Is routing single query blocks
to the optimizer.

Remains the „owner“ of all
data.

Orchestrating the distributed
Coordinator Coordinator tasks like LOAD or QUERY
Node Node execution.

Communication peers to
DB2

Have all the data in main
memory (spread across all
Worker Node Worker Node Worker Node Worker Node
worker nodes).

Do the compression and
query execution.

Worker Node Worker Node Worker Node Worker Node Always work together on a
distributed task.

All rights reserved

Query Execution Process Flow
Heartbeat
Application Optimizer
Interface
Worker 1

Smart Analytics Optimizer interface
Worker 2

Worker 3

Coordinators
.
Application
.
.
Query execution run-time for
queries that cannot be or should
not be off-loaded to Smart Worker n-1
Analytics Optimizer
Worker n

DB2 for z/OS Smart Analytics Optimizer
Heartbeat (Smart Analytics Optimizer availability and performance indicators)
Queries executed without Smart Analytics Optimizer
Queries executed with Smart Analytics Optimizer
All rights reserved

Additional DB2 Support - EXPLAIN
DSN_QUERYINFO_TABLE
Whether or not a query block is
eligible for automatic query rewrite,
and if not eligible show the reason
why it's not eligible.
If eligible for automatic query rewrite,
which materialized/accelerated query
tables were considered, and for each
one that wasn't chosen the reason
why it was not chosen.
– REASON_CODE – why AQT
matches and why not
– QI_DATA – part of SQL statement
that causes failing match

All rights reserved

Additional DB2 Support - Commands

Stop Accel-Command "-STO ACCEL(IWA001)"

Start Accel-Command "-STA ACCEL(IWA001)"

All rights reserved

Display Accelerator Command
DSNX810I -DB91 DSNX8CMD DISPLAY ACCEL FOLLOWS -
DSNX830I -DB91 DSNX8CDA
ACCELERATOR MEMB STATUS REQUESTS QUED MAXQ FAIL
-------------------------------- ---- -------- -------- ---- ---- ----
BACGEN DB91 STARTED 0 0 0 0
PORT=0 IPNAME=VIRTUAL
DEMOACC1 DB91 STARTED 365 0 1 28
PORT=1400 IPNAME=897A45D3
IPADDR=192.168.139.100 HEALTHY
DETAIL STATISTICS
LEVEL = AQT01010
STATUS = FULLY OPERATIONAL
AVERAGE QUEUE WAIT = 0
MAXIMUM QUEUE WAIT = 0
PROCESSING CAPACITY = 1314278
TOTAL NUMBER OF PROCESSORS = 224
AVERAGE CPU UTILIZATION ON COORDINATOR NODES = .35%
NUMBER OF ACTIVE COORDINATOR NODES = 3
PHYSICAL MEMORY AVAILABLE ON COORDINATOR NODES = 47622 MB
PHYSICAL MEMORY AVERAGE USAGE ON COORDINATOR NODES = 653 MB
AVERAGE CPU UTILIZATION ON WORKER NODES = .24%
NUMBER OF ACTIVE WORKER NODES = 11
PHYSICAL MEMORY AVAILABLE ON WORKER NODES = 40395 MB
PHYSICAL MEMORY AVERAGE USAGE ON WORKER NODES = 7879 MB
SHARED MEMORY DATA AVAILABLE ON WORKER NODES = 25423 MB
SHARED MEMORY DATA AVERAGE USAGE ON WORKER NODES = 6728 MB
MAXIMUM SHARED MEMORY DATA IN USE ON WORKER NODES = 6741 MB
DISPLAY ACCEL REPORT COMPLETE
DSN9022I -DB91 DSNX8CMD '-DISPLAY ACCEL' NORMAL COMPLETION

All rights reserved

Display Thread Command

DSNV401I -DB91 DISPLAY THREAD REPORT FOLLOWS -
DSNV402I -DB91 ACTIVE THREADS -
NAME ST A REQ ID AUTHID PLAN ASID TOKEN
BATCH AC * 7 PBECKERD PBECKER DSNTIB91 0035 737
V666 ACC=DEMOACC1,ADDR=192.168.139.100..1400:9991
V441-ACCOUNTING=56000999:00000000:47630000
V436-PGM=*.DSNTIAUL, SEC=2, STMNT=3644
TSO T 307 FNEUMAN FNEUMAN ADB 0080 417
V441-ACCOUNTING=3300
TSO T 7 PBECKER PBECKER ADB 0070 751
TSO T * 3 PBECKER PBECKER 0070 756
DISPLAY ACTIVE REPORT COMPLETE
DSN9022I -DB91 DSNVDT '-DIS THREAD' NORMAL COMPLETION
***

All rights reserved

Preconfigured Solution Sizes for the IBM Smart Analytics
Optimizer
Blades 7 14 28 42 56

Handles up to xx of Raw
0.5TB 1TB 2TB 3TB 4TB
DB2 Table Data

Enterprise Rack 1 1 1 2 2
DS5020
BladeCenter H Chassis 1 1 2 3 4

DS5020 with # of SATA disks
16 16 16 32 32
(1 TB)

Storage Expansion Units 0 0 0 1 1
zEnterprise 196 zBX Model 002

• Includes redundant hardware infrastructure for availability including PDUs, TOR 10GbE
switches for data, TOR 1GbE switches for management, FC switches, etc…

• For zEnterprise 196, order Machine Type 2458 Model 002 with feature #0610

• DS5020 disk needs to be ordered separately from zBX, and housed in its own rack.

All rights reserved

zBX with Smart Analytics Optimizer Sysplex configuration
zBX / SAOpt All members of a DB2 data
sharing group can access a
DS5020
shared Smart Analytics Optimizer.

Additional standalone DB2
Switch subsystems may share the same
Smart Analytics Optimizer.

Coupling Facility
Coupling Facility

DB2 Subsystem

DB2 Subsystem

DB2 Subsystem
DB2 Subsystem
DB2 Member

DB2 Member

DB2 Member
DB2 Member
DB2 Member

DB2 Member

LPAR LPAR LPAR LPAR LPAR LPAR LPAR LPAR LPAR
System z System z System z

DS8x00 DS8x00
external external
DB2 Storage DB2 Storage

All rights reserved

Multiple Smart Analytics Optimizer Configuration
zBX / SAOpt zBX / SAOpt

Storage Storage

1 DB2 Subsystem 2 Not Allowed ????

DB2 Subsystem

DB2 Subsystem
DB2 Subsystem

1 OK

One DB2 subsystem must
LPAR LPAR LPAR
not access more than one
System z
Smart Analytics Optimizer /
zBX.
2
DS8x00 Only one Smart Analytics
external Optimizer / zBX may be
DB2 Storage
connected to one CEC.

All rights reserved

Testing Results – Native on DB2
The problem queries provided by a customer
Expert database tuning done on all the queries
– Q1 – Q6 even after tuning run far too long and consume lots of resources
– Q7 improved significantly – no Smart Analytics Optimizer offload is needed
The table shows elapsed and CPU times measured in DB2 (without Smart Analytics
Optimizer)

Times measured in DB2 without Smart Analytics Optimizer
Query Total Elapsed CP zIIP Total CPU Time
Q1 0:02:43 0:03:52 0:02:39 0:06:31
Q2 0:38:31 0:11:52 0:36:10 0:48:02
Q3 0:00:25 0:00:04 0:00:15 0:00:19
Q4 0:26:33 0:13:43 0:20:50 0:34:33
Q5 0:00:35 0:00:09 0:00:29 0:00:38
Q6 1:30:35 5:53:30 1:29:56 7:23:26
Q7 0:00:02 0:00:02 0:00:00 0:00:02

All rights reserved

Testing Results – Smart Analytics Optimizer
Performance Improvement after Adding IBM Smart Analytics Optimizer
Query Elapsed Time
Query DB2 with Smart Analytics
DB2 only Speed-up
Optimizer

Q1 0:02:43.0 0:00:03.4 48

Q2 0:38:31.0 0:00:04.5 511

Q3 0:00:25.0 0:00:02.2 12

Q4 0:26:33.0 0:00:07.8 206

Q5 0:00:35.0 0:00:08.3 4

Q6 1:30:35.0 0:00:03.8 1424

Q7 0:00:02.0 0:00:02.0 1

Total 2:39:24.0 0:00:32.0 298

All rights reserved

What Is Smart Analytics Optimzer Ideally Suited For?
REGION
Star or snowflake schema Complex, OLAP-style queries that typically:
• Need to scan large subset of data (unlike
MONTH
CITY QUARTER OLTP queries)
• Involve aggregation function such as
COUNT, SUM, AVG.
STORE
• Look for trends, exceptions to assist in
PERIOD making actionable business decisions

SALES
SELECT PRODUCT_DEPARTMENT, REGION, SUM(REVENUE)
FROM FACT_SALES F
INNER JOIN DIM_PRODUCT P ON F.FKP = P.PK
INNER JOIN DIM_REGION R ON F.FKR = R.PK
PRODUCT
LEFT OUTER JOIN DIM_TIME T ON F.FKT = T.PK
WHERE T.YEAR = 2007
AND P.TYPE = ‘SOFTWARE‘
CATEGORY AND R.GEO = ‘SOUTH‘
BRAND
GROUP BY PRODUCT_DEPARTMENT, REGION

All rights reserved

The “Sweet – Spot” Schema

Small dimension tables


Big fact tables


All rights reserved

Join Strategy within the Cluster

The Fact table is split into multiple parts and distributed evenly across the
Worker nodes within the cluster.
–Bigger Fact tables “just“ require enough Worker nodes to contain the
compressed data in memory.
The Join Strategy between Dimension Tables and the Fact table data is
always a collocated join.
–This means that all dimension tables are fully replicated to each of the
worker nodes.
–Space requirements for dimension tables therefore needs to be
multiplied with cluster size (amount of Worker Nodes)

All rights reserved

Matching of queries for Smart Analytics Optimizer

DB2 for z/OS reuses partial MQT functionality to find out which queries
are eligible for Smart Analytics Optimizer offload and which are not.
This implies that a subset of the MQT restrictions is inherited
–Only a single query block at a time can be routed to the Optimizer
(Queries which consist of several Query Blocks, are not seen as whole
query by the accelerator but only as single, independent blocks)
–The results of subqueries can not be used by the Optimizer in the outer
query (DB2 would need to pass a subselect result to Smart Analytics
Optimizer )

All rights reserved

Examples with potentially multiple query blocks
Derived table (nested table expression)
SELECT * FROM
(SELECT C1+C2 FROM TA) TX

Derived table (common table expression=CTE)
WITH DTOTAL (deptno, totalpay) AS
The intermediate results (SELECT deptno, sum(salary+bonus)
(data) may ONLY flow
from the accelerator to FROM DSN8810.EMP GROUP BY deptno)
DB2 and not the other way SELECT deptno FROM DTOTAL
WHERE totalpay = (SELECT max(totalpay) FROM
DTOTAL);

IN predicate with subquery
SELECT ... FROM ... WHERE ...
AND ( (A11.STORE_NUMBER IN
(SELECT C21.STORE_NUMBER
FROM USRT004.VL_CSG_STR C21
WHERE C21.CSG_NUMBER in (4643) ))

All rights reserved

Supported schemas

A MART consists of a set of tables together with their referential
constraints.
–Fact tables are considered to be the tables which have the highest join
depth.
Only Queries, including at least the fact table, can be routed to Smart
Analytics Optimizer (Queries which are only scanning the dimensions
have to be handled by DB2)
Multiple fact tables are allowed within the same MART definition but:
–Queries can not handle table across MART boundaries

D1
D1
DA
DA F1
F1 DB
DB D3
D3
D2
D2

F2
F2

All rights reserved

Summary why a query may not be routed

Because it uses static SQL
Because it uses CURRENT REFRESH AGE = 0
Because it contains syntax that is not supported (e.g. Subselect or full
outer join)
Because the accelerator or MART are disabled
Because it references a table or column that is not in the accelerated mart
(may be due to unsupported datatypes)
Because the query does not reference a fact table
Because the optimizer decides DB2 for zOS can do better (DB2 has a
cost-based threshold)
–E.g. Query with selective predicate on indexed column is executed in
DB2

All rights reserved

First Release Restrictions
Limited support for very large dimension tables
–Especially if the predicates on them are not selective
No static SQL
No full outer join, no right outer join
Only equi-joins (no range join predicates)
No queries that do not include at least one fact table
No queries that spread across multiple marts
Not all DB2 functions
–No mathematical functions such as SIN, COS, TAN.
–No user defined functins
–No advanced string functions such as LOCATE, LEFT, OVERLAY.
–No advanced OLAP functions such as RANK, ROLLUP, CUBE
Not all DB2 data types such as LOBs, ROWID, XML
All rights reserved

Pause
simply for effect

All rights reserved

Data Types Support

Not supported:
Any kind of LOB
ROWID
XML
Binary data
Decimal Float

All rights reserved

Prerequisits
DB2 9 for z/OS – With maintenance
z/OS 1.10 - With maintenance
USS pipe support for TEMPLATE utility - PTF UK43948 for APAR
PK70269
XML Toolkit for z/OS V1.10.0, Program Number 5655-J51, FMID
HXML190
Data Studio V2.2 (IDE, no-charge download)
IBM Smart Analytics Optimizer Studio
System z BladeCenter Extension (zBX)
–z196 w/zBX
OSA Express3 10 GbE connection
PC Windows XP or higher, 5 GB free disk, 2 GB RAM
All rights reserved

Options for Workload Analysis

Stage Purpose
Questionnaire Initial assessment based on size, query
response time, update characteristics and
customer pain points

Quick Workload Test Assessment based on dynamic customer
workload, runtime statistics, table sizes and
SQL.

Detailed Online Workload Analysis Assessment based on data mart definition
for customer data model and offload
capabilities in a real Smart Analytics
Optimizer environment. Addresses all
inhibitors for offload and data mart
definition questions.

All rights reserved

Quick Workload Test
Report for a first assessment:
Query offload potential
Estimated performance gain
Customer
– Collecting information from dynamic Estimated CP cost saving
statement cache, supported by step-by-step
instruction and REXX script (small effort for
customer)
– Uploading compressed file (up to some MB)
to IBM FTP server
IBM / Center of Excellence
– Importing data into local database
– Quick analysis based on known Smart
Analytics Optimizer capabilities

1 2 3
Data package
Data package
Documentation Pre-process and Quick Workload Report
Customer
Customer Documentation (mainly unload
(mainly unload Pre-process and IBM lab
IBM lab Quick Workload
and REXX procedure
and REXX procedure load
load Test Tool
Test Tool Assessment
Database
Database data sets)
data sets) Database
Database

All rights reserved

Start Trace for Cache (1 of 2)

Non-Data-Sharing
– -START TRACE(MON) CLASS(1) IFCID(316,317,318) DEST(SMF)
Data-Sharing
– -START TRACE(MON) CLASS(1) IFCID(316,317,318) DEST(SMF)
SCOPE(GROUP)
Verify
– -DIS TRACE(*)

All rights reserved

Start Trace for Cache (2 of 2)

IFCID 316
– First 60 bytes of SQL statement plus identifying information and statistics
• http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/index.jsp?topic=/com
.ibm.omegamon.xe_db2.doc/ko2rrd20228.htm
IFCID 317
– Used in addition to IFCID 316 to obtain the full SQL statement text
• http://publib.boulder.ibm.com/infocenter/tivihelp/v15r1/index.jsp?topic=/com
.ibm.omegamon.xe_db2.doc/ko2rrd20228.htm
IFCID 318
– Acts as a switch for IFCID 316 to collect all available information
• Stop and Start for new interval

All rights reserved

Create Tables Used by EXPLAIN

Create tables used by EXPLAIN
– Optimization Service Center (OSC)
• or
– Optim Query Tuner (was Optimization Expert)
• or
– DSNTIJOS job in SDSNSAMP
• or
– A copy of DSNTIJOS is also included in the Assessment ZIP file

All rights reserved

Get Dynamic Statement Cache

Execute
– EXPLAIN STMTCACHE ALL
– Can use SPUFI, DSNTEP2, or anything that allows this SQL statement to be
executed
Ensure SQLID (SCHEMA) is set correctly for the EXPLAIN tables
Verify/validate correct information was gathered

All rights reserved

Run REXX Exec

Provided REXX exec will EXPLAIN all SQL dumped from the statement cache
using
– owner.PLAN_TABLE
– owner.DSN_STATEMNT_TABLE
– owner.DSN_FUNCTION_TABLE
– owner.DSN_PREDICAT_TABLE
– owner.DSN_STRUCT_TABLE
– owner.DSN_QUERY_TABLE
• with CLOB column for statement text in XML format (V9)
– owner.DSN_DETCOST_TABLE

All rights reserved

Unload Everything and Send

Using supplied job, unload EXPLAIN tables and selected DB2 Catalog tables
• Read documentation carefully and pay attention to warnings
Prepare unloads using ADRDSSU and TRSMAIN
FTP to testcase at boulder

All rights reserved

IBM Smart Analytics Optimizer - Summary
Summary of Value Proposition Seamless integration of new computing
paradigms into proven technology
– Massive multi-core and vector processing
– In-memory database
Database Administration BI/DW Applications
– No changes to the applications; applications
Cubing continue to attach to DB2
Caching • When applicable query needs
execution, DB2 transparently
exploits Smart Analytics Optimizer
– Preserving traditional System z and DB2
Quality of Service (QoS), full fencing and
DB2 APIs protection of DB2 against possible Smart
Analytics Optimizer failures
OLTP

Smart Analytics
– Hands free operation
data
Optimizer Augmenting System z value proposition as the
overall Enterprise Data Hub
– System z hybrid topology enables additional
transparency and management integration
DW Order of magnitude performance improvement
data – Linear scaling with the number of CPUs
Reducing need for tedious tuning of DB2 (MQTs,
aggregates, indexes, etc.)
Providing building block for Dynamic Data
Warehousing and Operational BI

All rights reserved

50TB System z Data Warehouse Benchmark

Poughkeepsie Lab / Silicon Valley Lab joint effort
Deliver proof points of System z scalability in Business
Intelligence environment
Establish capability of System z to scale to larger volumes
Develop best practices of managing large data warehouses
Drive unique value of System z
– Workload Manager: capabilities to manage mixed
workloads
– Operational BI: large volumes of users, smaller queries
– zIIP: lower cost BI solution
– Data compression: minimal overhead with hardware
compression
SG24-7674

All rights reserved

Co-locating Transactional and Data Warehouse Workloads

Poughkeepsie Lab / Silicon Valley Lab joint effort
Managing analytic and traditional workload best practices
Using workload manager (WLM) to manage mixed workloads
OLAP vs data warehouse workloads
Data sharing vs non data sharing considerations
Single vs multiple LPAR configurations
Implementation considerations
Benchmark results

SG24-7726

All rights reserved

Other Significant System z Data Warehouse Redbooks

SG24-7637 SG24-7813 REDP-4606
Enterprise Data InfoSphere Warehouse: A Using IBM System z As the
Warehousing with DB2 9 Robust Infrastructure for Foundation for Your
for z/OS Business Intelligence Information Management
Architecture

All rights reserved

More Significant System z Data Warehouse Redbooks

REDP-4668
REDP-3927 SG24-6489
IBM zEnterprise
Workload Management Best Practices for SAP BI
BladeCenter Extension
for DB2 Data using DB2 9 for z/OS

All rights reserved

More information on zEnterprise

IBM zEnterprise Announcement Landing Page:
ibm.com/systems/zenterprise196
IBM zEnterprise HW Landing Page: ibm.com/systems/zenterprise196
IBM zEnterprise Events Landing Page: ibm.com/systems/breakthrough
IBM Software: ibm.com/software/os/systemz/announcements
IBM System Storage: ibm.com/systems/storage/product/z.html
IBM Global Financing: ibm.com/financing/us/lifecycle/acquire/zenterprise/
Global Technology Services:
–vanity: ibm.com/services/zenterprise

All rights reserved

Thai
Traditional Chinese

Russian
Gracias Spanish

Thank You English

Merci
French
Obrigado
Brazilian Portuguese
Arabic

Bedankt Danke
German
Nederlands
Simplified Chinese

Japanese

All rights reserved

Willie Favero
Senior Certified Consulting IT Software Specialist
Dynamic Warehousing on System z Swat Team
IBM Silicon Valley Laboratory
IBM Academic Initiative Ambassador for System z
IBM Certified Database Administrator - DB2 Universal Database V8.1 for z/OS
IBM Certified Database Administrator – DB2 9 for z/OS
IBM zChampion

wfavero@attglobal.net

All rights reserved

Smart analytic optimizer how it works

More Related Content

What's hot (16)

Viewers also liked (9)

Similar to Smart analytic optimizer how it works (20)

More from Willie Favero (12)

Recently uploaded (20)

Smart analytic optimizer how it works