LRN3322 - In-Memory - 10 - Year - Anniversary V3 - 1724971329244001resz
LRN3322 - In-Memory - 10 - Year - Anniversary V3 - 1724971329244001resz
"We're looking
forward to a future
of real-time
analytics at ungodly
speeds."
2 Oracle DatabaseWorld @ CloudWorld Copyright © 2024, Oracle and/or its affiliates
Real-Time Enterprises need In-Memory
SALES
Row Format
Fast for OLTP!
Slower for Analytics
Dual Format
(+) • Best for both Fast Analytics
and fast OLTP (No need for
SALES Analytic indexes)
Column Format
Fast for Analytics!
REALLY slow for OLTP!
Buffer Cache
In-Memory • BOTH row and column formats
Column Store
for same table
• Simultaneously active and
transactionally consistent
Benefits Benefits analytics
OLTP and reporting • Analytics & reporting use new
in-memory Column format
STATE
CPU
CA
Vector Register
Load Vector
CA
multiple CA
Compare
State all values
values in 1 cycle
CA
Access only the Scan & filter data in Prune out any Process multiple
columns you need compressed format unnecessary data column values in a
from the column single CPU instruction
AMOUNT
1–3 10 – 20 1–3
OLTP Analytic OLTP
Indexes Indexes Indexes
In-Memory
Column Store
Table Table
REPLACE
• Inserting one row into a table requires • Column Store not persistent so updates are: Fast!
updating 10-20 analytic indexes: Slow!
• Fast analytics on any columns
• Fast analytics only on indexed columns
• No analytic indexes: Reduces database size
• Analytic indexes increase database size
In-Memory
Buffer Cache
“When you update one Column Store
consistent between
those two formats.”
- Larry Ellison
Operational
Data
ITEMS
“When you put data in
memory, one of the Example:
Join Sales and Items
Vector Register
reasons you do that is SALES
Vector Register
to make it go faster.”
- Larry Ellison
change.”
Populate
Remove cold data,
Hot Table
populate hot data
- Larry Ellison
Hot Data
21 +
c
All the features from Oracle
21c Innovation Release
300+ New Features and
Enhancements
MB
Step 1. Customer starts advisor tracking
SQL> exec 1 0 6000
dbms_inmemory_advisor.start_tracking(); 1 1000 4500 Table Owner:
TPCH, Table Name:
Step 2. Customer stops advisor tracking LINEORDER,
PARTITION NAME
2023_MAY;
SQL> exec
dbms_inmemory_advisor.stop_tracking();
1 2000 3000 Table Owner:
Step 3. Customer generates report advisor tracking TPCH, Table Name:
LINEORDER,
SQL> exec PARTITION NAME
2023_MAY;
dbms_inmemory_advisor.generate_advise(); Table Owner:
TPCH, Table Name:
Step 4. Customer queries the view LINEORDER,
PARTITION NAME
dba_inmemory_advise_recommendation to view 2023_APR;
recommendation.
SQL> select * from
dba_inmemory_advisor_recommendation;
26 Oracle DatabaseWorld @ CloudWorld Copyright © 2024, Oracle and/or its affiliates
Automatic In-Memory
Management
• The remaining non-populated data can be accessed from Cell Memory In-Flash Columnar scans
Columnar Cache via Exadata Smart Scans
• The Wide Tables feature in 23ai allows up to 4096 columns per table, so we have
modified the "NO INMEMORY" syntax to make it easier to manage tables with large
numbers of columns
• Previously you could only exclude columns from population by specifying a column list
in the NO INMEMORY subclause
• Now in 23ai you can specify ALL columns as INMEMORY or NO INMEMORY and then
selectively include or exclude columns from population:
• ALTER TABLE <table name> NO INMEMORY(ALL) INMEMORY(col1, col2);
ITEMS
In-Memory Deep Vectorization Framework was
introduced in Oracle Database 21c
• The first feature that leveraged the framework Example:
improved the performance of single level hash Join Sales and Items
joins
Vector Register
SALES
In Oracle Database 23ai the In-Memory Deep
Vector Register
Vectorization Framework adds more join support:
• Multiple Levels of Joins
• Multiple Join Keys
• Semi and Outer Join
• Full Grouping and Aggregation
This translates to faster join performance by utilizing
SIMD optimizations during more types of join
processing
3
36 Oracle DatabaseWorld @ CloudWorld Copyright © 2024, Oracle and/or its affiliates
In-Memory Optimized Dates
Accelerating DATE Queries
Consider a query to find the Total sales amount for every month in 2022
select IM Column Store
Extract(month)
extract(month from order_date) MONTH, order_date
sum(order_amount) TOTAL_SALES Extract(year)
from SALES
where extract(year from order_date) = 2022
group by extract(month from order_date);
ADB tenants today automatically leverage Cell-Memory today. But In-Memory can be much faster.
3.6X 4X
4
2.3X 2X
2
0
SSB SF=300
SSB SF=300 EML
INTERNAL_1 MERCK
INTERNAL_2 DREY
INTERNAL_3 TPC-DS SF=300 TPC-H
TPC-DS SF=300 SF=300
TPC-H SF=300
As of Oracle Autonomous Database Dedicated version 19.20.0.1.0, you can enable Database In-Memory:
You must allocate 4 or more OCPUs.
You can enable between 5 and 70 percent of the available SGA per ADB.
In-Memory
(DB DRAM)
Cell-Memory
(Storage Flash)
Fusion ERP runs daily bulk update job using a single large transaction
Performance of concurrent queries drops by 12x without In-Memory (CR overhead)
In-Memory queries can ignore the active transaction because of the IM Private Journal (non RAC only)
Query Executions
Bulk update Bulk update
46
starts commits
Oracle DatabaseWorld @ CloudWorld Copyright © 2024, Oracle and/or its affiliates
Enterprise Performance Management with In-Memory
The Archives database tracks EPM customer metrics
• There are hundreds of partitions/tables – not all segments can fit In-Memory
• Queries typically access the newest partitions
• Daily report job takes 8 hours to complete
60GB of In-Memory space was provisioned based on 15 OCPU ATP shape
• I/O activity disappeared almost immediately after IM was enabled
• Daily report job now takes 4 hours to complete
• Ad hoc queries now complete in 15 seconds vs. more than a minute previously
• Auto IM works because good performance continues as new monthly partitions are created
PDB close, CPU reduction, and oversubscription adjustment can shrink In-Memory space
In-Memory data could be auto distributed on different instances when CPU count is high
Service MEDIUM or HIGH is required to take full advantage of the distributed In-Memory footprint
Users need to know the available In-Memory space to fit their dataset
• Users can check sga_target value from v$parameter
Fusion Enterprise “Significant boost seen” with IM vs Cell Memory (4x gains) “
Resource Planning
Application
“Queries in some cases that took 56 seconds to complete ran in 400 milliseconds for
Performance the same data set when using In-Memory”
Monitoring
“Power BI Vertipaq and Tableau Hyper are faster than OAC today. We do see a
Analytics Cloud throughput increase with use of IM and we see specific queries which are faster”
Per instance
• 12 OCPUs
• 20 GB In-Memory
PQ is needed if table is
distributed over RAC
IM query: 2s
Non-IM query: 16s
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
Hisense
What’s the Use-Case?
They use SAP ERP system for both domestic and international business
• Domestic runs on traditional HW and overseas business on Exadata (previously SUN T5 systems)
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
Hisense
How did In-Memory help
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
Hisense
In-Memory Features and Technologies used:
• Compression Advisor
• In-Memory Advisor (used during evaluation and production)
• IM Duplicate (with Exadata) –fast performance because of local access
• In-Memory Distribute for Service on RAC
• Foundational Features (e.g. SIMD Vectorization, IM Storage Indexes)
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
Stef-IT
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
Stef-IT
What’s the Use-Case?
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
MyToys
Oracle CloudWorld Copyright © 2022, Oracle and/or its affiliates | Confidential: Restricted
myToys
What’s the Use-Case?
Cloud
On-Prem
Mixed Workloads:
“There’s actually very little overhead in
maintaining the column store in-memory in
addition to Oracle’s traditional row store”
Automatic In-Memory Management:
“You say how much memory you want to use in
the computer, tell me what partitions or tables to
be in memory, and drop your analytic indexes”
Performance:
“Queries run 100 times faster and updates,
inserts run two, three times faster.”