New lessons in connection management

WhyThisTalk?
Connection pool on
application server
(UCP)
Connection pool in
database server
(DRCP)
Connection pool
between application
and database server
(PRCP)
Pools
Pools, everywhere…
What are these? How do they work? Does combining
them, make sense?
At end of talk, you’ll
be comfortable with
these questions

• Connections, Processes, Sessions
• Recap: UCP, Universal Connection Pool
• Connection Reservation
• SizingYour Connection Pool
• DRCP: Database Resident Connection Pool
• Combine UCP with DRCP?
• PRCP: Proxy Resident Connection Pool
• Combine UCP with PRCP?
Overview

• Connection
• Process
• Session
Connecting with Dedicated Servers
TNSNAMES.ORA:
your_alias =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = your_host)(PORT = your_port))
(CONNECT_DATA = (SERVER = DEDICATED)
(SERVICE_NAME = your_service)))

Connecting with Dedicated Server
sqldev
Database Instance
Oracle
Listener
Port 1521
Port 1801 Dedicated Server
CONNECT
GOTO 1801
: UGA, session state
sqldev
Dedicated Server
Port 1802
sqldev
Dedicated Server
Port 1803
sqldev
Dedicated Server
Port 1804
sqldev
Dedicated Server
Port 1805
sqldev
Dedicated Server
Port 1806
sqldev
Dedicated Server
Port 1807
authenticate
Session is created

Connection vs. Process vs. Session
sqldev
Database Instance
sqldev
Dedicated Server
Port 1802
sqldev
Dedicated Server
Port 1803
sqldev
Dedicated Server
Port 1804
sqldev
Dedicated Server
Port 1805
sqldev
Dedicated Server
Port 1806
sqldev
Dedicated Server
Port 1807
Oracle
Listener
Database connection
per client port-to-port
Database OS process per
client (aka foreground)
Database session per
client (in PGA)

What’s in a Session?
sqldev
Database Instance
sqldev
Dedicated Server
Port 1802
sqldev
Dedicated Server
Port 1803
sqldev
Dedicated Server
Port 1804
sqldev
Dedicated Server
Port 1805
sqldev
Dedicated Server
Port 1806
sqldev
Dedicated Server
Port 1807
Oracle
Listener
Authenticated user, transaction state , open-cursors,
bind-values, fetch position queries, NLS-settings,
PL/SQL package states, GTT pointers, …

Connecting
sqldev
Database Instance
sqldev
Dedicated Server
Port 1802
sqldev
Dedicated Server
Port 1803
sqldev
Dedicated Server
Port 1804
sqldev
Dedicated Server
Port 1805
sqldev
Dedicated Server
Port 1806
sqldev
Dedicated Server
Port 1807
Oracle
Listener
Connecting with dedicated server is
expensive for database:
1) OS process needs to be created
2) DB session needs to be created

A Database Call
Database Instance
sqldev
Dedicated Server
Time spent on
CPU of client
Database call
Time spent on
network
Results
Time spent on
network
Time spent in
database
User initiates
request
Client on CPU
On network
Foreground on CPU
time
Note: 1 process context switch
Session per client: state
can be maintained across
DB-calls

• Connection
• Process
• Session
Connecting with Shared Servers
TNSNAMES.ORA:
your_alias =
(DESCRIPTION =
(CONNECT_DATA = (SERVER = SHARED)
SPFILE (INIT.ORA):
DISPATCHERS = (PROTOCOL=TCP)(DISPATCHERS=3)
SHARED_SERVERS = 6

Connecting with Shared Server
sqldev
Database Instance
Oracle
Listener
Port 1521
Shared Server 02
Shared Server 03
Shared Server 04
Shared Server 05
Shared Server 06
Shared Server 01
Dispatcher 01
Dispatcher 02
Dispatcher 03
Port7001
Port7002
Port7003
CONNECT
GOTO 7001
sqldev
sqldev
sqldev
sqldev
sqlde
v
sqlde
v
sqlde
v
sqldev
sqlde
v
sqlde
v
sqlde
v
sqldev
SGA
Database connection
per client
Shared processes
Database session
per client
authenticate

Connecting with Shared Server
sqldev
Database Instance
Oracle
Listener
Port 1521
Shared Server 02
Shared Server 03
Shared Server 04
Shared Server 05
Shared Server 06
Shared Server 01
Dispatcher 01
Dispatcher 02
Dispatcher 03
Port7001
Port7002
Port7003
CONNECT
GOTO 7001
sqldev
sqldev
sqldev
sqldev
sqlde
v
sqlde
v
sqlde
v
sqldev
sqlde
v
sqlde
v
sqlde
v
sqldev
SGA
authenticate
Connecting with shared server is less
expensive for database:

A Database Call
sqldev
Database Instance
Shared Server
Shared Server
Shared Server
Shared Server
Shared Server
Shared Server
Dispatcher
Dispatcher
Dispatcher
SGA
Request queue
Response queue
Client on CPU
On network
Dispatcher on CPU
Shared server on CPU
time
Note: 3 process context switches
One of available shared
servers will execute
request within context of
applicable session
Session per client: state
can be maintained across
DB-calls

Universal Connection Pool (UCP) with Dedicated Servers

UCP with Dedicated Servers
JVM
C1 C2 C3
JDBC
Connection pool
Dedicated
Server
Dedicated
Server
Dedicated
Server
Showing three connection objects
C1, C2 and C3, each has own
connection + process + session
• UCP, provides
connection objects
• Application threads can
borrow these to execute
DB-calls

Requests CauseWorking Application Threads in JVM
Eight clients
currently waiting for
request to complete
These eight threads will time-
share connection objects
Shared processes
Shared sessions
Shared connections

Connection Reservation
Time during which thread has borrowed one of the
connection objects from pool to do database work

JVM
C1 C2 C3
JDBC
Thread 6 services request from browser client
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
TimelineT6
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3
now

JVM
C1 C2 C3
JDBC
Executes business logic and
needs to call database
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
TimelineT6
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

JVM
C1
C2
C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
Thread 6 borrows free
connection from pool
TimelineT6
Acquire
connection
Java thread on CPU
On network
Foreground on CPU
C2 now unavailable for
other threads
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

JVM
C1
C2
C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
Thread 6 submits SQL via C2
TimelineT6
Acquire
connection
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3
Server S2 wakes up from
idle state and starts work
on SQL

JVM
C1
C2
C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
TimelineT6
Acquire
connection
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3
S2 executes SQL
statement

JVM
C1
C2
C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
S2 sends result
back to C2
S2 idle again
TimelineT6
Acquire
connection
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

JVM
C1
C2
C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
T6 wakes up, gets
results from C2
TimelineT6
Acquire
connection
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

JVM
C1 C2 C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
T6 releases C2
back to pool
C2 available again for other
application threads
TimelineT6
Acquire
connection
Release
connection
Java thread on CPU
On network
Foreground on CPU
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

JVM
C1 C2 C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
TimelineT6
Acquire
connection
Release
connection
Java thread on CPU
On network
Foreground on CPU
Sends results back
to browser
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

JVM
C1 C2 C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
TimelineT6
Acquire
connection
Release
connection
Java thread on CPU
On network
Foreground on CPU
T6 ready for other
browser request
Dedicated
Server
Dedicated
Server
Dedicated
Server
S1 S2 S3

TimelineT6
JVM
C1 C2 C3
JDBC
T1 T2 T3 T4 T5 T6 T7 T8
Connection pool
Dedicated
Server
Dedicated
Server
Dedicated
Server
S2 is approximately 20% of
reservation time busy
80% of time it is never used
S1 S2 S3
In real world, we
see much smaller
percentages, think
< 5%
Acquire
Release
40%
40%
20%

Was Only Showing One Database Call
In real world,
application will do
multiple calls per
reservation
During connection reservation:
state can be maintained across
DB-calls, as the session is
exclusively in use by one of
clients
Across connection
reservations: application
cannot assume presence of
any state in session
Next request from same
browser client, could be
serviced with a different
session

JDBC
Connection pool
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
At some point, many
threads are working
on application
requests
The Big Picture

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
Majority of those
will have borrowed
a connection
object
The Big Picture
These are executing
Java code outside
connection reservation

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
Some connections
may still be
available in UCP
pool
The Big Picture

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
All connection objects
will have connection,
process and session
The Big Picture

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
The Big Picture
Only a small percentage
will actually be active in
database call
Recall: 20%, or much
less

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
During connection
reservation, majority will
be on-wire or in JVM
These are
currently inactive
The Big Picture

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
All of these will become
active multiple times during
 process context switching
The Big Picture
active
active
active

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
These are truly
inactive:
consume memory
The Big Picture
Showing one
application server

Multiple Application Servers
Active in DB-call now
Active in connection reservation
but not in DB-call at this moment
Never active

• Number of active sessions in database <<
number of borrowed connection objects from UCP pool(s)
• 5X smaller in this case
• Network time is a big player here!
• If you are able to reduce network latency app-server  db-server
then you will immediately see reduction in number of borrowed connection objects, and thus in
required pool-sizes
Important Point
This will be important when
we investigate combining UCP
with DRCP or PRCP

• If all connections are borrowed, i.e. when these are exhausted
• And new request arrives, then pool will attempt to grow
• Note: callers’ thread is used to create new connection
• If many callers arrive at same time: connections will be created simultaneously
• “Connection storm”
How and When Does UCP Grow?
If due to some database
issue all current
reservations take longer
than usual, then
connection storm can be
very aggressive.

• Shrinking is based on setting these parameters:
• Inactive Connection Timeout
Timeout since last reservation of connection object
• Maximum Connection Reuse Time
Timeout since creation of connection object: designed to deal with firewall timeouts
• Maximum Connection Reuse Count
Maximum count of reservations per connection object: addresses memory leaks
• By default UCP does not shrink the pool (all three default to 0 = OFF)
How and When Does UCP Shrink?

• Common issue we see in real-world is connection leaking
• Java thread starts connection reservation (via: pds.getConnection())
• Does a few database calls
• But, never ends connection reservation (never executes: pds.close())
• I.e. connection remains idle, yet still borrowed to Java thread
• Connection leaking is bug in application and will grow your connection pool as far
as it can grow
• DBA’s typically have “session-kill” scripts to find and kill sessions with high
v$session.LAST_CALL_ET values
Additional Material: Reclaiming Leaked Connections

• UCP is able to automatically reclaim leaked connections
• "Abandon ConnectionTimeout“
Additional Material: Reclaiming Leaked Connections
https://docs.oracle.com/en/database/oracle/oracle-database/19/jjucp/stale-ucp-connections.html

• Minimum size:
• How many threads in JVM are working on application requests, on average?
• Maximum size: see last year’s presentation:
• https://www.aioug.org/events/connection-pool-sizing-concepts
• You want to “protect” database from CPU oversubscription
• If connection reservation keeps dedicated server busy 20% of time
• Then five connection objects can keep one core @database-server busy
Assuming it’s all DB CPU
• Maximum size of connection pool: five * #cores
SizingYour Connection Pool
In real world, we
see much lower
percentages, think
< 5%
https://docs.oracle.com/en/database/oracle/oracle-database/21/jjucp/controlling-pool-size.html
You really should
have an idea of
this number for
your application

JDBC
Connection pool
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C
C C C C C C C C C C
C
C
C
C
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
T
JVM
T
T
T
T
T
T
T
T
T
T
T
T
UCP with Shared Servers
Dispatcher Dispatcher Dispatcher
Shared
server
Shared
server
Shared
server
Shared
server
Shared
server
Shared
server
Shared
server
Shared
server
No difference here
A lot less processes here
but recall: more context
switching…

• Shared server pro’s
• Less processes  you save on memory
• Connecting is less expensive  connection storms are less bad
• Shared servers con’s
• More process context switches per DB-call  potential increase in SYS cpu
• Slightly increased latency in DB per call
• Risk of “artificial deadlocks” (google ‘oracle shared server artificial deadlocks’)
• If on Exadata: you won’t have Smartscans
UCP + Shared Servers vs. Dedicated Servers: Pro’s, Con’s

• Pool of pre-spawned foreground processes in database
• Designed for applications that frequently connect/disconnect (PHP)
• Why? Because connect/disconnect is cheap with DRCP
What Is Database Resident Connection Pool?

• DRCP has two components:
• One or more connection brokers
• Set of pooled servers
Database Resident Connection Pool (DRCP)
EXEC DBMS_CONNECTION_POOL.start_pool;
EXEC DBMS_CONNECTION_POOL.stop_pool;
EXEC DBMS_CONNECTION_POOL.alter_param(…);
- num_cbrok
- minsize
- maxsize
DBA_CPOOL_INFO
GV$CPOOL_STATS
GV$CPOOL_CC_STATS (connection class)
Database Instance
Oracle
Listener
Pooled Server
Pooled Server
Pooled Server
Pooled Server
Broker
Upon pool start,
none of servers has
session yet
N0xx
L0xx

Connecting with DRCP
TNSNAMES.ORA:
your_alias =
(DESCRIPTION =
(CONNECT_DATA = (SERVER = POOLED)

Database Instance
Oracle
Listener
Broker
Pooled Server
Pooled Server
Pooled Server
Pooled Server
Authenticate (SCOTT)
sqldev
Note: showing sqldev as client just to explain architecture
(this definitely isn’t DRCP’s use-case)

Database Instance
Oracle
Listener
Broker
Pooled Server
Pooled Server
Pooled Server
Pooled Server
Authenticate (SCOTT)
sqldev
SCOTT
session

Disconnecting from DRCP
Database Instance
Oracle
Listener
Broker
Pooled Server
Pooled Server
Pooled Server
Pooled Server
Disconnect
sqldev

Disconnecting from DRCP
Database Instance
Oracle
Listener
Broker
Pooled Server
Pooled Server
Pooled Server
sqldev
Pooled Server
Connection is
gone
Process and
session still
exist
Connection
class “SCOTT”
If openTX, will
be rolled back

Reconnecting to DRCP
Database Instance
Oracle
Listener
Broker
Pooled Server
Pooled Server
Pooled Server
sqldev
Pooled Server
If you now reconnect,
broker will (try to)
handout pooled server of
appropriate connection
class
Connecting with pooled server is
least expensive for database:
Reconnect (SCOTT)

• UCP has special behavior with DRCP
• Upon UCP startup, connection objects will connect, and receive pooled server
• Available connection objects (those that aren’t borrowed) will detach from pooled server, but
will remain connected to DRCP
• Upon reservation start, they will attach to pooled server
Combining UCP with DRCP

UCP Startup with DRCP
JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
UCP has just started
Each connection object
will issue connect to
database

JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
After contacting the
listener, they all go to
broker and request a
pooled server

JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
Broker dispatches
pooled server to
each of them

JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
Connection objects
authenticate  create
session

UCP Startup with DRCP: Detach
JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
They then issue
conn.detachServerConnection
1) Connection goes back to broker
2) Pooled servers back in pool

What Happens on Connection Reservation?
JVM
C1
C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
Application thread
starts connection
reservation
T

JVM
C1
C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
UCP will issue attach
T
Broker will dispatch
pooled server of
appropriate
connection class
Thread can now
do DB calls

JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
T
Thread done,
connection back
in UCP pool
UCP will issue detach

JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
T
UCP will issue detach
C1’s connection goes
back to broker

UCP + DRCP In Summary
JVM
C1 C2 C3
JDBC
Connection pool
Pooled
Server
Pooled
Server
Pooled
Server
Broker
T
While connection objects
are not borrowed, they do
not claim pooled server
Only during connection
reservation will they
claim pooled server
UCP’s magic sauce
3rd Party pools don’t do this

Remember Multiple Application Servers?
Active at some point
Never active
With DRCP, we save
server processes for
these
With DRCP, we save
server processes for
these
With DRCP, we could save
server processes for these
(3*20)
100 100 100
20 20 20
Broker

• DRCP also, grows and shrinks depending on load
• Similar configuration capabilities:
Why Could…
Just like configuring UCP,
DRCP also requires careful
configuration to prevent the
pool being too big

• DRCP was designed to help applications that have high connect-rate
• Well-configured UCP pool shouldn’t have high connect-rate
• Prevent connection storms by setting:
- pds.setMaxPoolSize([your max size]);
- pds.setConnectionWaitTimeout(Integer.MAX_VALUE);
(https://www.aioug.org/events/connection-pool-sizing-concepts)
• Don’t combine UCP with DRCP to save on processes and sessions
• Instead: tune shrinking of UCP pool and setMaxPoolSize to save connections in database
UCP with DRCP: Our Recommendation
DRCP can be bandaid if you do not control UCP configurations
you still have to configure DRCP carefully

High level architecture:
• PRCP is feature of Connection Manager (CMAN)
• CMAN sits between application server
and database server
• Typically installed in separate ORACLE_HOME/server
• You connect to database “through” CMAN
• CMAN forwards incoming connections
• CMAN manages pool of connections (= PRCP)
Proxy Resident Connection Pool (PRCP)
Application server
PRCP
CMAN

Connecting with PRCP
TNSNAMES.ORA:
your_alias =
(DESCRIPTION =
(CONNECT_DATA = (SERVER = POOLED)
Same as with DRCP
CMAN host CMAN listener port
And you need to configure CMAN so it
knows *where* to forward to…
cman.ora
oraaccess.xml

CMAN
CMAN Instance: Components
One or more
Gateway processes
Each GW process
“has” connection pool
(PRCP)
Gateway
process
Gateway
process
Upon startup, all
connections have
authenticated with
CMAN’s TDM user
(CC = TDM)
CC = Connection Class
PRCP PRCP
C1 C2 C3 C1 C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
TDM
TDM
TDM
TDM
TDM
TDM
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
Separate set of
authentication
servers
TDM
TDM
Compare with: dispatcher
(shared) or broker (DRCP)
GW process accepts
incoming connect requests

• UserTDM must be able to “proxy connect” into other users
• Each user that wants to connect to database through CMAN needs to grant
“connect throughTDM”
alter user SCOTT grant connect through TDM;
• ConnectTDM[SCOTT]/<TDM-PW>
• This creates session for SCOTT, without having to know SCOTT’s password
CMAN Uses “Proxy Connect”
https://oracle-base.com/articles/misc/proxy-users-and-connect-through

Connecting with PRCP
CMAN
Gateway
process
Gateway
process
PRCP PRCP
C1 C2 C3 C1 C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Connect SCOTT/tiger
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
One of auth
servers used to
verify un/pw
Upon success,
connection is marked as
authenticated
CC = SCOTT
connect SCOTT/tiger
(re)connect TDM/…
TDM
TDM
TDM TDM
TDM
TDM TDM
TDM
Client now thinks it’s
connected to DB

A Database Call
CMAN
Gateway
process
Gateway
process
PRCP PRCP
C1 C2 C3 C1 C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
SQL
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
TDM TDM
TDM TDM
TDM
TDM
TDM
TDM

A Database Call
CMAN
Gateway
process
Gateway
process
PRCP PRCP
C1 C2 C3
C1
C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
SQL
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
CMAN silently performs proxy-
connect into SCOTT
Then borrows C1 out to GW thread
TDM TDM
SCOTT
TDM
TDM
TDM TDM
TDM
Client can do multiple calls
C1 remains bound to this connection
until client disconnects

Disconnecting from PRCP
CMAN
Gateway
process
Gateway
process
PRCP PRCP
C1 C2 C3 C1 C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
TDM TDM
SCOTT
Connection goes back into pool
CC remains SCOTT
Any openTX will have been rolled
back
Connection to
client is gone
Next time client connects
with SCOTT:
 Verify password by
auth server
 Borrow out C1
TDM
TDM
TDM
TDM
TDM

• If no pooled server of correctCC exists: one connect = three connects
• Authentication server:
1. Connect scott/tiger
2. Reconnect tdm/…
• Borrowed pooled server:
3. Proxy connect tdm[scott]/…
• If pooled server of correctCC exists: one connect = two connects
• Authentication server:
1. Connect scott/tiger
2. Reconnect tdm/…
PRCP Increases Connect-RateTowards Database

CMAN also Supports Detach/Attach
CMAN
Gateway
process
Gateway
process
PRCP PRCP
C1 C2 C3 C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
TDM TDM
Upon Detach:
 C1 back in pool (remains SCOTT)
 Connection to client stays alive
C1
SCOTT
C1
SCOTT

CMAN also Supports Detach/Attach
CMAN
Gateway
process
Gateway
process
PRCP PRCP
C1 C2 C3 C2 C3
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Dedicated
Server
Auth servers
A1 A2
Dedicated
Server
Dedicated
Server
TDM TDM
Upon Attach:
Broker will handout available
connection of appropriate CC
Just like with DRCP, UCP will
detect PRCP is used and
automatically perform
Attach/Detach upon
start/end
C1
SCOTT

UCP with PRCP: Same Dynamics as with DRCP…
100 100 100
20 20 20
Active at some point
Never active
300 Connections in PRCP required
60 Sessions into database could
potentially be saved
Requires careful configuration of
gateway processes and PRCP pools

• Con’s
• Every DB call gets latency hit in CMAN layer (extra network hop)
• This will further increase that “80%”!
- Which in turn could require your connection pools to be larger
• Pro’s
• Compared to DRCP, the broker’s work is outside database  Gateway process
• Our recommendation:
• Don’t use PRCP with UCP, unless you have *a lot* of application servers each with UCP pool
- May bring savings in number of database sessions
- Very much depends on *your* workload
• First, properly configure your UCP pools before looking into PRCP
Combining UCP with PRCP: Pro’s and Con’s

• We see customers spending little time on configuringUCP pools
• Proper configuration can reduce #DB-sessions and prevent connection storms
• DRCP has very specific use-case:
• Applications with high connect-rate
• PRCP is complex beast
• We potentially see one use-case: many application servers with large UCP pools each
• Contact us
• Combining UCP with DRCP/PRCP shifts responsibility of configuring pools
Wrapping up

• Developers should aim for high %busy-rate of foreground processes during
connection reservations
• The higher %busy-rate, the less big UCP pool needs to be
• If possible: bring application-servers closer to database server
• Reducing latency between these two will decrease that 80%  pools can be smaller
• If you move to cloud  move both DB- and app-server!
Wrapping up

• “With DRCP (or PRCP) we could save those 60 processes/sessions in DB”
• There’s another point to make here…
- Lets assume UCP maxsize set at 100
- And arrival rate of requests is 50/second (Poisson distribution, λ = 50)
- And average service time is 2 seconds (Exponential distribution, 𝜇 = 2)
◦ Assume all of 2 seconds are inside connection reservation
- This results in, on average, 100 requests in-progress at any point in time, and 100 connections from
UCP being borrowed (used)  this is referred to as “ingress load” E
◦ E = λ*𝜇 is normalized ingress load (also called: offered traffic stated in Erlang unit)
- Each UCP pool can be viewed as “0.02/2/100/100” queue (Kendall’s notation)
◦ [inter-arrival-time]/[service-time]/[#of service stations]/[capacity of system (queue-size + service stations)]
◦ If #of service stations == capacity of system  no queuing (i.e. assume ConnectionWaitTimeout=0)
Additional Material: QueuingTheory

• Each UCP pool can be viewed as “0.02/2/100/100” queue
• For such queue we can compute chance of new arrival (= application request) not
finding available service station (= connection)
• This is done with ErlangB function
• Inputs: ingress load in Erlang + number of service stations
• Output: above chance
• Example: ErlangB(100,100) = 0.076
• So, at each application server, there’s 7.6% chance that new request cannot immediately find
available connection in pool
• In which case, request hits Java exception when ConnectionWaitTimeout is set at zero
ErlangB Function
https://en.wikipedia.org/wiki/Erlang_%28unit%29#Erlang_B_formula
Additional material

• If we remove maxsize at UCP for each application server
• And use DRCP with maxsize at 1000
• Then this chance becomes ErlangB(1000,1000) = 2.5%!
• Same goes with PRCP and single(!) gateway process
• Questionable whether you want 1 GW process in this scenario though
• With two gateway processes and pool maxsize capped at 500 per GW process, this
chance becomes ErlangB(500,500) = 3.5%
• In next slide we’ll graph ErlangB(n,n) function
ErlangB Function
Additional material

P = ErlangB(n,n) vs. P = ErlangB(10*n,10*n)
7.6% chance of queuing at 100/100
2.5% chance of queuing at 1000/1000
Put differently: 7.6% chance
of additional connects due to
CP growth, vs 2.5%
• Assume: 10 Applications
servers
• Each UCP sized @n
• Using dedicated servers
• Assume ingress load of n per
application server
• By using DRCP under UCP
you replace these by:
• Single DRCP sized @10*n
• With ingress load 10*n
Additional material

• Let’s say you’d want same 2.5% chance with UCP without DRCP
• ElangB(100,100 = 0.076
ErlangB(100,111) = 0.024 
ErlangB(100,110) = 0.027
• So you’d need UCP pools to be around 11% bigger
How Much Bigger Should UCP be to Have Same Chance?
Additional material

PL/SQL Source code ErlangB
create or replace function erlangb
(p_ingress_load in number -- Ingress in Erlang units
,p_pool_size in integer) return varchar2 as
--
-- Source: https://en.wikipedia.org/wiki/Erlang_%28unit%29#Erlang_B_formula
-- Returns probability of blocking of incoming request.
--
invB number := 1.0;
l_return number;
--
begin
--
for j in 1..p_pool_size
loop
--
invB := 1.0 + (InvB * (j / p_ingress_load ));
--
end loop;
--
l_return := 1.0 / InvB;
return round(l_return,3);
--
end;
Unit Erlang (offered traffic):
Arrival-rate (requests/sec) *
Service-time (sec)
Example:
42 Erlang ==On average there are
42 requests “being serviced” at
any point in time
Arrival rate  Poisson distribution
Service time  Exponential distribution
Additional material

New lessons in connection management

New lessons in connection management

In this document