BRM Technical Overview

Elastic Charging Engine

CAGBU Enablement Team

Safe harbor statement

The following is intended to outline our general product direction. It is intended for information
purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any
material, code, or functionality, and should not be relied upon in making purchasing decisions. The
development, release, timing, and pricing of any features or functionality described for Oracle’s
products may change and remains at the sole discretion of Oracle Corporation.

2 Copyright © 2020, Oracle and/or its affiliates

Elastic Charging Engine

1 Introduction
2 Coherence
3 Components
4 Subscriber caching

3 Copyright © 2020, Oracle and/or its affiliates

Technical Architecture

WSM SOAP API BC REST API TMF REST API JCA PCM API

Pricing
BRM Server
Design
Center Connection Manager

DM DM DM DM DM DM DM
Oracle AQ Vertex Fusa Ldap Invoice IFW BRM
Billing Database
Care

Elastic Charging Engine

Business
Operations
Center
Elastic charging servers

Persistence
Database

HTTP/2 Diameter RADIUS

Gateway Gateway Gateway

Convergent Charging Controller Offline Mediation Controller

4 Copyright © 2020, Oracle and/or its affiliates

 Detailed Architecture

Pricing Design Center BRM Server BRM

Database

PDC Transformation Connection DM

JMS
Application Engines Manager Oracle

Rated Event
Kafka Loader

Pricing EM BRM Customer

Updater Gateway Gateway Updater

Offline Rated Event

Mediation Elastic charging servers Formatter
Controller

Persistence
Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
Elastic Charging Engine

5 Copyright © 2020, Oracle and/or its affiliates

Elastic Charging Engine

1 Introduction
2 Coherence
3 Components
4 Subscriber caching

6 Copyright © 2020, Oracle and/or its affiliates

Coherence

Elastic charging engine is built on top of Oracle Coherence:

- Coherence is embedded in ECE

- No need to install Coherence separately

Coherence is an in-memory data grid and provides:

- In-memory data storage

- Co-location of data and processing
- No single point of failure
- Single view of data
- Predictable scaling

7 Copyright © 2020, Oracle and/or its affiliates

Coherence Terminology

Terminology Meaning

Server Physical machine

Node One Java Virtual Machine (JVM) process
running Coherence
Member Same as node.
Cluster Group of node using the same cluster name

Cache Data store of a particular type containing key-

value pair

Cache partition Slice of a cache stored on a node

8 Copyright © 2020, Oracle and/or its affiliates

Cluster node

Cluster Node or Member:

• Java process (JVM) loaded with the coherence library

• Manages data: application objects
• Manages processing: code co-located with its data
• Can be storage enabled or non-storage enabled

9 Copyright © 2020, Oracle and/or its affiliates

Distributed Cache

• Partitioned: The data in a distributed cache A A B

A
is spread out over all the servers in such a B Primary Primary B
way that no two servers are responsible for C C
C D
the same piece of cached data. D D
Backup Backup
Logical Logical
Partitioned Cache Partitioned Cache
• Load-Balanced: Since the data is spread out
evenly over the servers, the responsibility for Node 1 Node 2
managing the data is automatically load-
balanced across the cluster.
A C A
D
• Synchronously maintains backup partitions B
Primary Primary B
on different physical machines for failover C C
D B
and recovery A D
Logical Backup Backup
Partitioned Logical
• Two copies of the data are held in cluster Partitioned Cache Cache
Node 3 Node 4

10
 Distributed Cache

A B
• Operations against data in the cache can be A A
accomplished with a "single hop”, in other B Primary Primary B
words, involving at most one other server. C C
D C D
D
Backup Backup
Logical Logical
• Location Transparency: Although the data is Partitioned Cache Partitioned Cache
spread out across cluster nodes, the exact
Node 1 Node 2
same API is used to access the data. This is
called location transparency.

• Failover: All Coherence services provide A C A

D
B
failover and failback without any data loss, Primary Primary B
C
and that includes the distributed cache C
D B A
service. D
Logical Backup Backup
Partitioned Logical
Partitioned Cache Cache
Node 3 Node 4

11
Distributed Cache – get()

A
get() A A
A B
B Primary Primary B
C get() C C
D C D D
Logical Backup Backup Logical
Business Layer Partitioned Cache Partitioned Cache Business Layer
Node 1 Node 2

A C D A
get() B B
B Primary Primary
C C
D get() D B A D
Logical Backup Backup Logical
Business Layer Partitioned Cache Partitioned Cache Business Layer
Node 3 Node 4

12
Distributed Cache – put()

A
put() A A
A B
B Primary Primary B
C C put() C
D C D D
Logical Backup Backup Logical
Business Layer Partitioned Cache Partitioned Cache Business Layer
Node 1 Node 2

A C D A
B Primary Primary B
C C
D B A D
Logical Backup Backup Logical
Business Layer Partitioned Cache Partitioned Cache Business Layer
Node 3 Node 4

13
 Node, Machine, Site and Rack Safety

Coherence Data Center Rack 1 Rack 2

Machine 1 Machine 2 Machine 5 Machine 6
Safety P1 P2 P5 P6
P8’ P7’ P4’ P3’

§ Node safe Machine 3 Machine 4 Machine 7 Machine 8

P3 P4 P7 P8
§ Machine safe P6’ P5’ P1’ P2’

§ Rack safe
§ Site safe Site 1 Site 2
P1 MAN P2
P2’ P1’

14
Elastic Charging Engine

1 Introduction
2 Coherence
3 Components
4 Subscriber caching

15 Copyright © 2020, Oracle and/or its affiliates

Elastic Charging Server

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

• Caches customer, pricing and Kafka

Rated Event
Loader
configuration data
• Processes charging, policy and top-
up requests Pricing
Updater
EM
Gateway
BRM
Gateway
Customer
Updater
• Exposes JMX for ECE configuration
and monitoring Rated
Event
Elastic charging servers Formatter

Persistence
Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
ECE

16 Copyright © 2020, Oracle and/or its affiliates

Kafka

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

• Handles notifications generated by Rated Event

Loader
Kafka
the elastic charging servers
• Producer: ECS
Pricing EM BRM
• Consumers: BRM Gateway, DGW, Updater Gateway Gateway
Customer
Updater
HTTP2 GW and third-party systems
Rated
Event
Elastic charging servers
• Required for HTTP2 GW Formatter

• WL JMS queue is also supported Persistence

Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
ECE

17 Copyright © 2020, Oracle and/or its affiliates

BRM gateway

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

Rated Event
Kafka Loader

• Handles
- Trigger billing
- Subscriber lifecycle state change Pricing
Updater
EM
Gateway
BRM
Gateway
Customer
Updater
- First usage
- POID replenishments Rated
Event
Elastic charging servers Formatter

• Reads from Kafka

Persistence
Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
• Calls BRM Opcodes ECE

18 Copyright © 2020, Oracle and/or its affiliates

EM gateway

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

• Synchronizes subscriber data updates, Rated Event

Loader
Kafka
balance queries and rerating requests

Pricing EM BRM
• Called from CM as opcode Updater Gateway Gateway
Customer
Updater

Rated
• Writes to ECE Coherence cache Elastic charging servers
Event
Formatter

Synchronous flow
Persistence
• Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
ECE

19 Copyright © 2020, Oracle and/or its affiliates

Pricing Updater

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

• Synchronizes config,
metadata and pricing
data from PDC to ECE
Rated Event
Kafka Loader

• Reads from the PDC JMS queues (to

which data needs to be published) Pricing EM BRM Customer
Updater Gateway Gateway Updater

• Writes to ECE Coherence cache Rated

Event
Elastic charging servers Formatter

• Asynchronous flow Persistence

Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
ECE

20 Copyright © 2020, Oracle and/or its affiliates

Customer Updater

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

Rated Event
Kafka Loader

• At start-up, loads subscriber data

from BRM DB
Pricing EM BRM
- Direct DB access Updater Gateway Gateway
Customer
Updater

• At run-time, handles updates for Rated

Event
Elastic charging servers
rerating requests, account migration, Formatter

product and discount data

Persistence
- Via DB AQ Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
ECE

21 Copyright © 2020, Oracle and/or its affiliates

Persistence Options

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

Rated Event
• NoSQL database Kafka Loader

- Only stores rated events

Pricing EM BRM Customer
Updater Gateway Gateway Updater
• Oracle database
- Stores all cached objects Rated
Event
- Requires support for partitioning Elastic charging servers Formatter

Persistence
Radius Diameter HTTP/2 Database
Gateway Gateway Gateway
ECE

22 Copyright © 2020, Oracle and/or its affiliates

Rated Event Formatter

Pricing Design Center BRM Server BRM

Database

PDC Transformat Connection DM

JMS
Application ion Engines Manager Oracle

Rated Event
Kafka Loader

• Extracts rated events from the

Oracle DB Pricing EM BRM Customer

• Purges rated events after

Updater Gateway Gateway Updater

extraction based on configured Rated

interval Elastic charging servers

Event
Formatter

• One instance per schema

Persistence

• Supports plugins for sending Radius

Gateway
Diameter
Gateway
HTTP/2
Gateway
Database

data to 3rd party systems ECE

23 Copyright © 2020, Oracle and/or its affiliates

Elastic Charging Engine

1 Introduction
2 Coherence
3 Components
4 Subscriber caching

24 Copyright © 2020, Oracle and/or its affiliates

Subscriber caching

Selective loading

- Only load selection of subscribers based on fixed filter conditions

- No events can be rated for excluded subscribers

Partial loading

- Only load selection of subscribers based on high water mark

- On demand loading of subscribers
- Only supported with the persistence database

25
Selective Loading of Subscribers

Selective loading

- Only load selection of subscribers

- No events can be rated for excluded subscribers

Filter conditions

- Add filter conditions to preselect_customers.groovy file

- Excluded subscribers cannot be used for rating e.g. closed accounts
- Groovy scripts populates BRM tables ECE_WORK_ITEM_SUMMARY_T and ECE_WORK_ITEM_DETAIL_T
- Change in filter condition requires ECE restart

26
Selective Loading of Subscribers

Steps to load or re-load subscribers:

- System in USAGE_PROCESSING
- Run preselect_customers.groovy script to create new work item entries with the newly created account POIDs in
BRM schema (can use the filtering capability in the groovy script)
- Invoke the customer loader application from ECC console to trigger the incremental load, this would be done per
BRM schema

ecc:000> start customerLoader -incremental customerUpdater1

27
Partial Loading of Subscribers

Supported with Persistence Database option only.

Number of Cached Objects/cache

Configure High and Low watermarks
- High indicates trigger for data offloading
High Watermark
- Low indicates trigger for eager fetching
- Configuration through JMX
Low Watermark
Data offloaded
- Least used or unused data in Cache

Data fetched ECE Cache

- An account not residing in cache but a request for charging for
the account has arrived
 Number of Cached Customer Objects/node
Cache Management

Example
• Total 10000 customers High Watermark1500
• 60% in cache = 6000 customers
• 3 machines M1, M2 and M3 each with 2 ECE nodes
• Each ECE node = 6000/(3*2) = 1000 customers
Low Watermark 1000
• If one machine goes down – 2000 customers will be divided between
the existing two machines, each node will get another 2000/(2*2) =
500 customers. So each node will host 1500 customers.
• In order to accommodate one machine failure, you can set
- High water mark at 1500 customers
ECE Cache
- Low watermark at 1000 customers

X
Initial loading up to High Watermark 60% of total 10000 which is 6000 is cached
Stretched distribution 3000
Subsequent reloads up to Low Watermark Normal Distribution 2000
3000
2000 2000
High Watermark 1500 1500 1500 1500
Low Watermark 1000 1000 1000 1000 1000 1000

M1 M2 M3
Initial Data Loading

• One time loading for the first time

• Data is loaded into the cache and then into Oracle DB in raw object format
• High and Low watermark rules apply for cache store
• Only recently accessed data in cache up to High Watermark, but DB to store all data

ConfigLoader Loa
d from
File ECE Oracle
DB
Load from PDC
PricingUpdater
Cache
Store
Table
BRM
rom
Lo ad f Charging Grid
CustomerUpdater
Data Reloading

• Reason – partition/node loss

• Automatic reload
• Data reloaded into the cache up to Low Watermark
• Quicker reloading (warm restart) as only a subset is loaded in the same raw object format
that coherence maintains
Pull Keys from Persistent
Store into ECE

ConfigLoader Trig
g er R
eloa ECE Oracle
d
DB
Pull Data
PricingUpdater Trigger Reload into Cache
Cache
Store
Table
Reload
r
CustomerUpdater Tr igge Charging Grid
Lazy Loading – Load only when needed

Scenario
• When a charging request arrives for a customer and
relevant data is not in cache
ECE Oracle
DB
Lazy loading or on-demand loading Pull Data
into Cache
• Data for the subscriber is loaded into cache from ECE Cache
Store
Oracle DB Table

• If the number of objects was already at high watermark Charging Grid

limit, then one least used customer data is off-loaded
• Charging operation continues