Oracle Flex ASM and Flex Cluster

PRESENTER
Y V RaviKumar
Oracle ACE & Oracle Certified Master (OCM)

Patterns 1
 ORACLE CERTIFIED MASTER (OCM)
http://education.oracle.com/education/otn/10g_OCM_17_Jun_11/ykumar.htm

ORACLE ACE
https://apex.oracle.com/pls/apex/f?p=19297:4:::NO:4:P4_ID:13023

ABOUT ME
AUTHOR OF ORACLE TECHNOLOGY NETWORK (OTN) ARTICLES
http://www.oracle.com/tecchnetwork/es/articles/index.html - (17 Articles for OTN Spanish)
http://www.oracle.com/tecchnetwork/pt/articles/index.html - (14 Articles for OTN Portuguese)
http://www.oracle.com/tecchnetwork/articles/index.html - - ( 2 Articles for OTN English)

ORACLE TECHNOLOGY SPEAKER @USER GROUPS

Speaker @Sangam Y2014
Speaker @Independent Oracle User Group (IOUG) Y2014
Speaker @Oracle Technology Network (OTN) Yathra Y2013, Y2014 & Y2015
Speaker @All India Oracle User Group (AIOUG) Tech Day Y2013 & Y2014 ORACLE CERTIFICATIONS
Speaker @All India Oracle User Group (AIOUG) ExadataSIG Y2014 Oracle Database 10g: Certified Master (10g OCM)
Oracle Database 10g & 11g: Administering RAC Certified Expert
Oracle Database 11g: Performance Tuning Certified Expert
AUTHOR OF TOAD WORLD (CONNECTED INTELLIGENCE)
Oracle Exadata 11g Essentials
https://www.toadworld.com
Oracle Golden Gate 10 Essentials
Oracle Database 11g: SQL Tuning Certified Expert
Author Of OTech Magazine Oracle 9i & 10g: Managing Oracle on Linux Certified Expert
http://otechmag.com/magazine/2015/summer/ravikumar-yv.html Oracle Certified Professional (OCP) – Oracle 11g, 10g, 9i and 8i
SUN Certified – Solaris System Administrator in SUN Solaris 9
Author Of All things ORACLE from redgate
http://allthingsoracle.com/upgrading-a-database-using-recovery-manager-rman-duplicate-command-in-oracle-12c/

CO-FOUNDER OF
http://www.oraworld-team.com/
Patterns 2
ORACLE
Flex ASM and Flex Cluster

Patterns 3
ARCHITECTURE OF ASM

ORACLE DATABASE ORACLE DATABASE

ASM
File System
File System & Volume Management
Logical Volume Manager
OPERATING SYSTEM OPERATING SYSTEM

HARDWARE HARDWARE

Patterns 4
Oracle 10g RAC – VD & OCR Locations

Patterns 5
Oracle 11g RAC – VD & OCR Locations

Patterns 6
Oracle 12c RAC – VD & OCR Locations

Patterns 7
CAPABILITIES OF ASM
Database Database Database Database Database
Instance Instance Instance Instance Instance

ASM ASM ASM ASM ASM

Instance Instance Instance Instance Instance

Oracle
RAC
Servers
Database Database Database
1-1 ASM to SERVER

ASM Cluster Pool of Storage

SHARED DISK GROUPS
Disk Group A Disk Group B

File 3 File 4

File 1 File 2
WIDE FILE STRIPING
Disk1 Disk 2 Disk 3 Disk 4 Disk 5 Disk 6 Disk 7

Patterns 8
AUTOMATIC STORAGE MANAGEMENT (ASM) OVERVIEW
Current State 

RAC CLUSTER
Database Instance

ONE TO ONE DBA

ASM Instance
DBA DBB DBB DBB DBC
MAPPING OF ASM ASM Instance

INSTANCES TO
SERVERS Node1 ASM Node2 ASM Node3 ASM Node4 ASM Node5 ASM

ASM Cluster Pool of Storage

SHARED DISK GROUPS Disk Group A Disk Group B ASM Disk

WIDE FILE STRIPING

Patterns 9
Oracle Database 11gR2 with ASM

LOG INTO RAC DATABASE INSTANCE

login as: oracle (RAC1)
[email protected]’s password: ********
Last login: Fri Sep 27 06:05:44 2013
CHECK STATUS OF ASM & RAC DATABASE
[oracle@rac1 ~]$ ps -ef | grep pmon INSTANCES
oracle 3053 1 0 05:56 ? 00:00:00 asm_pmon_+ASM1
oracle 3849 1 0 05:57 ? 00:00:00 ora_pmon_flavia1
CHECK STATUS OF ASM INSTANCE IN RAC
[oracle@rac1 ~]$ srvctl status asm DATABASE INSTANCES1 (RAC1)
ASM is running on rac2,rac1
CHECK STATUS OF CLUSTER IN RAC
[oracle@rac1 ~]$ crsctl check cluster
DATABASE INSTANCES1 (RAC1)
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online STOP THE ASM INSTANCE IN
RAC DATABASE INSTANCE1
[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f (RAC1)

Patterns 10
Oracle Database 11gR2 with ASM

CHECK THE STATUS OF ASM INSTANCE IN

RAC DATABASE INSTANCE1 (RAC1)

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f

[oracle@rac1 ~]$ srvctl status asm

Patterns 11
Oracle Database 11gR2 with ASM

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f

[oracle@rac1 ~]$ srvctl status asm
ASM is running on rac2

Patterns 12
Oracle Database 11gR2 with ASM

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f THE STATUS OF ASM & RAC
CHECK
[oracle@rac1 ~]$ srvctl status asm DATABASE INSTANCE (RAC1)
ASM is running on rac2
[oracle@rac1 ~]$ ps -ef | grep pmon

Patterns 13
Oracle Database 11gR2 with ASM

SUMMARY
 The database instance is strongly linked to the ASM instance
 If an ASM instance fails so will the database instance on the
same node

[oracle@rac1 ~]$ srvctl stop asm -n rac1 -o abort -f

[oracle@rac1 ~]$ srvctl status asm
ASM is running on rac2
[oracle@rac1 ~]$ ps -ef | grep pmon
oracle 7885 5795 0 06:20 pts/0 00:00:00 grep pmon

Patterns 14
FLEX ASM: ELIMINATE 1:1 SERVER MAPPING

 New: ASM Storage Consolidation in Oracle Database 12c 

RAC Cluster
Database Instance

DATABASES SHARE DBA

ASM Instance
DBA DBB DBB DBB DBC
ASM INSTANCES ASM Instance

Node1 Node2 ASM Node3 ASM Node4 ASM Node5

Node1
Node1 Node2 Node5
runs
runs as
as runs as runs as
ASM
ASM Client
Client ASM Client ASM Client
to
to Node4
Node2 to Node3 to Node4
ASM Cluster Pool of Storage
SHARED DISK Disk Group A Disk Group B ASM Disk

GROUPS

WIDE FILE STRIPING

Patterns 15
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac1 Desktop]$ hostname

oel6-112-rac1.localdomain LOG INTO RAC DATABASE
INSTANCE 1 (RAC1)
[oracle@oel6-112-rac1 Desktop]$ ps -ef | grep pmon
oracle 3325 1 0 17:39 ? 00:00:00 asm_pmon_+ASM1
oracle 3813 1 0 17:40 ? 00:00:00 mdb_pmon_-MGMTDB CHECK THE STATUS OF ASM
oracle 5806 1 0 17:42 ? 00:00:00 ora_pmon_orcl1 & RAC DATABASE
oracle 6193 1 0 17:42 ? 00:00:00 apx_pmon_+APX1 INSTANCES

[oracle@oel6-112-rac1 Desktop]$ srvctl status asm

CHECK THE STATUS OF ASM
ASM is running on oel6-112-rac2,oel6-112-rac1
INSTANCE IN RAC DATABASE
INSTANCE FROM INSTANCE 1
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster (RAC 1)
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online CHECK THE STATUS OF CLUSTER

>>>
IN INSTANCE 1 (RAC 1)
 [oracle@oel6-112-rac1 Desktop]$ asmcmd
Oracle Database 12cR1 with FLEX ASM CHECK IF ORACLE FLEX ASM IS ENABLED
ASMCMD> showclustermode OR NOT (RAC1)
ASM cluster : Flex mode enabled
ASMCMD> showclusterstate
Normal
ASMCMD> exit

[oracle@oel6-112-rac2 Desktop]$ asmcmd

CHECK IF ORACLE FLEX ASM IS ENABLED
ASMCMD> showclustermode
ASM cluster : Flex mode enabled OR NOT (RAC2)
ASMCMD> showclusterstate
Normal
ASMCMD> exit
[oracle@oel6-112-rac1 Desktop]$ srvctl stop asm -node oel6-112-rac1 -stopoption
abort -force
BRING DOWN THE ASM INSTANCE IN
RAC DATABASE INSTANCE 1 (RAC 1)
[oracle@oel6-112-rac1 Desktop]$ srvctl status asm
PRCR-1070 : Failed to check if resource ora.asm is registered Cannot communicate
with crsd
CHECK STATUS OF ASM INSTANCE IN RAC

>>>
DATABASE INSTANCE1 (RAC1)
[oracle@oel6-112-rac1 Desktop]$ crsctl check cluster
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CHECK STATUS OF CLUSTER
CRS-4533: Event Manager is online
SERVICES IN RAC DATABASE
INSTANCE1 (RAC1)
[oracle@oel6-112-rac1 Desktop]$ ps -ef | grep pmon
oracle 3813 1 0 17:40 ? 00:00:00 mdb_pmon_-MGMTDB
oracle 5806 1 0 17:42 ? 00:00:00 ora_pmon_orcl1
oracle 6193 1 0 17:42 ? 00:00:00 apx_pmon_+APX1

[oracle@oel6-112-rac1 Desktop]$
CHECK STATUS OF ASM & RAC
DATABASE IN INSTANCE1
NOTE
(RAC 1)
Here a database instance is associated with the specific ASM instance running
in the specific node. If in case due to some reason if the ASM instance was
unable to be brought up/services goes down, still the database instance can be
brought up as the database instance will look for ASM instance running in the
same cluster.

>>>
Oracle Database 12cR1 with FLEX ASM

[oracle@oel6-112-rac2 Desktop]$. oraenv

ORACLE_SID = [orcl1] ? orcl1
ORACLE_HOME = [/home/oracle] ? /u01/app/oracle/product/12.1.0/db_1
The Oracle base remains unchanged with value /u01/app/oracle

CHECK STATUS OF RAC DATABASE

INSTANCE RUNNING WITHOUT ASM
INTANCE IN RAC DATABASE INSTANCE1
(RAC1)

Patterns 19
Oracle Database 12cR1 with FLEX ASM

LOG INTO DATABASE INSTANCE FROM

[oracle@oel6-112-rac1 Desktop]$ sqlplus /nolog
SQL*Plus: Release 12.1.0.1.0 Production on Wed Sep 25 18:24:36 2013 Copyright RAC 2013,
(c) 1982, DATABASE INSTANCE
Oracle. (RAC1)
All rights reserved.

SQL> connect sys/oracle@orcl as sysdba

Connected.
SQL> select instance_name, instance_number from gv$instance;
INSTANCE_NAME INSTANCE_NUMBER
-------------------------------------------
orcl2 2
orcl1 1
SQL> select instance_name, instance_number from v$instance;
INSTANCE_NAME INSTANCE_NUMBER
-------------------------------------------
orcl2 2
SQL> connect sys/oracle@orcl as sysdba
Connected.
SQL>

Patterns 20
Oracle Database 12cR1 with FLEX ASM

SQL> select instance_name, instance_number from gv$instance;

INSTANCE_NAME INSTANCE_NUMBER
-------------------------------------------
orcl1 1
CONNECTING TO ASM INSTANCE OF RAC
DATABASE INSTANCE2 (RAC2) FROM RAC
[oracle@oel6-112-rac1 Desktop]$. oraenv DATABASE INSTANCE1 (RAC1)
ORACLE_SID = [orcl1] ? +ASM2
ORACLE_HOME = [/home/oracle] ? /u01/app/12.1.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@oel6-112-rac1 Desktop]$ asmcmd --privilege sysasm --inst +ASM2

Patterns 21
Oracle Database 12cR1 with FLEX ASM

ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB
Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 15342 4782
0 4782 0 Y DATA/
ASMCMD>

SUMMARY
The database instance was using a dedicated ASM instance and that
ASM instance was forced to stop working simulating a failure, so the
database instance reconnected to an existent ASM instance on another
node, for this example node 2 (rac2).

Patterns 22
FLEX ASM: SUPPORTING ORACLE DATABASE 11g

 Previous Database versions will host Local ASM Instance 

RAC CLUSTER
Database Instance

DATABASES SHARE 11.2 11.2

DBA
ASM Instance
DB DBA DBB DBB DBB DB DBC
ASM INSTANCES

Node1 ASM Node2 ASM Node3 ASM Node4 ASM Node5 ASM

ASM Cluster Pool of Storage

SHARED DISK GROUPS Disk Group A Disk Group B ASM Disk

WIDE FILE STRIPING

Patterns 23
Oracle FLEX CLUSTER
 Comprises of a Hub and Leaf architecture Depicts a typical Oracle flex cluster with four Leaf nodes and two Hub
nodes. In a nutshell Oracle Flex Cluster requires Oracle Flex ASM.
where in only the Hub nodes will only have
direct access to Oracle Cluster Registry
(OCR) and Voting Disk (VD)
 However application can access the
database via Leaf nodes without ASM
instance NOT running on Leaf nodes
 The connection to the database is through
Hub making it transparent for the
application
Patterns 24
HUB NODES & LEAF NODES
Hub Nodes
They are connected among them via private network and have direct access to
the shared storage just like previous versions. These nodes are the ones that
access the Oracle Cluster Registry (OCR) and Voting Disk (VD) directly.

Leaf Nodes
These nodes are lighter and are not connected among them, neither access the
shared storage like the Hub Nodes. Each Leaf Node communicates with the Hub
Node that is attached to, and its connected to the cluster via the Hub Node that
is linked to.
Patterns 25
FLEX ASM & FLEX CLUSTER
Flex ASM Flex CLUSTER
 Higher Availability (DB  Less complex configuration and management for
instances can use remote ASM non-database nodes
 Less complex configuration for Non-Database
instances)
nodes, such as Processing Nodes, Application /
• Planned or Unplanned
Web Server Nodes etc.
downtime
 Reduces per-node ASM
instance overhead
• ASM only needs to run on 3
nodes in the cluster
 Facilitates DB, Cluster & server
consolidation
 Multiple DB’s & applications in
a cluster
Patterns 26
PREVIOUS ASM CLUSTER

Cluster
Server Server

 ASM architecture utilized an ASM ASM ASM

instance on every server

DB1 DB2 DB3 DB1 DB2 DB4

 Database instances dependent on

Server Server
node-specific ASM instance
ASM ASM
 ASM overhead scaled with size of
cluster DB1 DB5 DB4 DB1 DB5 DB6

Patterns 27
NEW ASM CLUSTER
 Eliminates requirement for an ASM instance on every cluster
server Cluster
Server Server Server Server

 Database instances connects to any ASM ASM

instance in the cluster Server Server Server Server


ASM

Database instances can failover to a

secondary ASM instance Server Server Server Server

ASM

 Administrators specify the cardinality of ASM

instances (default is 3)
 Clusterware ensures ASM cardinality is maintained
Patterns 28
FLEX ASM NETWORK
 Flex ASM adds the ASM network, which is used for
communication between ASM and its clients
 Oracle 12c, you can now create a separate network
dedicated for ASM communication.
 Need to mention the network address during the
installation
 This dedicated ASM network is completely optional.
The default is the pre-12c behavior of using the
Clusterware interconnect for ASM communication
Patterns 29
DEPLOYING FLEX ASM
Managing Flex ASM
 Flex ASM requires minimal monitoring and ongoing management
 Primary objective is that instances are up and running
 SRVCTL Command for:
 Checking ASM instance status
 Setting cardinality
 Starting – Stopping - and - Relocating ASM instances
 No Flex ASM-specific instance parameters are required
 Default settings will effectively support most situations
 ASM server instances use automatic memory management
 Flex ASM is managed by ASMCA, CRSCTL, SQL*Plus and SRVCTL
Patterns 30
NEW FLEX ASM

Flex Features
 Increase maximum number of Disk Groups to 511 (Previous
limit was 63)
 Command for renaming ASM Disk
 ASM Relocate Command
 Clients are automatically relocated to another instance if
an Oracle ASM instance fails
 When necessary, Clients can be manually relocated
using the ALTER SYSTEM RELOCATE CLIENT command

Patterns 31
New Option in Grid Infrastructure

Oracle ASM Deployment Model can be

chosen during the installation of
Oracle Grid Infrastructure
Oracle ASM Configuration Assistant
(ASMCA) can be used to enable Oracle
Flex ASM after the installation /
upgrade is performed
The functionality is available only in an
Oracle Grid Infrastructure
Configuration

Patterns 32
Conversion of Standard Cluster to
Oracle Flex ASM Cluster

Patterns 33
Patterns 34
 CHECK CLUSTER MODE AND CLUSTER STATE AFTER
LOGGING INTO ASMCMD PROMPT
[oracle@racnroll1 ~]$ asmcmd
ASMCMD> showversion
ASM version : 12.1.0.1.0
ASMCMD>showclusterstate
Normal
ASMCMD>showclustermode
ASM cluster : Flex mode enabled

Patterns 35
 ORACLE
Flex ASM and Flex Cluster

Patterns Software Design Institute Pvt. Ltd.

www.patterns-sdi.com

Patterns 36