7.

IDMS (Integrated Database

Management System)

IDMS was originally developed in 1971 by B.F. Goodrich Chemical Company to

support their distribution billing and accounting applications. The commercial
possibilities were soon realised by the company, but they felt they were in no
position to offer the necessary support. It was therfore decided to sell the pro-
duct. In September 1973, the Cullinane Corporation puchased the marketing
and development rights to IDMS. Early in 1975, SCICON were given the UK
marketing rights for the IBM 360/370 range, ICL bought the rights to IDMS for
their computers and developed a separate version of IDMS for the 2900, which
they market world-wide, as have UNIVAC for their 1100 and 90 series. ADV-
Orga of West Germany hold the marketing rights for most of the German-speak-
ing countries of Western Europe.
ADV-Orga have developed an IDMS version to run on the Siemens 4004 series,
and its successor the UN ID ATA/Siemens 7000 series.
IBM is the only major main-frame manufacturer not to offer a CODASYL-based
DBMS. This gap has been filled by two developments: SIBAS and IDMS. The
latter is by far the more successful.
The main advantage of the CODASYL DBMS is that the user interface remains
the same irrespective of the hardware supporting the system. This protects the
investment made in currently running applications and EDP training. This advan-
tage is somewhat negated by the dramatic changes proposed in the 1978
CODASYL DBTG report.
While there is no necessity for IDMS to implement the 1978 recommendations,
failure to do so would seriously detract from IDMS's portability, with a conse-
quent negative impact on sales. On the other hand, current users will have a large
amount of rewriting to do to upgrade their IDMS applications should the new
recommendations be implemented.
This report examines the IBM 360/370 version. The comments may or may not
apply to other versions. The newly announced PDP-11 version will almost cer-
tainly not contain all the functions of the main-frame version.
IDMS has been interfaced to the following TP monitors:
- SHADOW II
- CICS
- TASK/MASTER
- INTERCOM
- WESTI
- ENVIRON/1.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
234 7. IDMS (Integrated Database Management System)

A few years ago, the Cullinane Corporation entered into an agreement with
Altergo to market SHADOW II in North America. This would have obviated the
necessity for Cullinane to develop their own TP monitor. However, about a year
later, the agreement was nullified. Altergo set up their own marketing organi-
sation in North America and Cullinane developed their own TP monitor IDMS—
DC, which is fully integrated with IDMS—DB.
A Cullinane product, CULPRIT, a generalised report generator, has been inter-
faced to IDMS. The other two supporting software products required by any
successful DBMS - a query language and a data dictionary — have both recently
been introduced by Cullinane. They are both rather primitive when compared to
the leading products in their respective fields. Nevertheless, it can be expected
that they will be further developed over the next few years until they reach the
high standard set by IDMS itself.
B.F. Goodrich has stated publicly that their agreement with Cullinane specifies
that IDMS will only be developed in line with CODASYL ideals. This does not,
however, mean that Cullinane must implement the whole of the 1978 CODA-
SYL DBTG recommendations, merely that any extensions which are made will
be in keeping with CODASYL recommendations. All these extensions have in
any case to be agreed between Cullinane and representatives of the current users.

7.1 Classification
IDMS is a host language system using a preprocessor to produce a COBOL,
FORTRAN, Assembler or PL/1 source program.
All other computer languages that contain a 'CALL' statement can also access an
IDMS database.
IDMS is a pointer-based system, capable of implementing both hierarchies and
networks. The pointer information is held within the record itself, but it is never
made available to the user. The pointer contains a physical block address.
The network structures of IDMS are based on CODASYL SET relationships. A
set consists of an OWNER record type and one or more MEMBER record types.
The rules for relating OWNERS to MEMBERS are as follows:
— a set may have only one OWNER type record,
— a record type cannot be both OWNER and MEMBER in the same set,
— a set may contain multiple record types as MEMBERS,
— records may be OWNERS and MEMBERS in an unlimited number of different
sets.
There are two main languages within IDMS: the DDL (Data Definition Language)
and the DML (Data Manipulation Language). The DDL consists of three com-

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.2 Data Definitions (including Set and Pointer Definition) 235

pilers: the schema compiler; the subschema compiler (see 7.5. Data Independ-
ence); and the DMCL (Device/Media Control Language) which is used to map
the schema — the logical structure — onto physical storage.

7.2 Data Definitions (including Set and Pointer

Definitions)
Data ITEM definitions are based on ANS COBOL. (For a detailed treatment of
this subject, refer to the ITEM description in the IDMS 'Database Design and
Definition Guide'.)
The LEVEL NUMBER must be the first entry in the ITEM description, immedi-
ately followed by the 'item-name'. The remaining entries can follow in any order.
— LEVEL NUMBER is an unsigned integer in the range 02—49 or 88, allowing
logical relationships to be defined between ITEMs. 01 level is generated by
the system with the record name.
— ITEM—NAME is up to 32 characters long, and should be unique
— REDEFINES clause enables a previously defined ITEM'S representation in
storage to be changed
— PICTURE clause describes the characteristics of the ITEM
— USAGE clause describes the way an ITEM is held in storage. The options are:
DISPLAY, BINARY. LONG or SHORT PRECISION FLOATING POINT,
PACKED DECIMAL.
— VALUE clause assigns an initial value to the ITEM
— SYNCHRONIZED clause is treated as comment by the SCHEMA DDL pro-
cessor
— OCCURS clause is used to define repeating groups or ITEMS, between 2 and
9999 being permissible
— INDEXED clause is related to the OCCURS clause
— COMMENT clause is used to enter the text for inclusion in the data dictio-
nary.
There is no limit to the number of ITEMS that can be defined within a logical
record.
The logical record is itself described by five parameters in the RECORD
DESCRIPTION:
— RECORD NAME is used to identify the record type uniquely within the
SCHEMA. It can consist of up to 16 alphanumeric characters (a maximum of
8 characters are allowed for Assembler synonyms.)
— RECORD ID is a number between 100 and 9999 which may be used by the
DBA to identify a record type within the SCHEMA. The RECORD IDs 1 - 9 9
are reserved for internal use within IDMS.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
236 7. IDMS (Integrated Database M a n a g e m e n t S y s t e m )

Record to be inserted CURRENT of its S E T

HEADER

AS

AT
PAGE 123

(41 (3) (2)

(2) 0) FOOTER

|1233|
D B KEV is returned to the user

Fig. 7.2a: Storage Using LOCATION MODE 'VIA C U R R E N T of S E T '

- WITHIN. This option allows the DBA to specify an AREA of the SCHEMA
within which the record occurrence should be stored. A range of PAG ES may
also be specified.
- LOCATION MODE is the mechanism which allows the DBA to control the
way the record occurrences are stored within the database. The efficiency of
the database design will depend on the competence of the DBA in using this
parameter. Three options are available:
(1) CALC (RANDOM) is a placement method whereby a specified ITEM
within the record occurrence is subjected to a hashing algorithm which pro-
duces a PAGE address. DUPLICATE ITEM values can be either allowed or
disallowed (see Fig. 7.2b).
(2) DIRECT. IDMS allocates a unique DATABASE KEY to every record
stored within the database. With the Dl RECT mode, it is possible for the user

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.3 Data Storage Structures 237

HEADER

1234 ABC Co. Ltd.

free space

(4) FOOTER

[76541
DB-KEY THAT IS RETURNED
TO THE USER

Fig. 7.2b: Storage Using CALC LOCATION MODE

to recommend a PAGE number to be allocated to the record on its being

stored into the database. If this key has already been allocated, then IDMS
will allocate the next available PAGE number to the record occurrence (see
Fig. 7.2c).
(3) VIA. This placement mode allows record occurrences to be placed physi-
cally close to each other, eg. all MEMBER occurrences of a SET occurrence
to be placed in the same PAGE (or an adjacent PAGE), so minimising the disc
accesses for processing these records (see Fig. 7.2a).
(4) INDEXED SPF assigns a PAGE number to a record occurrence.

7.3 Data Storage Structures

7.3.1 Physical Representation
An IDMS database is held on direct storage and is accessed using BDAM (under
DOS—DAM). Relative block BDAM addressing is used with IDMS, providing the

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
238 7. I OMS (Integrated Database Management System)

HEADER

AS

AT

free space

(-) (4) (3) (2) (1) FOOTER

The 'suggested' DB-KEY

supplied by the user
is already in use so
IDMS allocates a free
DB-KEY from the same
PAGE
Record tobe inserted and
the'suggested' DB-KEY

HEADER

AS

AT

AV

free space

(5) (4) (3) (2) (1) FOOTER

112351
DB-KEY returned to user

Fig. 7.2c: Storage Using DIRECT MODE

mapping from the schema AREAS to the physical storage.

VSAM is now also available as access method.
Each record occurrence stored in IDMS consists of two parts: a prefix for the
structure data which contains a number of four-byte pointer fields; and the user
data. If variable length records are being used, then the prefix will contain a

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.3 Data Storage Structures 239

further field of 8 bytes. Variable length records result either from data compres-
sion (via a user exit) or by using the OCCURS . . . DEPENDING ON construc-
tion. The length of the prefix can vary greatly depending on the number of
relationships the particular record type participates in.
The records are stored in an ID MS PAGE, which is the basic unit of physical
storage. The size of the PAGE is fixed by the DBA at system generation time for
each AREA, and is typically between 2K and 6K bytes. An IDMS pointer
(DATABASE KEY) is 4 bytes in length. This is broken into two parts: 23 bits to
represent the LOGICAL PAGE NUMBER (which is unique across the whole
SCHEMA), and 8 bits for the LINE (record) INDEX within the PAGE. An IDMS
database can contain up to 223—1 PAGES and each PAGE can hold up to 255
records.
Fixed length records must be stored wholly within one PAGE. Variable length
records can be broken up into one root segment and a number of fragments
which are all chained together.
Each PAGE consists of five parts (see Fig. 7.3.1):
— a 16-byte PAGE HEADER
— a number of data records
— free space
— control information to locate and manage each record
— a 16-byte PAGE FOOTER.
The PAGE HEADER holds the following information:
— the unique PAGE number (LPN)
— pointers for CALC record placement
— a free space indicatior
— switches to aid buffer management.
The 8-bit LINE INDEX in the DATABASE KEY does not point directly to the
record but to the RECORD LOCATION POINTER at the foot of the PAGE.
This is because the location of a record within a PAGE can vary, (see Fig. 7.3.2
Space Management).

7.3.2 Space Management

The RECORD LOCATION POINTERS (see Fig. 7.3.1) contain a record length
field, which means the IDMS operates only on variable length records. Records
are inserted into a PAGE directly after the header. No space is needed after a
record to allow for future expansion. Should a record be deleted, then IDMS auto-
matically reorganises the PAGE in order to maintain the largest possible area of
contiguous free space. The record location pointer for the deleted record will be
set to zero and is immediately available for reuse (see Fig. 7.3.2).

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
240 7. IDMS (Integrated Database Management System)

PAGE NEXT POIN1ER PRIOR POINIER FREE SPACE NOT

NUMBER FDR CALC SET FOR CALC SET COUNT USED
4 bytes 4 bytes A bytes 2 bytes 2 bytes

structural User Data

(SET) pointers

RECORD OF
VARIABLE
LENGTH

16 byte HEADER

Record 2

free space

(3) (2) 0) 16 byte FOOTER

record record postion position of

type length in page data in record

RECORD LOCATION POINTER

( « BYTES)

Fig. 7.3.1: Physical Storage PAGE Layout

If a variable length record expands and there is insufficient space within the
PAGE for the new enlarged record, the record will be split into two parts, the
root containing the principle key, and a pointer to the rest which will be ac-
commodated in a neighbouring PAGE. This process can be repeated, resulting in
a number of fragments linked together with pointers. Should pointer mainte-
nance be necessary due to the deletion of a fragment, the system will try to re-
organise the record to avoid or at least reduce the fragmentation.
The first 1000 PAGES of an I DMS database are reserved for use by the Data
Dictionary. The first PAGE of each AREA is used for SPACE MANAGEMENT.
They are only updated when the occupancy of a particular PAGE exceeds 70%.
There are no overflow PAGES within an I DMS database, although it is possible
to specify that only a part of each PAGE will be occupied during an initial load;

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.3 Data Storage Structures 241

HEADER RECORD! RECORD 2

RECORD 3 RECORD4

F R E E SPACE

(4) 0) (2) (1) FOOTER

PAGE 123 BEFORE AND AFTER RECORD 3

( D B - K E Y = 1 2 3 3 ) H A S B E E N DELETED

HEADER RECORD 1 RECORD 2

F R E E SPACE

(4) (-) (2) (1) FOOTER

• The pointer to RECORD 4 must be updated to reflect the

record's new position on the R&GE following the delete
• The Inde* Position 3 a n d DK-Key value 1233 are now
available for the next record stored on the RAGE

Fig. 7.3.2: Space Re-use After Deletion

this free space can be used later to accommodate expanded records and new
records.
Essentially, space management is only necessary with the CALC records. The
VIA or DIRECT LOCATION MODES only indicate where the record should be
placed. If there is no space available, a neighbouring PAGE is equally acceptable.
The actual DATABASE KEY is returned and can be used to retrieve these
records. With CALC placement, however, the randomizing algorithm generates a
particular PAGE as address. If this particular PAGE is full, an overflow condition
occurs, and an overflow pointer must be maintained in this PAGE to the PAGE
where the record is actually stored. This will adversely affect performance. The
DBA must therefore either change the algorithm or change the PAGE size,

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
242 7. IDMS (Integrated Database Management System)
should a large number of records be stored in a PAGE other than that indicated
by the CALC algorithm.
The general objective with I/O buffers is to reduce to a minimum movement
between main storage and secondary storage. PAGES are returned to secondary
storage only when absolutely necessary. This can be caused by a COMMIT
command, which requires that all PAGES containing record occurrences which
have been changed by the RUN UNIT which has issued the command should be
secured. A second possibility is that the buffer becomes full. Each PAG E would
then be examined and that PAGE with the lowest priority, containing a modifi-
cation, would be returned to secondary storage. BEFORE and AFTER IMAGE
PAGES for the JOURNAL FILE are managed in a similar manner.

7.3.3 Logical Storage Structures

There are two ways of Unking data within an IDMS database, either by inter-
field or inter-record relationships.
— Inter-field Relationships. An ITEM within an IDMS logical record is equivalent
to a field within COBOL. A hierarchical relationship between ITEMS is estab-
lished by means of level numbers in the record description (in the SCHEMA
DDL). A maximum of 48 levels is permitted.
- Inter-record Relationships. The CODASVL name for an inter-record relation-
ship is a SET, which is defined using the SCHEMA DDL and implemented by
physical pointers embedded within each record prefix.
A SET can be used to define both network and hierarchical structures within
the following constraints:
(1) a SET has only one OWNER type record,
(2) a record type cannot be both OWNER and MEMBER of the same SET,
(3) a SET may consist of different types of MEMBER records,
(4) records may be OWNERS and MEMBERS in an unlimited number of
different SETS.

Fig. 7.3a: Types of pointer

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.3 Data Storage Structures 243

New MEMBER record New MEMBER record

stored here with FIRST stored here with LAST
ordered SETS ordered SETS

P - PRIOR
MEMBER) O - OWNER
N- NEXT

N B PRIOR pointers
a r e n e c e s s a r y with
LAST ordered SETS

Fig. 7.3b: Insertion Rules 1

NB PRIOR pointers
are necessary
with PRIOR order

Inserted here
with NEXT order

Fig. 7.3c: Insertion rules 2 (PRIOR/NEXT)

NB PRIOR pointers
* Inserted here
are not absolutely
™ with ASCENDING
n e c e s s a r y with
order independent
SORTED SETS
of current position

Fig. 7.3d: Insertion Rules 3 ( A S C E N D I N G / D E S C E N D I N G Order)

There are three different types o f pointer ( D A T A B A S E K E Y ) used to relate

OWNER and MEMBER within a SET (see Fig. 7.3a). N E X T pointers are auto-
matically inserted by IDMS, P R I O R and OWNER pointers are optional. It is
always possible to reach the 'prior' record to the current position simply by
following the N E X T pointer chain. OWNER pointers can be used to increase
access efficiency when the OWNER record occurrence is required o f a MEM-

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
244 7. IDMS (Integrated Database Management System)
BER record occurrence which has been accessed via another SET relationship
or a secondary index. The PRIOR pointer can save having to traverse the
whole SET. It is particularly useful in volatile SETS where, for the same
reason, it increases delete efficiency. (The NEXT pointer of the previous record
has to be changed to point to the following record when the current record
is to be deleted). Thus order in which records are inserted into SETS can be
controlled (see Fig. 7.3b, c and d), and the following options are available:
(1) FIRST, the next record to be inserted will appear directly after the
OWNER (giving LIFO sequence).
(2) LAST, the next record to be inserted will be placed at the end of the
MEMBER chain (giving FIFO sequence).
(3) NEXT, the new record will be inserted immediately after the CURRENT
record in the SET.
(4) PRIOR, the new MEMBER will be inserted immediately before the CUR-
RENT record in the SET.
(5) Sorting on a field within the record, achieving an ascending or descending
logical sequence within the SET occurrence.
The NEXT and PRIOR options leave the record sequence under programmer
control.
There are two further qualifications required to fully define SET MEMBER-
SHIP, either AUTOMATIC or MANUAL and either MANDATORY or
OPTIONAL. These parameters are concerned with establishment and termina-
tion of SET MEMBERSHIP:
(1) AUTOMATIC or MANUAL MEMBERSHIP. The former means what it
implies ie. that as a result of a STORE command, this record becomes a
MEMBER of all SETS in which it is defined. MANUAL MEMBERSHIP can
only be established for a record occurrence after it has been stored in the
database.
(2) MANDATORY or OPTIONAL MEMBERSHIP. With theformer,an occur-
rence cannot be removed from a SET without being deleted from the data-
base. With the latter an occurrence's participation in a SET can be deleted,
without disturbing its MEMBERSHIP in any other SET. The commands
associated with these different MEMBERSHIP types are explained in the
'Modification Commands' section of 7.4 'Data Manipulation'.

7.3.4 Secondary Processing Sequence

The SEQUENTIAL PROCESSING FACILITY (SPF) provides a secondary pro-
cessing capability, so that any record may be declared a database entry record.
This facility is implemented in IDMS as a normal SET, however, the user does
not have to maintain it. The secondary index may be based on any field(s)
within the desired record type. The designer must choose whether duplicated

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.4 Data Manipulation 245

occurrences of a particular value are to be allowed. There is no restriction on the

number or type of other relationships which the object record type may partici-
pate in. The SPF locates the desired entry record with a few accesses via a binary
search of the sparse-to-dense hierarchically arranged index.

7.3.5 Generic Key Accessing

Partial key retrieval is available for both primary and secondary index values.
However with a CALC key it has no meaning.

7.4 Data Manipulation

The original intention of the CODASYL database activities was to extend the
COBOL language by providing database facilities. The commands necessary to
access the database are provided within a DATA MANIPULATION LANGUAGE
(DML). The commands are embedded in a COBOL program, in a way similar to
the normal COBOL commands. However, as these commands are not yet (if ever)
recognised as part of the COBOL language, they are translated into 'CALL' state-
ments by a preprocessor. Preprocessors are also available for PL/1, FORTRAN
and Assembler.
The logical structure that is available to the program is defined in the SCHEMA
SECTION of the DATA DIVISION.
The IDMS interface module is link-edited to the user's relocatable object program
prior to execution. Fig. 7.4 reflects the actions that occur during the execution
of a typical DML statement:
1. The COBOL program issues a CALL instruction to the interface routine.
Parameters of the CALL instruction identify what action, to which database, is
required.
2. The interface routine passes the request to the I DMS nucleus which will carry
out the required processing.
3. In order to be able to process the request, the nucleus will require the infor-
mation held in the object SCHEMA. The object SCHEMA contains a representa-
tion of the data structure, CURRENCY status, record placement control, record
and SET characteristics, database usage statistics and database usage limitations.
4. After the request has been processed, the contents of the object SCHEMA
must be updated to reflect the new conditions. Should the processing request
have involved the location of a record, then the DATABASE KEY and other
record related information is moved from the I DMS system buffers to I DMS
COMMUNICATIONS BLOCK in the user program.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
246 7. IDMS (Integrated Database Management System)

Fig. 7.4: Execution of a DML Command

5. If the processing request involved data retrieval (a GET command), the data
will be moved from the IDMS system buffers to the program RECORD DELIV-
ERY AREA. (Data movement in the reverse direction will occur in response to a
STORE or MODIFY command.)
6. Control is returned to the interface routine with an indication as to the suc-
cess or failure of the request.
7. The interface routine moves status information to the user's IDMS COMMU-
NICATIONS BLOCK!
8. Control is returned to the user program.
9. The user program can now continue processing. The results of the request are
held in summarised form in the IDMS COMMUNICATIONS BLOCK.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.4 Data Manipulation 247

The I/O area for the database resides in the program work area. Each record type
specified in the SUBSCHEMA is included in the source program as a record level
entry followed by the record contents description. On retrieval, a record occur-
rence is always placed in its like-named area in the program work area. The
reverse is true when a record occurrence is to be stored in the database.
Three operations are required each time a database command is issued:
— initialisation of data items as required by the DM L command to be issued
— DML command to initiate the required action
— error checking to determine the outcome of the DML command just executed.

7.4.1 C U R R E N C Y
The concept of CURRENCY is used in a number of systems to identify the
most recently accessed record, so as to be able to access NEXT, PRIOR, FIRST,
LAST, OWNER etc records. IDMS takes this idea much further and has four
types of CURRENCY:
- CURRENT OF RUN-UNIT (program). This is the last record successfully
manipulated by a STORE or a FIND/OBTAIN DML command. The DATA-
BASE KEY of this record is placed in a control location within the WORK-
ING STORAGE SECTION.
- CURRENT OF AREA. This is the most recent record from the AREA that
was CURRENT OF RUN-UNIT.
- CURRENT OF SET. For each SET, it is that record which was most recently
the CURRENT OF RUN-UNIT.
- CURRENT OF RECORD TYPE. This is that record which last was CUR-
RENT OF RUN-UNIT within each record type.

7.4.2 DML Commands

The DML commands are split into three categories: control, retrieval and modi-
fication.
- Control Commands. There are a number of commands available within
IDMS's DML which do not cause any user data to be moved, but rather signal
processing intention or retrieve metadata, ie. information about user data.
The sequence of these commands is as follows:
BIND RUN-UNIT issued once per RUN-UNIT
BIND RECORD NAME etc. issued at least once
READY issued at least once
IF does not have to be used

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
248 1• IDMS (Integrated Database Management System)
manipulation (retrieval/modification) commands
KEEP does not have to be used
COMMIT does not have to be used
FINISH issued once per RUN-UNIT
ROLLBACK used to restore the database to its orig-
inal state after an irrecoverable pro-
gram error or at the end of a test run
(1) The BIND Command. This is used to sign-on the RUN-UNIT, and as such
must precede all other DML commands. It informs IDMS of the location of
the IDMS COMMUNICATIONS BLOCK and identifies the SUBSCHEMA
which should be loaded for the subject RUN-UNIT. It is formulated as fol-
lows —
TO subschema-control
BIND RUN-UNIT [FOR subschema-name]
identifier
The command is further used to establish the addressability of any required
record from within the 'signed-on' schema. It is formulated —

BIND i r e c o r d - n a m e [TO identifier]

1 identifier WITH record-name-identifier

The final use of this command it to establish communication between the

RUN—UNIT and a DBA-written database procedure. It is however unusual
that more information is required by such routines. Normally these proce-
dures are executed without the intervention of the RUN—UNIT. This final
command is formulated —
BIND PROCEDURE FOR proc-name-ident TO proc-common-area
(2) The COMMIT Command. This creates a CHECKPOINT, releases all
records held as the result of a KEEP command and flushes the buffers. The
only difference between a COMMIT command and a FINISH—BIND—
READY command sequence is that in the former, all CURRENCY pointers
remain undisturbed. The COMMIT command is formulated —
COMMIT
(3) The FINISH Command. This causes the following actions to occur:
- control over all AREAS is withdrawn from the RUN-UNIT
- any other RUN—UNIT is permitted to assume control over the released
AREAS
- a CHECKPOINT is written
- statistical information relating to those database operations performed
during RUN—UNIT execution is released to the JOURNAL file.
The FINISH Command is formulated:

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.4 Data Manipulation 249

FINISH
( 4 ) The IF Command. This falls between control commands and data manipu-
lation commands. It operates on user data but does not manipulate it. It is
used to:
- detect the presence of member record occurrence
— evaluate the membership status of a record occurrence in a specified S E T
occurrence.
It is formulated:

IF set-name IS [NOT] EMPTY imperative-statement

IF [NOT] set-name MEMBER imperative-statement

( 5 ) The KEEP Command. This command can be used in stand-alone mode or

as part o f a FIND or OBTAIN command. It sets the select ( K E E P ) or update
( K E E P E X C L U S I V E ) lock for a record, S E T or AREA. It is formulated:

record-name
KEEP [EXCLUSIVE] CURRENT WITHIN set-name
WITHIN area-name

( 6 ) The READY Command. This command serves two purposes:

— to specify those A R E A S to be accessed and the intent, ie. read/update
- to specify the initial CHECKPOINT condition for the R U N - U N I T for
possible rollback/recovery operations following a system-failure.
It is formulated:

PROTECTED RETRIEVAL
READY [area-name] U S A G E - M O D E IS
EXLUSIVE UPDATE J

- Retrieval Commands. These are: FIND/OBTAIN, GET, ACCEPT and the

modification commands.

( 1 ) T h e FIND/OBTAIN commands. The FIND command locates and the

OBTAIN command both locates and retrieves a record according to the selec-
tion criteria specified. (Strictly speaking, the FIND command locates the record
of interest and retrieves it into the IDMS buffer, and the G ET command moves
it into the program work area.) The OBTAIN command functions as if a FIND
command followed by a GET command had been executed. The successful
completion of a FIND/OBTAIN command results in the located record
becoming C U R R E N T of AREA, all S E T S in which the record participates
and record name. Its DATABASE KEY is returned in a user-specified loca-
tion. There are six main variation of this command:

(a) via D B - K E Y . The record islocated via a fieldin storage containing its key.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
250 7. IDMS (Integrated Database Management System)

FIND [record-name] D B - K E Y IS db-key-identifier

(b) C U R R E N T record. That record will be retrieved which is C U R R E N T o f

S E T , record or A R E A name.

record-name
FIND C U R R E N T WITHIN set-name
.WITHIN area-name

NB. Issuing 'FIND C U R R E N T ' without further qualification ie. for the CUR-
RENT of RUN—UNIT, does not accomplish anything, as IDMS has not imple-
mented the RETAINING CURRENCY option.
( c ) from within a S E T AREA or Index. Records from within S E T S or
A R E A S are located with this command.

NEXT
PRIOR
FIRST J set-name
FIND [record-name] WITHIN >
LAST area-name
integer
identifier

The NEXT and PRIOR options are based on the C U R R E N C Y o f the specified
A R E A or S E T ; the F I R S T and LAST options relate to logical order for S E T S
and to DB K E Y order for A R E A S ; the 'integer' and 'identifier' options allow
the retrieval o f the n t h record occurrence o f a S E T .

(d) OWNER record within a S E T . This command retrieves the OWNER record
o f the C U R R E N T S E T .

FIND OWNER WITHIN set-name

(e) CALC records. With this command, records whose LOCATION MODE is
CALC can be retrieved.

ÍCALC (ANY))
FIND record-name
(DUPLICATE J

N.B. The 'CALC' option functions in conjunction with a specific 'CALC'

key quoted in WORKING S T O R A G E . The 'DUPLIKATE' option
accesses the next record with the same 'CALC' key as the C U R R E N T
of record type.

( f ) from an ordered S E T . This command retrieves a MEMBER record from a

sorted S E T using a specific key.

FIND record-name WITHIN set-name [ C U R R E N T ] USING identifier

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.4 Data Manipulation 251

(2) The GET Command. This command simply transfers the CURRENT record
of the R U N - U N I T or record name f r o m the I/O buffer to the program work
area.
GET [record-name]

(3) The ACCEPT Command. This allows the user to move CURRENCY indi-
cators (DATABASE KEYS) and storage-address information to specified
location in his program. It is formulated

record-name
ACCEPT db-key-identifier FROM set-name CURRENCY
area-name

If neither record nor SET nor AREA name is specified, the D A T A B A S E -

KEY for the CURRENT record of R U N - U N I T is returned. It is also possible
to retrieve the D A T A B A S E - K E Y S of the NEXT, PRIOR and OWNER
records as follows

[NEXT I
ACCEPT db-key-identifier FROM set-name j PRIOR | CURRENCY
(OWNER)

The other three variations of the ACCEPT command are beyond the scope of
this report.
- Modification Commands. These are ERASE, CONNECT, MODIFY, DIS-
CONNECT, STORE.

( 1 ) T h e CONNECT command. With this command, a record occurrence

(which must be CURRENT record of its record type) is made a MEMBER of
a SET occurrence. The record type must be defined as OPTIONAL AUTO-
MATIC, OPTIONAL MANUAL or MANDATORY MANUAL. It is formulat-
ed:

CONNECT record-name TO set-name

(2) The DISCONNECT command. This removes a record occurrence f r o m its
participation in a SET occurrence. The record must be defined as an OPTIO-
NAL MEMBER of the SET concerned. The record's occurrences in all other
SET occurrences are totally unaffected. This c o m m a n d does not remove the
record occurrence f r o m the database. It is formulated:

DISCONNECT record-name FROM set-name

(3) The ERASE command. This enables the user:
(a) to disconnect the object record occurrence f r o m all set occurrences in
which it participates as a member

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
252 7. IDMS (Integrated Database Management System)

(b) to erase all those record occurrences which are MANDATORY MEM-
BERS of a particular SET owned by the object record.
(c) optionally to ERASE all record occurrences which are MANDATORY
MEMBERS of SET occurrences where the object record is OWNER
(d) optionally to DISCONNECT or ERASE all record occurrences which are
OPTIONAL MEMBERS of SET occurrences where the object record is
OWNER.
The command is formulated:

PERMANENT
ERASE record-name SELECTIVE I MEMBERS
ALL

(4) The MODIFY command returns a previously accessed record occurrence

t o the database. Any or all of the data items within the object record may
have been replaced with new values. It is formulated:
MODIFY record-name
(5) The STORE command inserts a record occurrence into all the SET occur-
rences in which it has been defined as an AUTOMATIC MEMBER. It is for-
mulated:
STORE record-name

7.5 Data Independence

The DBA, having defined the global view of the database (SCHEMA) can define
a subset (SUBSCHEMA) of this overall view for each user, thus limiting the user
to only that information which is absolutely necessary. This insulates the user
from changes to the database structure which do not affect his application.

7.5.1 The Local View

IDMS was originally designed (in keeping with CODASYL recommendation) to
interface with COBOL via a preprocessor which would translate the DML
command, embedded in the COBOL program into COBOL CALL commands.
Preprocessors are now also available for Assembler PL/1 and FORTRAN.
Compiling the program only binds the specified DML commands. The SUB-
SCHEMA and the DMCL mapping statements are compiled separately and pro-
duce the SUBSCHEMA OBJECT MODULE and the DMCL OBJECT MODULE.
These modules are bound to the program at run time. Data is bound to the pro-

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.6 Concurrent Usage 253
gram partly at compile time and partly at run time, the former generating relative
addresses and the latter absolute addresses.
It is not possible to vary ITEM formats and/or lengths between SCHEMA and
SUBSCHEMA. However, the user's view (SUBSCHEMA) of the database can be
restricted to:
— particular SET types ie. logical relationships
— particular record types
— particular fields within record types
There is, however, one restriction when defining the SUBSCHEMA, namely that
this local view must contain all record and item descriptions which could be
affected by insertion and deletion activity.

7.6 Concurrent Usage

The latest version of IDMS - Release 5 - offers a dramatic improvement in
throughput as a result of a multithreaded CENTRAL VERSION (CV).The S I G N -
ON command issued by R U N - U N I T is received by the D A T A B A S E -
RESOURCE-CONTROLLER module (IDMSDBRC) which allocates an EXTER-
NAL—RUN—UNIT—SERVICE module (IDMSERUS). This latter serves as a link
between the R U N - U N I T and the IDMS nucleus. The SUBSCHEMA TABLE is
now part of the CV, whereas in previous releases it was attached to the R U N -

BATCH INTERFACE WAIT DISPATCHER BATCH INTERFACE

IDMSERUS , IDMSERUS •
A B
k 1 IDMSDBRC
SUBSCHEMA . SUBSCHEMAI
WORK AREA'WORK AREA"
WORK AREA
USER A USER
1 B J PROGRAM
PROGRAM
A B

IDMS NUCLEUS

DMCL } SUBSCHEMA TABLES

TABLES I (SHARED)

Fig. 7.6.1: BATCH MODE

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
254 7. IDMS (Integrated Database Management System)
UNIT. The advantage of the new configuration is that one SUBSCHEMA
TABLE can be shared b e t w e e n R U N - U N I T S . The SUBSCHEMA T A B L E S are
held on secondary storage and read into main m e m o r y as required. It is possible,
however, to m a k e t h e m resident t o improve effeciency.
The SUBSCHEMA TABLES, which only contain database descriptions, are re-
entrant, thus allowing a single copy t o be used concurrently b y multiple R U N -
UNITS. Other data specific t o each R U N - U N I T eg. C U R R E N C Y pointers, is
stored in its own 4 K , dynamically-allocated, work-area held within t h e CV.
The IDMS—DC nucleus is resident in the C E N T R A L VERSION f u n c t i o n i n g as a
'wait dispatcher', monitoring the activities of the master DATA—BASE—
R E S O U R C E - C O N T R O L L E R m o d u l e and all E X T E R N A L - R U N - U N I T - S E R -
VICE modules. This step integrates the IDMS database and teleprocessing
systems. The IDMSDBRC also communicates with the operator, and m o n i t o r s
the IDMSERUS m o d u l e for abnormal R U N - U N I T termination.

The C E N T R A L VERSION together with the integrated I D M S - D C nucleus is

used in b o t h batch and on-line m o d e s of operation.

7.6.1 Batch Processing

Figure 7.6.1 shows programs A and B running in batch m o d e . If they have a
c o m m o n SUBSCHEMA, then only one copy of the SUBSCHEMA T A B L E S
would be required. A SUBSCHEMA WORK A R E A is present in the CV for each
program. It is about 4KB large and holds C U R R E N C Y i n f o r m a t i o n and c o m m u -
nication block data. An E X T E R N A L - R U N - U N I T - S E R V I C E m o d u l e is avail-
able for each active application and provides c o m m u n i c a t i o n b e t w e e n the appli-
cation program and the required CV functions.

7.6.2 On-line Processing

IDMS supports t w o different m o d e s of on-line operation. ATTACH m o d e (see
Fig. 7.6.2a) allows the user programs to run in the same partition as the CEN-
T R A L VERSION. There are t w o variations with ATTACH m o d e : ATTACH
MONITOR and ATTACH C O N T R O L modes. The Control R o u t i n e acts as the
main task in the latter case, the C o m m u n i c a t i o n s Monitor in the f o r m e r . Using
the Control Routine has the advantage that the abnormal termination of the
C o m m u n i c a t i o n s Monitor does cause the DBMS t o terminate.

It is also possible t o run the DBMS in a separate partition f r o m the Communica-

tions Monitor and its associated user programs (A and B) and this is called EXE-
CUTE Mode (see Fig. 7.6.2b). The situation is similar when operating under
MVS (see Fig. 7.6.2c and 7.6.2d) with the addition that a Packet-Data-Move-

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.6 Concurrent Usage 255

SUPERVISOR

SUBSCHEMA,SUBSCHEHA! IDMSDBRC
WORK AREA|W0RK AREA
A I B J

USER IDMS NUCLEUS USER

PROGRAM PROGRAM
A DMCL • SUBSCHEMA TABLES
TABLES ! (SHARED!

I n ATTACH CONTROL mode the main task is the Control Routine

In ATTACH MONITOR mode the Communications Monitor is the
maintask (the Control Routine is not present )

Fig. 7.6.2a: C E N T R A L V E R S I O N and Communications Monitor in A T T A C H MODE

SUPERVISOR

VJ0AIT DISPATCHER

I O M S I R U S - 1 ÎDMS£RUS T COMMUNICATIONS
. A I B '
MONITOR
SUBSCHEMAÌSUBSCHEMAJ IDMSDBRC
WORK AREA (WORK AREA.
A ' B
USER USER
IDMS NUCLEUS PROGRAM PROGRAM
A
DMCL ~~! SUBSCHEMA TABLES
TABLES ' (SHARED)
I
Fig. 7.6.2b: C E N T R A L V E R S I O N and Communication Monitor in E X E C U T E MODE

ment-Buffer in the Common Storage Area must be made available to each pro-
gram not executing in the same address space as the CENTRAL VERSION. In
ATTACH mode, a buffer is made available to the batch user program ' C \ In
EXECUTE mode, buffers must be made available for each user program.
The CENTRAL VERSION may run with a user protect key for all user programs
using Packet-Data-Movement-Buffers, which protects against storage destruction
outside that region controlled by the CV.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
256 7. IDMS (Integrated Database M a n a g e m e n t S y s t e m )

• ATTACH CONTROL mode - Control Routine is the main task

• ATTACH MONITOR mode - Communications Monitor is t h e m a i n task
• The SVC is not necessary for programs r u n n i n g in the same Region a s the
C E N T R A L V E R S I O N , when u s i n g a SYSCTL lileat run-time

Fig. 7.6.2c: MVS C E N T R A L V E R S I O N and C o m m u n i c a t i o n Monitor in A T T A C H MODE

MVS SUPERVISOR C ^PACKJT-DAJA-OTVM_NT_ByFFE_R A _

A pPACKET-OATA-MCMEMENTÌiUFFER C '

INTERFACE WAIT D I S P A T C H E R
- , —
IDMSERUS • IDMSERUS iIOMSERUS
+ _ = 1
SUBSCHEMA.SUBSCHEMA (SUBSCHEMA
WORK AREA I WORK AREA 'WORK AREA
_A _ _B ' C COMMUNICATIONS
MONITOR
USER IDMSPBRC
PROGRAM
C
IDMS NUCLEUS

I SUBSCHEMA TABLES
I (SHARED BETWEEN A • C I
DMCL
TABLES I SUBSCHEMATABLES
< I B)

Fig. 7 . 6 . 2 d : MVS C E N T R A L V E R S I O N and C o m m u n i c a t i o n Monitor in E X E C U T E MODE

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.7 Integrity Controls and Recovery 257

7.7 Integrity Controls and Recovery

Although IDMS does not perform format checks on user data, it is possible for
the DBA (or user) to write such routines which could then be incorporated into
IDMS via the PROCEDURES option. Structure data, which is physically stored
in the record prefix, is protected simply by never making it available to the RUN
UNIT. Concurrent updating can be prevented by specifying the appropriate
AREA USAGE MODE.
The problem of not supplying obsolete data while one program is updating can
be avoided by specifying EXCLUSIVE UPDATE USAGE MODE for the AREA
and the updating program. This is particularly necessary if a RUN—UNIT reads a
number of records for a processing unit while, during this time, some of them
could be updated from another RUN—UNIT.
Deadly embrace (deadlock) is resolved by aborting and ROLLING BACK the
offending program. The aborted RUN—UNIT is informed via a status code that
the ROLL BACK has occurred. The program may decide to start the same trans-
action again, or a new transaction. In either case, the first command must be
BIND RUN-UNIT.
IDMS may also abort and ROLL BACK a RUN-UNIT after a specified period of
time, it it is waiting to update a record occurrence with a SELECT LOCK held
by another program. Run-time LOCKS have the advantage over PROTECTED
and EXCLUSIVE READY options as they tie up only those records required
and not whole AREAS. This is true for short duration transactions, but less
applicable for batch processing, where serious performance degradation could
occur.

The whole JOURNALING and recovery conception has been changed in Release
5.0. JOURNAL entries are maintained at the record level — previously this was
done at the PAGE level. Entries are made for CONNECT, DISCONNECT,
MODIFY, ERASE and STORE commands.
The JOURNAL BUFFER is written to the JOURNAL file when:
- the buffer is full
- a PAGE containing an updated record occurrence, whose BEFORE IMAGE is
in the JOURNAL BUFFER, is returned to the database
- a RUN UNIT aborts
- a RUN UNIT issues a COMMIT, ROLLBACK or FINISH command.
The JOURNAL file may be either on tape or disc. When a disc file is full, a new
utility (IDMSAJNL) copies the contents to a tape file and also rebuilds the disc
JOURNAL file putting all BEFORE IMAGES of active RECOVERY UNITS in
new blocks at the beginning of the new JOU R N AL file.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
258 7. I D M S (Integrated Database Management System)

With the C E N T R A L V E R S I O N and disc J O U R N A L file, automatic recovery can

be accomplished in one o f t w o ways:

- W A R M R E S T A R T . When the C E N T R A L V E R S I O N is restarted after a failure,

it will recognise the previous failure and then R O L L B A C K all incomplete
RUN UNITS.
- R O L L B A C K . This is the simpler case, where the CV continues processing
other RUN U N I T S while returning a R E C O V E R Y U N I T to the last CHECK-
P O I N T ( R O L L B A C K ) . This can be as a result o f a R O L L B A C K command, or
when a R E C O V E R Y U N I T fails.

7.8 Privacy (Access) Controls

I DMS provides various privacy and security control facilities at the program,
A R E A , SET and record levels.

7.8.1 Program Level

Each R U N - U N I T must invoke a SUBSCHEMA (via the SCHEMA SECTION in
the program). The D B A should be the only person who has the authority and
knowledge to generate SUBSCHEMAS. A program is only allowed to access
those SCHEMA elements ( A R E A S , SETS, records ITEMS) copied into the SUB-
SCHEMA.

7.8.2 A R E A Level
A separate P R I V A C Y LOCK may be allocated to each A R E A specified in the
SUBSCHEMA, for the USAGE MODES:

- RETRIEVAL
- PROTECTED R E T R I E V A L
- EXCLUSIVE R E T R I E V A L
- UPDATE
- EXCLUSIVE U P D A T E
- PROTECTED UPDATE.

EXCLUSIVE USAGE MODE denies access to the specified A R E A to all other

RUN U N I T S until that R U N - U N I T issues a FINISH command. P R O T E C T E D
USAGE MODE allows all other R U N - U N I T S to access the specified A R E A , but
only for retrieval.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.9 Performance 259
7.8.3 SET Level
It is possible to forbid the use of certain DML commands within SET types. This
is done in the SUBSCHEMA with a PRIVACY LOCK. The DML commands are:
- CONNECT
- DISCONNECT
- FIND.

7.8.4 Record Level

The manipulation of each record type can be restricted to certain DML com-
mands.

7.9 Performance
The performance of IDMS Release 5.0 should certainly prove to be better than
previous releases simply by virtue of the multi-threading capability of the IDMS
nucleus. However, the total performance of any DBMS is dependent on many
aspects of the system and on external factors such as how easily it can be learned
and used by EDP personnel.

7.9.1 Growth
An IDMS database can accommodate over 4 billion records, a database being
limited to 8 million PAGES and each PAGE capable of holding a maximum of
255 record. This should pose no threat to almost any EDP environment. But as
with all hierarchical/network- based systems, the initial design of the data struc-
ture is very important. However, utilities are available to help incorporate new
logical relationships and new record types into existing databases. Data indepen-
dence insulates existing application programs from later developments that do
not affect their processing.

7.9.2 Programmer Skill

The concepts within the CODASYL DBMS are unambiguous and the terminol-
ogy is clear if somewhat complicated. Nevertheless, even a moderately competent
programmer would be quite capable of grasping the concepts of logical SETS
and the method of navigating through the database structure with the help of
the various CURRENCY pointers.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
260 7. IDMS (Integrated Database Management System)
The flexibility of IDMS does however leave it open to abuse: a programmer can
write substantially inefficient programs. The power and elegance of IDMS more
than compensates for the complexity, but nevertheless, this gives the DBA an
extra level of responsibility, that is, to guard against inefficient coding. Although
it is not essential for the programmer to be aware of the physical placement of
the records, this knowledge would definitely permit him to use IDMS more effi-
ciently.

7.9.3 Reorganisation
IDMS continuously reorganises the database, which means that it degrades only
very little with time. Space management ensures that the largest possible contig-
uous area of free space is available on each PAGE. This continuous reorganisa-
tion means that immediately after a record has been deleted, the free space is
incorporated into the contiguous free space area for the PAGE, even when this
means shifting records within the PAGE. This is possible because the location of
any record occurrence within a PAGE is noted at the end of that PAGE. Exter-
nal to the PAGE, the pointers only indicate which PAGE contains the desired
record occurrence. This solution has resulted in volatile databases not being
reorganised for months or possible years, even when they contain variable length
records.
Variable length records present no performance problem as long as there is suffi-
cient space within the PAGE to accommodate any expansion. When no more
space is available, IDMS splits the record into a root (which remains in the
original PAGE) and a fragment (stored in some other PAGE in the vicinity). This
process can be repeated infinitely, in theory at least, with pointers linking the
root to the fragments. IDMS tries to reduce the number of fragments to a mini-
mum at every opportunity, even when the record is just retrieved and not up-
dated.

Although data compression facilities are not automatically included in IDMS,

they can be included on a record type basis via a user exit. The record type to be
compressed must be declared as of variable length. The compression routine is
supplied by the vendor. As it is of a general nature, it does not operate on the
separate fields of a record, but views the record as a continuous data string. All
repeating characters and character pairs are suppressed.

7.9.4 Monitoring, Tuning and Utilities

Statistical information about database operations related to each RUN—UNITis
written both to the log (JOURNAL) file and to the CONTROL AREA when a
FINISH (real or implied) command is executed. This information includes:

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.9 Performance 261

— P A G E S read and updated

— which CALC records overflow the home PAGE
— V I A records placed with their OWNER and those placed in overflow
— those records which became C U R R E N T o f R U N - U N I T
— the number o f C A L L S made t o IDMS
— the number o f LOCKS held.
The S E C U R I T Y DUMP, containing space-usage statistics, is used by the DBA to
monitor the database. Neither this nor any other utility is available t o assist the
DBA t o tune the system, which must be done on a hit-or-miss basis. One o f the
most critical points with most systems is the I/O buffer size, and IDMS is no excep-
tion. However, beyond a certain size, the search time will decrease the efficiency.
Each DML retrieval command examines the contents o f the buffer pool before
issuing the necessary I/O command. This search time can become significant.
No matter how good a DBMS might be, it cannot be used to full advantage with-
out an adequate set o f supporting utilities. In this respect, IDMS is well provided
for:
1. The I D M S A J N L facility will unload a disc log file t o an archive magnetic tape.

2. IDMSCALC is accessed via a subroutine and returns to the user program a

suggested PAGE number to store the record whose CALC key was supplied
in the original C A L L together with a target PAGE range.

3. IDMSCLUC is the data directory update utility.

4 . IDMSDBLU assists the user to load a database. The user program which
addresses this utility via a C A L L statement must provide:

— one record occurrence descriptor

— one OWNER descriptor for each S E T in which the object record participates
as an A U T O M A T I C M E M B E R . OWNER descriptors may also be provided for
MANUAL S E T S at the user's or DBA's discretion.

All DMBSs using embedded pointers have more difficulty loading data, and
IDMS is no exception. One particular weakness is apparent with this utility:
violations o f the D U P L I C A T E S NOT A L L O W E D clause cannot be detected in
the first step but will be detected in the second.

5. IDMSDUMP performs any or all o f the following functions:

— creating a space utilisation report, which contains information about space

distribution per record type, space distribution per P A G E , file utilisation, and
PAGE size or sizes.
— dumping to magnetic tape or disc file the database or data directory, either in
their entirety or by A R E A or file.
— punching the D A T A B A S E K E Y S o f logically deleted records, for use by the
I D M S L D E L utility.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
262 7. IDMS (Integrated Database Management System)
6. IDMSINIT is responsible for the following activities:
— initialising the data directory
- initialising or re-initialising all or portions of a database.
- initialising the SYSCTL file, used by the Central Version under OS to pass
parameters
— initialising the disc log file or files.
7. IDMSJFIX processes the tape log file which was in use at the time of a system
crash. It can perform the following activites:
— writing ABRT checkpoints for all RUN—UNITS active at the time of the crash
- printing checkpoints and program statistics for RUN—UNITS on the log file.
8. IDMSLDEL physically erases from the database logically deleted records as
listed in the punched output from the IDMSDUMP utility.
9. IDMSPFIX can perform all or any of the following activites:
- printing a specific PAGE or PAGES in display and/or hexadecimal characters
- printing the contents of a PAGE containing a specific CALC key in both dis-
play and hexadecimal characters
- removing an in-use lock from an AREA
— verifying the contents of a PAGE.
10. IDMSRBCK performs any or all of the following activities using a tape log
file:
— restoring a database by processing the log file in a reverse direction and
applying the BEFORE IMAGES and using the AFTER IMAGES to verify the
contents of the database
- printing the BEFORE and AFTER IMAGES
— printing the checkpoints.
11. IDMSRFWD can perform one or more of the following functions:
— working in a forward direction, copying the AFTER IMAGES from the log
file onto the database
- printing the BEFORE and AFTER IMAGES from the log file
— printing the checkpoints from the log file.
12. IDMSRPTS prints out the contents of the data directory.
13. IDMSRSTR can be used to restore a database (or data directory) in its
entirety or by file or AREA. The database copy created by the utility IDMS-
DUMP is used as input.
14. IDMSRSTU offers the user the possibility of making a limited number of
structural modification. These are:
— adding new data items to a record type
- changing the position of existing data items

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
 2bi
7.10 Conclusion
- deleting existing data items
- expanding the record type prefix to include new SET relationships, new
PRIOR pointers or new OWNER pointers.
- deleting NEXT, PRIOR and OWNER pointers
- changing fixed length records to variable length records and vice-versa.
- executing a new DP-PROCEDURE on existing record occurrences
None of these changes requires the database to be reloaded. This utility cannot
be used on databases containing logically deleted records. When PRIOR or
OWNER pointers are to be added, the utility IDMSPCON must be used addi-
tionally.
15. IDMSUNLD unloads, and, in conjunction with IDMSDBLU, reloads the
database. It cannot be used on databases containing logically deleted records.
16. IDMSXPAG expands the database PAGE size.
The foregoing shows that IDMS is supported by all the necessary utilities.

7.10 Conclusion
One of the more difficult problems when examining a CODASYL-based DBMS is
separating the CODASVL IDMS implementation from the CODASVL specifica-
tion. This report is limited to examining IDMS, which is, coincidentally an imple-
mentation of a subset of the CODASVL recommendations. No comments are
levelled directly at the CODASYL recommendations themselves, but some
observations about IDMS do inevitably relate directly or indirectly to the origins
of this system.
Systems based on embedded pointers are relatively easy to compare, using such
criteria as:
— the system's ability to generate hierarchical/network structures
— how easily the structures can be defined
— how easy it is to change the structures
— the clarity and flexibility of the command layout
— which commands are available and their flexibility
— the flexibility of placing a record within a SET and within the data space.
IDMS is at least as good as its competitors in these areas, and normally superior.
The IDMS implementation of secondary indexing is in some aspects, however,
inferior to that offered by IMS.
The normally precise and clear terminology of IDMS has one particularly glaring
irregularity, the LOCATION MODE clause. Historically, this clause stems from
two separate concepts of IDS/1; namely the Storage Mode and Retrieval Mode.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
264 7. IDMS (Integrated Database Management System)
Unfortunately it has been debased to mean simply Storage Mode and is therefore
a misnomer.
One particular failing in the original CODASYL specification and hence in IDMS
was the explicit exclusion of the capability of allowing an OWNER record
occurrence to participate as a MEMBER record occurrence in its own SET
occurrence. An example of where this type of relationship is necessary is in an
electoral register, where the elected representative is OWN E R of all the voters
(MEMBERS) in an electoral area, but where the elected representative may also
vote. He can therefore be both OWNER and MEMBER. This type of relationship
is not currently offered by the main competing embedded pointer based systems.
It is however offered by SIBAS, another CODASYL-based system available for
IBM hardware. This relationship is often called Involuted or Incestuous.
An advantage of IDMS which should be mentioned is its literature, which is
among the very best offered for any software product. This can, to some extent,
be attributed to the clarity of the CODASYL specifications upon which it is
based. However, Cullinane has expended considerable effort in producing
manuals which reduce to a minimum the extremely high learning requirement
needed before any embedded pointer system can be used competently. Once the
complexity of IDMS is mastered, the user is rewarded with a degree of flexibility
not available from any of the other DBMSs examined, which uses embedded
pointers. This in no way means that the CODASYL concept is flawless; simply
that IDMS offers the user a greater degree of flexibility for a lower hardware
cost than any similar DBMS.

IDMS must now be considered a mature DBMS although some of the supporting
products should be regarded as somewhat provisional. The Query Language, and
the Sequential Processing Facility will certainly be improved extensively in the
near future.
A full range of utilities is offered to cope with all ancillary functions from warm
restart to reorganisation. Warm restart is completely automatic. Any DBMS
evaluation which does not include IDMS cannot be considered complete.

7.11 Advantages of IDMS

— Clear and precise terminology.
— Both hierarchical and network structures can be expressed.
— Best documentation of all the systems examined.
— Good transaction-oriented processing capabilities.
— Commands easy to learn and use.
— Command formats are excellent because of preprocessor.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.12 Disadvantages of IDMS 265
— Extremely good record placement and manipulation commands.
— Direct access for fast response real-time applications.
— Portability.
— On-going DB reorganisation (even on retrieval for variable length records)
largely obviating the need for off-line reorganisation runs.
— Report writer (CULPRIT).
— Data dictionary.
— Interfaces — see Appendix III.
— Different SET types under one OWNER type may have different LOCATION
MODES; so allowing very flexible tuning, with the LOCATION MODE cho-
sen to suit the SET characteristics not the average DB characteristics.
— Logging file may be disc (recommended). It is preformatted so as to avoid
recovery problems.
— I DMS requires relatively few system resources.
— Many utilities can be run on-line.
— It also runs on DEC PDP-11 hardware.
— Automatic warm restart.
— Multithreading nucleus.
— Fully integrated TP monitor.
— Backing storage of 2MB (excluding SPF).
— The 'CALC' record placement can accept unique or duplicate keys as the
design requires.
— Data compression is available via a User-Exit — the routine is supplied by
Cullinane. It treats the record as a string.
— Logging at record level.
— Field level sensitivity for the user-view, but although the order of the fields
may differ, the characteristics must be the same as in the SCHEMA.

7.12 Disadvantages of IDMS

— The secondary indexing is somewhat clumsily implemented. It does not use
inverted file techniques but internally generated SETS.
— Query language is somewhat primitive; no on-line 'ad-hoc' complex query
facility.
— TP Monitor somewhat primitive.
— The uncertain pace of development to reach the level of the 1978 CODASVL
recommendations.
— The learning and design difficulties associated with all complex embedded
pointer systems.
— Dual logging only available via a user exit.
— No security violation audit.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
266 7. IDMS (Integrated Database Management System)
— Intersection data is hard to handle in M:M relationships.
— Data is bound partly at compile time.
— The load program must be written by the user.
— No user-messages may be written to the log file.
— No format checks are made when records are loaded.
— Involuted relationships cannot be expressed directly.
— Data Independence is somewhat limited because the SUBSCHEMA cannot
vary the SCHEMA'S ITEM characteristics.
— The data compression routine, if used, has to expand the whole record irre-
spective of the number of ITEMS requested by the SUBSCHEMA.

7.13 I D M S Glossary

AREA: A group of PAGES within an IDMS database.

AUTOMATIC: This is a rule of SET MEMBERSHIP (cf.

MANUAL) which specifies that a record, on being
entered into the database, will be placed in an
occurrence of each SET type in which it is an
AUTOMATIC MEMBER.

CALC: A method of storing records by using a random-

izing routine on a particular field to produce a
PAGE address. All records which map to a partic-
ular PAGE are chained together in ascending key
sequence.

CURRENCY: Information held internally indicating the most

recently located record of each RUN-UNIT,
AREA, record-type and SET.

DATABASE KEY: Each record in an IDMS database is allocated a

unique relative address, the DATABASE KEY. It
is assigned for the life of the record and consists
of a PAGE number and LINE number in the
PAGE FOOTER where the address of the record
is held. This method is used so that the actual
location of the record may change, and has the
great advantage that it avoids constant reorgani-
sation runs.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.13 IDMS Glossary 267

(DATA) ITEM: This is the CODASYL name for a COBOL field.

Both elementary and group ITEMS may be
defined.

DDL: (DATA DEFINITION LANGUAGE). Used to

define the data storage structure for the
SCHEMAS and SUBSCHEMAS.

DIRECT: This LOCATION MODE gives the user the

greatest degree of control over the placement of
the records. The user specifies the DATABASE
KEY, ie. the PAGE and LINE numbers where the
record is to be stored. If this key is already in use,
then I DMS will allocate the next available DATA-
BASE KEY in the specified AREA.

DMCL: (DATA MEDIA CONTROL LANGUAGE). This

language maps the database onto physical storage,
defines the names and sizes of the I/O buffer
pools, and names those files whose changes are to
be logged.

DML: (DATA MANIPULATION LANGUAGE). This

language, similar in format to COBOL statements,
is used to insert, delete and modify the data in
the databases.

LINE NUMBER: This is an index to a series of 8-byte addresses

held at the end of each PAGE referring to all
records stored on the PAGE. Each PAGE can hold
a maximum 255 LINE NUMBERSie. RECORDS.

LOCATION MODE: This term means 'storage mode'. There are three
methods of storing — and hence retrieving — a
record: CALC, DIRECT or VIA. This storage
mode must be specified for every record type.

MANDATORY: This is a SET MEMBERSHIP rule (cf. OPTION-

AL), which specifies that once a record occur-
rence has become a MEMBER of a SET, it must
remain a MEMBER of that SET for as long as it
remains in the database.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
268 7. IDMS (Integrated Database Management System)

MANUAL: This is a SET MEMBERSHIP rule, (cf. AUTO-

MATIC), which specifies that a record occur-
rence only becomes a MEMBER of those SET
occurrences in which it is explicitly placed.

MEMBER: A record occurrence together with more record

occurrences of the same or of different types
dependent on a particular OWNER record occur-
rence.

OPTIONAL: This is a SET MEMBERSHIP rule (cf. MANDA-

TORY) which specifies that a record occurrence
may be removed from a particular SET occur-
rence without affecting its MEMBERSHIP in any
other SET occurrence.

OWNER: This is the record type which occurs only once in

each SET occurrence and on which all SET
occurrence MEMBERS are dependent.

PAGE: This is the CODASVL name for a physical block.

RECORD: Each record occurrence consists of a prefix con-

taining SET information and user information,
which are stored contiguously. The user never
sees the prefix.

RUN-UNIT: This is an executing instance of a user program.

SCHEMA: This is the description of the structure of a data-

base which includes all assoiated AREAS, records
and SETS. The definition is effected using the
DDL.

SET: A SET is the defined relationship between two or

more record types. A SET occurrence consists of
one, and only one, occurrence of the OWNER
record type and one or more occurrence of the
MEMBER record types.

SUBSCHEMA: This is a subset of the total user view as defined

in the SCHEMA. It is used by a R U N - U N I T to

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
7.13 IDMS Glossary 269

supply the necessary view of the database. The

DDL contains facilities for defining the SUB-
SCHEMA.

VIA: This LOCATION MODE causes record occur-

rences to be placed as close as possible physically
to their OWNER occurrence.

Brought to you by | The University of Texas at Dallas (UTDALLAS)

Authenticated
Download Date | 10/26/19 12:14 PM
Brought to you by | The University of Texas at Dallas (UTDALLAS)
Authenticated
Download Date | 10/26/19 12:14 PM