Visit https://ebookultra.

com to download the full version and

explore more ebooks

C network programming Vol 2 Systematic reuse

with ACE and frameworks 6. print Edition Schmidt

_____ Click the link below to download _____

https://ebookultra.com/download/c-network-programming-
vol-2-systematic-reuse-with-ace-and-frameworks-6-print-
edition-schmidt/

Explore and download more ebooks at ebookultra.com

Here are some suggested products you might be interested in.
Click the link to download

PHP Programming with PEAR 1st Edition Schmidt Stephan

https://ebookultra.com/download/php-programming-with-pear-1st-edition-
schmidt-stephan/

Programming the Network with Perl 1st Edition Paul Barry

https://ebookultra.com/download/programming-the-network-with-perl-1st-
edition-paul-barry/

Systematic Instruction for Students with Moderate and

Severe Disabilities Belva C. Collins

https://ebookultra.com/download/systematic-instruction-for-students-
with-moderate-and-severe-disabilities-belva-c-collins/

Programming in C 2 e Second Edition Dey

https://ebookultra.com/download/programming-in-c-2-e-second-edition-
dey/
STL Tutorial and Reference Guide C Programming With the
Standard Template Library 2 (Draft) Edition David R.
Musser
https://ebookultra.com/download/stl-tutorial-and-reference-guide-c-
programming-with-the-standard-template-library-2-draft-edition-david-
r-musser/

Programming with C Zambak 1st Edition Osman Ay

https://ebookultra.com/download/programming-with-c-zambak-1st-edition-
osman-ay/

Programming Social Applications Building Viral Experiences

with OpenSocial OAuth OpenID and Distributed Web
Frameworks 1st Edition Jonathan Leblanc
https://ebookultra.com/download/programming-social-applications-
building-viral-experiences-with-opensocial-oauth-openid-and-
distributed-web-frameworks-1st-edition-jonathan-leblanc/

Rapid French Vol 2 with Audio Second Edition Earworms

https://ebookultra.com/download/rapid-french-vol-2-with-audio-second-
edition-earworms/

Programming with Visual C Concepts and Projects 1st

Edition James Allert

https://ebookultra.com/download/programming-with-visual-c-concepts-
and-projects-1st-edition-james-allert/
C network programming Vol 2 Systematic reuse with
ACE and frameworks 6. print Edition Schmidt Digital
Instant Download
Author(s): Schmidt, Douglas C;Huston, Stephen D
ISBN(s): 9780201795257, 0201795256
Edition: 6. print
File Details: PDF, 2.08 MB
Year: 2002
Language: english
C++ Network Programming
Volume 2
The C++ In-Depth Series
Bjarne Stroustrup, Editor

“I have made this letter longer than usual, because I lack the time to make it short.”
—BLAISE PASCAL

T he advent of the ISO/ANSI C++ standard marked the beginning of a new era for C++
programmers. The standard offers many new facilities and opportunities, but how can a
real-world programmer find the time to discover the key nuggets of wisdom within this
mass of information? The C++ In-Depth Series minimizes learning time and confusion by
giving programmers concise, focused guides to specific topics.
Each book in this series presents a single topic, at a technical level appropriate to that
topic. The Series’ practical approach is designed to lift professionals to their next level
of programming skills. Written by experts in the field, these short, in-depth monographs
can be read and referenced without the distraction of unrelated material. The books are
cross-referenced within the Series, and also reference The C++ Programming Language by
Bjarne Stroustrup.
As you develop your skills in C++, it becomes increasingly important to separate essential
information from hype and glitz, and to find the in-depth content you need in order to grow.
The C++ In-Depth Series provides the tools, concepts, techniques, and new approaches to
C++ that will give you a critical edge.

Titles in the Series

Accelerated C++: Practical Programming by Example, Andrew Koenig and Barbara E. Moo
Applied C++: Practical Techniques for Building Better Software, Philip Romanik and Amy Muntz
The Boost Graph Library: User Guide and Reference Manual, Jeremy G. Siek, Lie-Quan Lee, and Andrew
Lumsdaine
C++ Coding Standards: 101 Rules, Guidelines, and Best Practices, Herb Sutter and Andrei Alexandrescu
C++ In-Depth Box Set, Bjarne Stroustrup, Andrei Alexandrescu, Andrew Koenig, Barbara E. Moo,
Stanley B. Lippman, and Herb Sutter
C++ Network Programming, Volume 1: Mastering Complexity with ACE and Patterns, Douglas C. Schmidt
and Stephen D. Huston
C++ Network Programming, Volume 2: Systematic Reuse with ACE and Frameworks, Douglas C. Schmidt
and Stephen D. Huston
C++ Template Metaprogramming: Concepts, Tools, and Techniques from Boost and Beyond, David Abrahams
and Aleksey Gurtovoy
Essential C++, Stanley B. Lippman
Exceptional C++: 47 Engineering Puzzles, Programming Problems, and Solutions, Herb Sutter
Exceptional C++ Style: 40 New Engineering Puzzles, Programming Problems, and Solutions, Herb Sutter
Modern C++ Design: Generic Programming and Design Patterns Applied, Andrei Alexandrescu
More Exceptional C++: 40 New Engineering Puzzles, Programming Problems, and Solutions, Herb Sutter

For more information, check out the series web site at www.awprofessional.com/series/indepth/
 C++ Network Programming
Volume 2

Systematic Reuse with ACE and Frameworks

Douglas C. Schmidt
Stephen D. Huston

Boston • San Francisco • New York • Toronto • Montreal

London • Munich • Paris • Madrid
Capetown • Sydney • Tokyo • Singapore • Mexico City
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where
those designations appear in this book, and Addison-Wesley was aware of a trademark claim, the designations have been
printed with initial capital letters or in all capitals.

The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any
kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages
in connection with or arising out of the use of the information or programs contained herein.

The publisher offers discounts on this book when ordered in quantity for bulk purchases and special sales. For more
information, please contact:

U.S. Corporate and Government Sales

(800) 382-3419
[email protected]

For sales outside of the U.S., please contact:

International Sales
(317) 581-3793
[email protected]

Visit Addison-Wesley on the Web: www.awprofessional.com

Library of Congress Cataloging-in-Publication Data

Schmidt, Douglas C.
C++ network programming / Douglas C. Schmidt, Stephen D. Huston.
p. cm.
Includes bibliographical references and index.
Contents: Vol. 2. Systematic reuse with ACE and frameworks.
ISBN 0-201-79525-6 (v. 2 : pbk.)
1. C++ (Computer program language) 2. Object-oriented programming (Computer
science) 3. Computer networks. I. Huston, Stephen D. II. Title.

QA 76.73.C153 S368 2002

005.2’762—dc21
2001053345
Copyright 
c 2003 by Pearson Education, Inc.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form,
or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher.
Printed in the United States of America. Published simultaneously in Canada.

For information on obtaining permission for use of material from this work, please submit a written request to:

Pearson Education, Inc.

Rights and Contracts Department
75 Arlington Street, Suite 300
Boston, MA 02116
Fax: (617) 848-7047

ISBN: 0-201-79525-6
2 3 4 5 6 7 8 9 10— O P M—06050403
Second printing, February 2005
Contents

Foreword vii

About This Book xi

Chapter 1 Object-Oriented Frameworks for Network Programming 1

1.1 An Overview of Object-Oriented Frameworks 1
1.2 Comparing Software Development and Reuse Techniques 4
1.3 Applying Frameworks to Network Programming 12
1.4 A Tour through the ACE Frameworks 14
1.5 Example: A Networked Logging Service 19
1.6 Summary 21

Chapter 2 Service and Configuration Design Dimensions 23

2.1 Service and Server Design Dimensions 24
2.2 Configuration Design Dimensions 34
2.3 Summary 38

Chapter 3 The ACE Reactor Framework 39

3.1 Overview 39
3.2 The ACE Time Value Class 42
3.3 The ACE Event Handler Class 46
3.4 The ACE Timer Queue Classes 61
3.5 The ACE Reactor Class 70
3.6 Summary 86

Chapter 4 ACE Reactor Implementations 87

4.1 Overview 87
4.2 The ACE Select Reactor Class 89
4.3 The ACE TP Reactor Class 99

v
vi CONTENTS

4.4 The ACE WFMO Reactor Class 103

4.5 Summary 113

Chapter 5 The ACE Service Configurator Framework 115

5.1 Overview 115
5.2 The ACE Service Object Class 118
5.3 The ACE Service Repository Classes 126
5.4 The ACE Service Config Class 138
5.5 Summary 154

Chapter 6 The ACE Task Framework 155

6.1 Overview 155
6.2 The ACE Message Queue Class 157
6.3 The ACE Task Class 183
6.4 Summary 202

Chapter 7 The ACE Acceptor-Connector Framework 203

7.1 Overview 203
7.2 The ACE Svc Handler Class 206
7.3 The ACE Acceptor Class 216
7.4 The ACE Connector Class 229
7.5 Summary 256

Chapter 8 The ACE Proactor Framework 257

8.1 Overview 257
8.2 The Asynchronous I/O Factory Classes 261
8.3 The ACE Handler Class 270
8.4 The Proactive Acceptor-Connector Classes 278
8.5 The ACE Proactor Class 286
8.6 Summary 296

Chapter 9 The ACE Streams Framework 297

9.1 Overview 297
9.2 The ACE Module Class 299
9.3 The ACE Stream Class 314
9.4 Summary 318

Glossary 319

Bibliography 329

Index 337
Foreword

The ADAPTIVE Communication Environment (ACE) toolkit has achieved enormous suc-
cess in the area of middleware for networked computing. Due to its flexibility, perfor-
mance, platform coverage, and other key properties, ACE enjoys broad acceptance by the
networked application software community, as evidenced by its use in thousands of applica-
tions, in scores of countries, and in dozens of domains. ACE has also received considerable
attention beyond the middleware community since it’s an open-source role model for high-
quality and well-designed pattern-oriented software architectures.
But why is ACE so successful? Addressing this question properly takes some thought.
To start off, let’s reconsider the Foreword from C++ Network Programming: Mastering
Complexity with ACE and Patterns (C++NPv1) and resume the mass transit analogy pre-
sented there by my colleague Steve Vinoski. Steve’s right that a high-quality mass transit
system consists of more than just aircraft, airports, trains, train stations, and rails. It also
needs less obvious infrastructure, such as scheduling, routing, ticketing, maintenance, and
monitoring. But even a complete collection of ingredients is still not sufficient to develop
an effective mass transit system. Arranging these ingredients so they seamlessly fulfill their
primary objective—fast and reliable transportation of people—is equally important. Would
you use a mass transit system whose ticketing was located in a train maintenance location
or an airport hangar, or whose planned and actual scheduling and routing weren’t available
to the public? I doubt it!
The success of mass transit systems depends on more than the knowledge of the infra-
structure parts that are provided—it depends on how these different parts must be connected
and integrated with their environment. This knowledge enables architects of mass transit
systems to integrate individual parts into higher-level building blocks and to connect these
building blocks effectively. For example, ticketing, information points, baggage offices, and
boarding are integrated in train stations located at city centers or major suburban centers.
Likewise, airports are often located near large cities and connected by frequent express
trains.

vii
viii Foreword

Even mass transit centers themselves are arranged so that activities can be performed
effectively. For example, when you enter a train station or airport via the main entrance, you
find ticket agents, information centers, and timetables. You also find shops to satisfy your
travel needs. As you enter the main train hall or airport concourse, you find other informa-
tion centers, up-to-date scheduling information, and the platforms and gates for boarding
the trains and planes. Mass transit centers thus not only provide all necessary services
to begin and end a journey, they also organize their internal “control flows” effectively.
While the core structures and control flows in most train stations and airports are similar,
their concrete realization can differ widely. Yet we all recognize these mass transit center
patterns immediately since they follow key invariants that we’ve learned through years of
experience.
So what’s the connection between successful mass transit system design and the success
of ACE? The answer is simple: In addition to the basic network computing ingredients (the
wrapper facades that Doug and Steve introduced in C++NPv1), ACE also includes use-
ful object-oriented frameworks that build upon these wrapper facades and provide useful
higher-level communication services, such as event demultiplexing and dispatching, con-
nection management, service configuration, concurrency, and hierarchically layered stream
processing. The ACE framework services satisfy many networked software needs by or-
ganizing the structures and internal control flows of your applications effectively via key
patterns learned through years of experience.
The ACE frameworks offer you a number of important benefits:

• You needn’t develop the capabilities provided by ACE, which will save considerable
time and effort. You can therefore focus on your key responsibility: implementing
the application functionality required by your customers and end users.
• The ACE frameworks reify the extensive network programming expertise that Doug,
Steve, and their colleagues have gained over several decades. In particular, the ACE
frameworks efficiently implement the canonical classes, class relationships, and con-
trol flows common to networked applications. The ACE frameworks are tested reg-
ularly by thousands of users from around the world, which has yielded many useful
corrections and improvements. As an ACE user, you can directly leverage the cor-
rectness, effectiveness, and efficiency of the ACE frameworks in your applications.
• A framework isn’t a framework if it can’t be adapted to specific user needs. This
means you can adapt the ACE frameworks at key points of variation in networked
applications. For example, the ACE Reactor framework can be adapted to use dif-
ferent event demultiplexer functions, such as WaitForMultipleObjects() or
select(). Likewise, the ACE Acceptor-Connector framework can be configured
with different IPC mechanisms. While this adaptability is beneficial by itself, ACE
goes a step further: for many adaptations you can configure the desired strategies
from available and interchangeable implementations. In addition to the different Re-
Foreword ix

actor implementations mentioned above, for instance, ACE provides wrapper facades
for various IPC mechanisms, such as the Sockets, SSL, TLI, and shared memory, that
help to configure the ACE Acceptor-Connector framework for specific platforms and
applications.
• Last but not least, the ACE frameworks don’t exist in isolation. You can therefore
combine them in novel ways to create networked applications and entirely new types
of middleware. For example, you can integrate the Reactor framework with the
Acceptor-Connector framework to separate connection establishment from service
processing functionality in event-driven applications. You can likewise introduce
various forms of concurrency into your applications using the ACE Task framework.

As a result of advising and leading many software projects over the years, I’ve found
that ACE greatly simplifies the task of employing reusable middleware that can be cus-
tomized readily to meet the needs of networked applications. Not all networked applica-
tions need heavyweight middleware, such as application servers, web services, and complex
component models. Yet most networked applications can benefit from portable and efficient
host infrastructure middleware like ACE. This flexibility is the core of ACE’s success since
you needn’t commit to an entire middleware suite if you don’t use all of it. Instead, you
can combine just the essential ACE middleware classes you need to compose applications
that are small, but as powerful as necessary. For this reason, I predict that ACE will still be
widely used long after the influence of today’s heavyweight middleware has waned.
ACE’s tremendous flexibility also needn’t lead to a sea of incompatible middleware
implementations. For example, if you build an embedded system that speaks the CORBA
Internet inter-ORB protocol (IIOP) to the outside world, you can use The ACE ORB (TAO),
which is a CORBA-compliant, open-source, real-time object request broker (ORB) built
using the ACE wrapper facades and frameworks. If CORBA is overkill for your application
needs, however, you can build custom, yet interoperable, middleware using the appropriate
ACE classes. Both solutions can be based on the same core structures and protocols, such
as the ACE Common Data Representation (CDR) classes and its TCP/IP Socket wrapper
facades. They can therefore communicate seamlessly with one another, just as you can
take a train from Paris to Istanbul—the famous Orient Express—and travel through many
European countries without having to change trains due to incompatible railroad networks.
As Steve Vinoski and I have pointed out, there are many similarities between high-
quality mass transit systems and high-quality networking middleware. To me and thousands
of other C++ developers around the world, ACE is the toolkit for building the latter! Af-
ter saying so many good things about ACE, however, let’s return to the main intent of this
foreword: introducing the second volume (C++NPv2) of the C++ Network Programming
series. As with all software technologies and middleware, the more you understand your
tools, the better you’ll be able to apply them. It turns out that using ACE in your appli-
cations is just one aspect of improving your networked software. To benefit significantly
x Foreword

from ACE’s many advantages, you therefore also need a sound understanding of the core
concepts, patterns, and usage rules that underlie its powerful frameworks.
For years, a common way to learn ACE involved studying its code, comments, and ex-
ample applications. Clearly, this process was time consuming and error prone. Moreover,
even after managing to read the several hundred thousand lines of C++ code in ACE, it was
easy to miss the forest for the trees. As the Greek philosopher Thucydides noted two mil-
lennia ago: “A man who has the knowledge but lacks the power to clearly express himself
is no better off than if he had never any idea at all.”
We’re therefore fortunate that Doug and Steve found time in their busy schedules to
create such a high-quality book on the ACE frameworks. C++NPv2 explains the ideas
and concepts underlying the ACE frameworks in an easily accessible form using the popu-
lar concurrency and networking patterns from the POSA [POSA1, POSA2] and “Gang of
Four” [GoF] patterns books. These patterns, in turn, reify thoughtful and time-proven solu-
tions to common networking problems. For example, they tell you what the problems are,
why these problems are hard, what the solutions to these problems are, and why these solu-
tions applied to ACE are of high quality. If you want thorough coverage of the patterns and
frameworks in ACE that are shaping the next generation of networked application software
then read this book. I’ve learned much from it and I’m sure you will too.

Frank Buschmann
Senior Principal Engineer
Siemens Corporate Technology
Munich, Germany
About This Book

Software for networked applications must possess the following qualities to be successful
in today’s competitive, fast-paced computing industry:

• Affordability, to ensure that the total ownership costs of software acquisition and
evolution are not prohibitively high
• Extensibility, to support successions of quick updates and additions to address new
requirements and take advantage of emerging markets
• Flexibility, to support a growing range of multimedia data types, traffic patterns, and
end-to-end quality of service (QoS) requirements
• Portability, to reduce the effort required to support applications on heterogeneous
OS platforms and compilers
• Predictability and efficiency, to provide low latency to delay-sensitive real-time ap-
plications, high performance to bandwidth-intensive applications, and usability over
low-bandwidth networks, such as wireless links
• Reliability, to ensure that applications are robust, fault tolerant, and highly available
• Scalability, to enable applications to handle large numbers of clients simultaneously

Writing high-quality networked applications that exhibit these qualities is hard—it’s

expensive, complicated, and error prone. The patterns, C++ language features, and object-
oriented design principles presented in C++ Network Programming, Volume 1: Mastering
Complexity with ACE and Patterns (C++NPv1) help to minimize complexity and mistakes
in networked applications by refactoring common structure and functionality into reusable
wrapper facade class libraries. The key benefits of reuse will be lost, however, if large
parts of the application software that uses these class libraries—or worse, the class libraries
themselves—must be rewritten for each new project.

xi
xii About This Book

Historically, many networked application software projects began by

1. Designing and implementing demultiplexing and dispatching infrastructure mecha-
nisms that handle timed events and I/O on multiple socket handles
2. Adding service instantiation and processing mechanisms atop the demultiplexing and
dispatching layer, along with message buffering and queueing mechanisms
3. Implementing large amounts of application-specific code using this ad hoc host in-
frastructure middleware
This development process has been applied many times in many companies, by many
projects in parallel. Even worse, it’s been applied by the same teams in a series of projects.
Regrettably, this continuous rediscovery and reinvention of core concepts and code has kept
costs unnecessarily high throughout the software development life cycle. This problem is
exacerbated by the inherent diversity of today’s hardware, operating systems, compilers,
and communication platforms, which keep shifting the foundations of networked applica-
tion software development.
Object-oriented frameworks [FJS99b, FJS99a] are one of the most flexible and pow-
erful techniques that address the problems outlined above. A framework is a reusable,
“semi-complete” application that can be specialized to produce custom applications [JF88].
Frameworks help to reduce the cost and improve the quality of networked applications by
reifying proven software designs and patterns into concrete source code. By emphasiz-
ing the integration and collaboration of application-specific and application-independent
classes, frameworks enable larger scale reuse of software than can be achieved by reusing
individual classes or stand-alone functions.
In the early 1990s, Doug Schmidt started the open-source ACE project to bring the
power and efficiency of patterns and frameworks to networked application development.
As with much of Doug’s work, ACE addressed many real-world problems faced by pro-
fessional software developers. Over the following decade, his groups at the University of
California, Irvine; Washington University, St. Louis; and Vanderbilt University, along with
contributions from the ACE user community and Steve Huston at Riverace, yielded a C++
toolkit containing some of the most powerful and widely used concurrent object-oriented
network programming frameworks in the world. By applying reusable software patterns and
a lightweight OS portability layer, the frameworks in the ACE toolkit provide synchronous
and asynchronous event processing; concurrency and synchronization; connection manage-
ment; and service configuration, initialization, and hierarchical integration.
The success of ACE has fundamentally altered the way that networked applications
and middleware are designed and implemented on the many operating systems outlined in
Sidebar 2 (page 16). ACE is being used by thousands of development teams, ranging from
large Fortune 500 companies to small startups to advanced research projects at universities
and industry labs. Its open-source development model and self-supporting culture is similar
in spirit and enthusiasm to that driving Linus Torvalds’s popular Linux operating system.
About This Book xiii

This book describes how the ACE frameworks are designed and how they can help
developers navigate between the limitations of
1. Low-level native operating system APIs, which are inflexible and nonportable
2. High-level middleware, such as distribution middleware and common middleware
services, which often lacks the efficiency and flexibility to support networked appli-
cations with stringent QoS and portability requirements

The skills required to produce and use networked application frameworks have traditionally
been locked in the heads of expert developers or buried deep within the source code of
numerous projects that are spread throughout an enterprise or an industry. Neither of these
locations is ideal, of course, since it’s time consuming and error prone to reengineer this
knowledge for each new application or project. To address this problem, this book illustrates
the key patterns [POSA2, POSA1, GoF] that underlie the structure and functionality of the
ACE frameworks. Our coverage of these patterns also makes it easier to understand the
design, implementation, and effective use of the open-source ACE toolkit itself.

Intended Audience
This book is intended for “hands on” C++ developers or advanced students interested in
understanding how to design object-oriented frameworks and apply them to develop net-
worked applications. It builds upon material from C++NPv1 that shows how developers
can apply patterns to master complexities arising from using native OS APIs to program
networked applications. It’s therefore important to have a solid grasp of the following top-
ics covered in C++NPv1 before reading this book:
• Networked application design dimensions, including the alternative communica-
tion protocols and data transfer mechanisms discussed in Chapter 1 of C++NPv1
• Internet programming mechanisms, such as TCP / IP connection management and
data transfer APIs [Ste98] discussed in Chapter 2 of C++NPv1
• Concurrency design dimensions, including the use of processes and threads, iter-
ative versus concurrent versus reactive servers, and threading models [Ste99] dis-
cussed in Chapters 5 through 9 of C++NPv1
• Synchronization techniques necessary to coordinate the interactions of processes
and threads on various OS platforms [KSS96, Lew95, Ric97] discussed in Chapter
10 of C++NPv1
• Object-oriented design and programming techniques [Boo94, Mey97] that can
simplify OS APIs and avoid programming mistakes through the use of patterns, such
as Wrapper Facade [POSA2] and Proxy [POSA1, GoF] discussed in Chapter 3 and
Appendix A of C++NPv1
xiv About This Book

The ACE frameworks are highly flexible and powerful, due in large part to their use of
C++ language features [Bja00]. You should therefore be familiar with C++ class inheritance
and virtual functions (dynamic binding) as well as templates (parameterized types) and
the mechanisms your compiler(s) offer to instantiate them. ACE provides a great deal of
assistance in overcoming differences between C++ compilers. As always, however, you
need to know the capabilities of your development tools and how to use them. Knowing
your tools makes it easier to follow the source code examples in this book and to build and
run them on your systems. Finally, as you read the examples in this book, keep in mind the
points noted in Sidebar 7 (page 46) regarding UML diagrams and C++ code.

Structure and Content

Our C++NPv1 book addressed how to master certain complexities of developing networked
applications, focusing on the use of ACE’s wrapper facades to avoid problems with oper-
ating system APIs written in C. This book (which we call C++NPv2) elevates our focus
to motivate and demystify the patterns, design techniques, and C++ features associated
with developing and using the ACE frameworks. These frameworks help reduce the cost
and improve the quality of networked applications by reifying proven software designs and
patterns into frameworks that can be reused systematically across projects and enterprises.
The ACE frameworks expand reuse technology far beyond what can be achieved by reusing
individual classes or even class libraries.
This book presents numerous C++ applications to reinforce the design discussions by
showing concrete examples of how to use the ACE frameworks. These examples provide
step-by-step guidance that can help you apply key object-oriented techniques and patterns to
your own networked applications. The book also shows how to enhance your design skills,
focusing on the key concepts and principles that shape the design of successful object-
oriented frameworks for networked applications and middleware.
The chapters in the book are organized as follows:
• Chapter 1 introduces the concept of an object-oriented framework and shows how
frameworks differ from other reuse techniques, such as class libraries, components,
patterns, and model-integrated computing. We then outline the frameworks in the
ACE toolkit that are covered in subsequent chapters.
• Chapter 2 completes the domain analysis begun in C++NPv1, which covered the
communication protocols and mechanisms, and the concurrency architectures used
by networked applications. The focus in this book is on the service and configuration
design dimensions that address key networked application properties, such as dura-
tion and structure, how networked services are identified, and the time at which they
are bound together to form complete applications.
About This Book xv

• Chapter 3 describes the design and use of the ACE Reactor framework, which imple-
ments the Reactor pattern [POSA2] to allow event-driven applications to demultiplex
and dispatch service requests that are delivered to an application from one or more
clients.
• Chapter 4 then describes the design and use of the most common implementations
of the ACE_Reactor interface, which support a wide range of OS event demul-
tiplexing mechanisms, including select(), WaitForMultipleObjects(),
XtAppMainLoop(), and /dev/poll.
• Chapter 5 describes the design and use of the ACE Service Configurator framework.
This framework implements the Component Configurator pattern [POSA2] to allow
an application to link/unlink its component service implementations at run time with-
out having to modify, recompile, or relink the application statically.
• Chapter 6 describes the design and effective use of the ACE Task framework. This
framework can be used to implement key concurrency patterns, such as Active Object
and Half-Sync/Half-Async [POSA2].
• Chapter 7 describes the design and effective use of the ACE Acceptor-Connector
framework. This framework implements the Acceptor-Connector pattern [POSA2] to
decouple the connection and initialization of cooperating peer services in a networked
system from the processing they perform once connected and initialized.
• Chapter 8 describes the design and use of the ACE Proactor framework. This frame-
work implements the Proactor and Acceptor-Connector patterns [POSA2] to allow
event-driven applications to efficiently demultiplex and dispatch service requests trig-
gered by the completion of asynchronously initiated operations.
• Chapter 9 describes the design and use of the ACE Streams framework. This frame-
work implements the Pipes and Filters pattern [POSA1] to provide a structure for
systems that process streams of data.
• The book concludes with a glossary of technical terms, a list of references for further
study, and a general subject index.
The chapters are organized to build upon each other and to minimize forward references.
We therefore recommend that you read the chapters in order.
Although this book illustrates the key capabilities of ACE’s most important frameworks,
we don’t cover all uses and methods of those frameworks. For additional coverage of ACE,
we refer you to The ACE Programmer’s Guide [HJS] and the online ACE reference doc-
umentation, generated by Doxygen [Dim01]. ACE’s reference documentation is available
at http://ace.ece.uci.edu/Doxygen/ and http://www.riverace.com/
docs/.
xvi About This Book

Related Material
This book is based on ACE version 5.3, released in the fall of 2002. ACE 5.3 and all the
sample applications described in our books are open-source software. Sidebar 3 (page 19)
explains how you can obtain a copy of ACE so you can follow along, see the actual ACE
classes and frameworks in complete detail, and run the code examples interactively as you
read the book.
To learn more about ACE, or to report errors you find in the book, we recommend you
subscribe to the ACE mailing list, [email protected]. You can subscribe by
sending a request to [email protected]. Include the following
command in the body of the e-mail (the subject is ignored):
subscribe ace-users [emailaddress@domain]

You must supply emailaddress@domain only if your message’s From address is not
the address you wish to subscribe. If you use this alternate address method, the list server
will require an extra authorization step before allowing you to join the list.
Postings to the ace-users list are also forwarded to the comp.soft-sys.ace
USENET newsgroup, along with postings to several other ACE-related mailing lists. Read-
ing the messages via the newsgroup is a good way to keep up with ACE news and activity
if you don’t require immediate delivery of the 30 to 50 messages that are posted daily on
the mailing lists.
Archives of postings to the comp.soft-sys.ace newsgroup are available at http:
//groups.google.com/. Enter comp.soft-sys.ace in the search box to go to
a list of archived messages. Google has a complete, searchable archive of over 40,000
messages. You can also post a message to the newsgroup from Google’s site.

Acknowledgments
Champion reviewing honors go to Alain Decamps, Don Hinton, Alexander Maack, Chris
Uzdavinis, and Johnny Willemsen, who reviewed the book multiple times and provided ex-
tensive, detailed comments that improved its form and content substantially. Many thanks
also to the official reviewers, Timothy Culp, Dennis Mancl, Phil Mesnier, and Jason Pa-
sion, who read the entire book and gave us many helpful comments. Many other ACE
users provided feedback on this book, including Marc M. Adkins, Tomer Amiaz, Vi Thuan
Banh, Kevin Bailey, Stephane Bastien, John Dilley, Eric Eide, Andrew Finnell, Dave Find-
lay, Jody Hagins, Jon Harnish, Jim Havlicek, Martin Johnson, Christopher Kohlhoff, Alex
Libman, Harald Mitterhofer, Llori Patterson, Nick Pratt, Dieter Quehl, Tim Rozmajzl, Irma
Rastegayeva, Eamonn Saunders, Harvinder Sawhney, Christian Schuhegger, Michael Sear-
les, Kalvinder Singh, Henny Sipma, Stephen Sturtevant, Leo Stutzmann, Tommy Svensson,
Bruce Trask, Dominic Williams, and Vadim Zaliva.
About This Book xvii

We are deeply indebted to all the members, past and present, of the DOC groups at
Washington University in St. Louis and the University of California, Irvine, as well as the
team members at Riverace Corporation and Object Computing Inc., who developed, refined,
and optimized many of the ACE capabilities presented in this book. This group includes Ev-
erett Anderson, Alex Arulanthu, Shawn Atkins, John Aughey, Luther Baker, Jaiganesh Bal-
asubramanian, Darrell Brunsch, Don Busch, Chris Cleeland, Angelo Corsaro, Chad Elliot,
Sergio Flores-Gaitan, Chris Gill, Pradeep Gore, Andy Gokhale, Priyanka Gontla, Myrna
Harbibson, Tim Harrison, Shawn Hannan, John Heitmann, Joe Hoffert, James Hu, Frank
Hunleth, Prashant Jain, Vishal Kachroo, Ray Klefstad, Kitty Krishnakumar, Yamuna Krish-
namurthy, Michael Kircher, Fred Kuhns, David Levine, Chanaka Liyanaarachchi, Michael
Moran, Ebrahim Moshiri, Sumedh Mungee, Bala Natarajan, Ossama Othman, Jeff Parsons,
Kirthika Parameswaran, Krish Pathayapura, Irfan Pyarali, Sumita Rao, Carlos O’Ryan,
Rich Siebel, Malcolm Spence, Marina Spivak, Naga Surendran, Steve Totten, Bruce Trask,
Nanbor Wang, and Seth Widoff.
We also want to thank the thousands of C++ developers from over 50 countries who’ve
contributed to ACE for over a decade. ACE’s excellence and success is a testament to the
skills and generosity of many talented developers and the forward-looking companies that
had the vision to contribute their work to ACE’s open-source code base. Without their
support, constant feedback, and encouragement, we would never have written this book.
In recognition of the efforts of the ACE open-source community, we maintain a list of all
contributors at http://ace.ece.uci.edu/ACE-members.html.
We are also grateful for the support from colleagues and sponsors of our research
on patterns and development of the ACE toolkit, notably the contributions of Ron Akers
(Motorola), Steve Bachinsky (SAIC), John Bay (DARPA), Detlef Becker (Siemens), Frank
Buschmann (Siemens), Dave Busigo (DARPA), John Buttitto (Sun), Becky Callison (Boe-
ing), Wei Chiang (Nokia Inc.), Joe Cross (Lockheed Martin), Lou DiPalma (Raytheon),
Bryan Doerr (Savvis), Karlheinz Dorn (Siemens), Scott Ellard (Madison), Matt Emerson
(Escient Convergence Group, Inc.), Sylvester Fernandez (Lockheed Martin), Nikki Ford
(DARPA), Andreas Geisler (Siemens), Helen Gill (NSF), Inc.), Jody Hagins (ATD), Andy
Harvey (Cisco), Sue Kelly (Sandia National Labs), Gary Koob (DARPA), Petri Koske-
lainen (Nokia Inc.), Sean Landis (Motorola), Patrick Lardieri (Lockheed Martin), Doug Lea
(SUNY Oswego), Joe Loyall (BBN), Kent Madsen (EO Thorpe), Ed Margand (DARPA),
Mike Masters (NSWC), Major Ed Mays (U.S. Marine Corps), John Mellby (Raytheon),
Jeanette Milos (DARPA), Stan Moyer (Telcordia), Ivan Murphy (Siemens), Russ Nose-
worthy (Object Sciences), Adam Porter (U. of Maryland), Dieter Quehl (Siemens), Vi-
jay Raghavan (Vanderbilt U.), Lucie Robillard (U.S. Air Force), Craig Rodrigues (BBN),
Rick Schantz (BBN), Andreas Schulke (Siemens), Steve Shaffer (Kodak), Tom Shields
(Raytheon), Dave Sharp (Boeing), Naval Sodha (Ericsson), Paul Stephenson (Ericsson),
Tatsuya Suda (UCI), Umar Syyid (Storetrax, Inc.), Janos Sztipanovits (Vanderbilt U.), Gau-
tam Thaker (Lockheed Martin), Lothar Werzinger (Krones), and Don Winter (Boeing).
xviii About This Book

Very special thanks go to Susan Cooper, our copy editor, for enhancing our written
material. In addition, we are grateful for the encouragement and patience of our editor,
Debbie Lafferty, our production coordinator, Elizabeth Ryan, the series editor and inventor
of C++, Bjarne Stroustrup, and everyone else at Addison-Wesley who made it possible to
publish this book.
Finally, we would also like to acknowledge our gratitude and indebtedness to the late
W. Richard Stevens, the father of network programming literature. The following poem by
Samuel Butler sums up our view of Richard’s enduring influence:

Not on sad Stygian shore, nor in clear sheen

Of far Elysian plain, shall we meet those
Among the dead whose pupils we have been . . .
Yet meet we shall, and part, and meet again,
Where dead men meet, on lips of living men.

Steve’s Acknowledgments
Wow. . . C++NPv1 took almost 3 years to complete—this volume took roughly nine months.
Thank you to my wife Jane who cheerfully endured this process. Your persistent exhorta-
tion to keep life in balance and “be the tortoise” really helped me stay the course, and
without your infinite patience through many long days and nights, I would not have com-
pleted this—thank you! Thanks to Doug Schmidt for getting the bulk of this book down
and organized in world-class time amidst a full-time job and his usual, amazing amount
of work on ACE. Finally, thank you to Riverace’s customers who supported this work so
enthusiastically. It’s a privilege to serve you.

Doug’s Acknowledgments
I’d like to thank my wife Sonja and my parents for their love and support during the writing
of this book. Now that it’s done we’ll have lots more time to have fun! Thanks also to Steve
Huston, who time-shared his overloaded schedule to wrap up the book. I’d also like to thank
my friends and colleagues at the College of William and Mary; Washington University, St.
Louis; University of California, Irvine; Vanderbilt University; DARPA; and Siemens—as
well as the thousands of ACE and TAO developers and users worldwide—who have greatly
enriched my intellectual and interpersonal life over the past two decades. I look forward to
working with all of you in the future.
C HAPTER 1

Object-Oriented Frameworks for

Network Programming

C HAPTER S YNOPSIS
Object-oriented frameworks help reduce the cost and improve the quality of networked ap-
plications by reifying software designs and pattern languages that have proven effective in
particular application domains. This chapter illustrates what frameworks are and compares
them with other popular software development techniques, such as class libraries, compo-
nents, patterns, and model-integrated computing. It then illustrates the process of applying
frameworks to networked applications and outlines the ACE frameworks that are the fo-
cus of this book. These frameworks are based on a pattern language [POSA1, POSA2]
that has been applied to thousands of production networked applications and middleware
worldwide.

1.1 An Overview of Object-Oriented Frameworks

Even as computing power and network bandwidth increase dramatically, the development
of networked application software remains expensive, time consuming, and error prone.
The cost and effort stems from the growing demands placed on networked software, as
well as the continual rediscovery and reinvention of core software design and implemen-
tation artifacts throughout the software industry. Moreover, the heterogeneity of hardware
architectures, diversity of OS and network platforms, and stiff global competition makes it
increasingly hard to build high-quality networked application software from scratch.
The key to building high-quality networked software in a time-to-market-driven envi-
ronment is the ability to reuse successful software designs and implementations that have
already been developed. Reuse has been a popular topic of debate and discussion for over
30 years in the software community [McI68]. There are two general types of reuse:

1
2 CHAPTER 1 Object-Oriented Frameworks for Network Programming

• Opportunistic reuse, in which developers cut and paste code from existing programs
to create new ones. Opportunistic reuse works in a limited way for individual pro-
grammers or small groups. It doesn’t scale up across business units or enterprises,
however, and therefore doesn’t significantly reduce development cycle time and cost
or improve software quality. Worse, opportunistic reuse can actually impede devel-
opment progress since cut-and-paste code often begins to diverge as it proliferates,
forcing developers to fix the same bugs multiple times in multiple places.
• Systematic reuse, which is an intentional and concerted effort to create and apply
multiuse software architectures, patterns, frameworks, and components throughout
a product line [CN02]. In a well-honed systematic reuse process, each new project
leverages time-proven designs and implementations, only adding new code that’s spe-
cific to a particular application. This type of reuse is essential to increase software
productivity and quality by breaking the costly cycle of rediscovering, reinventing,
and revalidating common software artifacts.
Middleware [SS02] is a class of software that can increase systematic reuse levels
significantly by functionally bridging the gap between the end-to-end functional require-
ments of networked applications and the underlying operating systems and network pro-
tocol stacks. Middleware provides capabilities that are critical to networked applications
because they automate common network programming tasks. Developers who use mid-
dleware can therefore program their networked applications more like stand-alone applica-
tions, rather than wrestling with the many tedious and error-prone details associated with
low-level OS event demultiplexing, message buffering and queueing, marshaling and de-
marshaling, and connection management mechanisms. Popular examples of middleware
include Java virtual machines (JVMs), Enterprise JavaBeans (EJB), .NET, the Common
Object Request Broker Architecture (CORBA), and the ADAPTIVE Communication Envi-
ronment (ACE).
Systematically developing high-quality, reusable middleware for networked applica-
tions presents many hard technical challenges, including
• Detecting and recovering from transient and partial failures of networks and hosts in
an application-independent manner
• Minimizing the impact of latency and jitter on end-to-end application performance
• Determining how to partition a distributed application into separate component ser-
vices
• Deciding where and when to distribute and load balance services in a network
Since reusable middleware is inherently abstract, it’s hard to validate its quality and to man-
age its production. Moreover, the skills required to develop, deploy, and support reusable
networked application middleware have traditionally been a “black art,” locked in the heads
of expert developers and architects. These technical impediments to systematic reuse are of-
ten exacerbated by a myriad of nontechnical impediments [Hol97], such as organizational,
Section 1.1 An Overview of Object-Oriented Frameworks 3

economic, administrative, political, sociological, and psychological factors. It’s therefore

not surprising that significant levels of software reuse have been slow to materialize in many
projects and organizations [Sch00].
While it’s never easy to make reuse work universally, we’ve led the development of
powerful host infrastructure middleware called ACE that’s designed specifically with sys-
tematic reuse in mind. During the past decade, we’ve written hundreds of thousands of
lines of C++ code while developing and applying ACE to networked applications as part
of our work with dozens of telecommunication, aerospace, medical, and financial services
companies. As a result of our experience, we’ve documented many patterns and pattern
languages [POSA2, POS00] that have guided the design of reuseable middleware and ap-
plications. In addition, we’ve taught hundreds of tutorials and courses on reuse, middle-
ware, and patterns to thousands of developers and students. Despite the many technical
and nontechnical challenges, we’ve identified a solid body of work that combines advanced
research, time-proven design knowledge, hands-on experience, and software artifacts that
can significantly enhance the systematic reuse of networked application software.
At the heart of this body of work are object-oriented frameworks [FJS99b, FJS99a],
which are a powerful technology for achieving systematic reuse of networked application
software.1 Below, we describe the three characteristics of frameworks [JF88] that help them
to achieve the important networked application qualities listed on page xi. Figure 1.1 (page
4) illustrates how these characteristics work together.
A framework provides an integrated set of domain-specific structures and function-
ality. Systematic reuse of software depends largely on how well frameworks model the
commonalities and variabilities [CHW98] in application domains, such as business data
processing, telecom call processing, graphical user interfaces, or distributed object com-
puting middleware. Since frameworks reify the key roles and relationships of classes in
application domains, the amount of reusable code increases and the amount of code rewrit-
ten for each application decreases.
A framework exhibits “inversion of control” at run time via callbacks. A callback
is an object registered with a dispatcher that calls back to a method on the object when
a particular event occurs, such as a connection request or data arriving on a socket han-
dle. Inversion of control decouples the canonical detection, demultiplexing, and dispatch-
ing steps within a framework from the application-defined event handlers managed by the
framework. When events occur, the framework calls back to virtual hook methods in the
registered event handlers, which then perform application-defined processing in response
to the events.
Since frameworks exhibit inversion of control, they can simplify application design be-
cause the framework—rather than the application—runs the event loop to detect events, de-
multiplex events to event handlers, and dispatch hook methods on the handlers that process
1
In the remainder of this book we use the term framework to mean object-oriented framework.
4 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Networking Event
loop

Callbacks GUI

Application- Domain-specific
specific Callbacks framework
functionality capabilities
Event
loop
Callbacks

Event
Database
loop

Figure 1.1: Synergy of Framework Capabilities

the events. The use of virtual hook methods in the handler classes decouples the applica-
tion’s classes from the framework, allowing each to be changed independently as long as
the interface signature and interaction protocols aren’t modified.
A framework is a “semi-complete” application that programmers can customize to form
complete applications by inheriting from and instantiating classes in the framework. Inheri-
tance enables the features of framework base classes to be shared selectively by subclasses.
If a base class provides default implementations of its methods, application developers need
only override those virtual methods whose default behavior doesn’t meet their needs.
Since a framework is a semi-complete application, it enables larger-scale reuse of soft-
ware than can be achieved by reusing individual classes or stand-alone functions. The
amount of reuse increases due to a framework’s ability to integrate application-defined and
application-independent classes. In particular, a framework abstracts the canonical control
flow of applications in a domain into families of related classes, which can collaborate to
integrate customizable application-independent code with customized application-defined
code.

1.2 Comparing Software Development and Reuse Techniques

Object-oriented frameworks don’t exist in isolation. Class libraries, components, patterns,
and model-integrated computing are other techniques that are being applied to reuse soft-
ware and increase productivity. This section compares frameworks with these techniques
to illustrate their similarities and differences, as well as to show how the techniques can be
combined to enhance systematic reuse for networked applications.
Section 1.2 Comparing Software Development and Reuse Techniques 5

Local
Application- invocations Math Event
Networking
specific classes loop
functionality
ADT Callbacks GUI
classes

Application-specific
Database
event handler Callbacks
classes
functionality
Event
GUI loop
classes Callbacks

Glue
Event Network Event
code Database
loop IPC classes loop

(1) Class library architecture (2) Framework architecture

Figure 1.2: Class Library versus Framework Architectures

1.2.1 Comparing Frameworks and Class Libraries

A class is a general-purpose, reusable building block that specifies an interface and en-
capsulates the representation of its internal data and the functionality of its instances. A
library of classes was the most common first-generation object-oriented development tech-
nique [Mey97]. Class libraries generally support reuse-in-the-small more effectively than
function libraries since classes emphasize the cohesion of data and methods that operate on
the data.
Although class libraries are often domain independent and can be applied widely, their
effective scope of reuse is limited because they don’t capture the canonical control flow,
collaboration, and variability among families of related software artifacts. The total amount
of reuse with class libraries is therefore relatively small, compared with the amount of
application-defined code that must be rewritten for each application. The need to reinvent
and reimplement the overall software architecture and much of the control logic for each
new application is a prime source of cost and delay for many software projects.
The C++ standard library [Bja00] is a good case in point. It provides classes for strings,
vectors, and other containers. Although these classes can be reused in many application
domains, they are relatively low level. Application developers are therefore responsible for
(re)writing much of the “glue code” that performs the bulk of the application control flow
and class integration logic, as shown in Figure 1.2 (1).
Frameworks are a second-generation development technique [Joh97] that extends the
benefits of class libraries in several ways. Most importantly, classes in a framework collab-
orate to provide a reusable architecture for a family of related applications. Class collab-
oration in a framework yields “semi-complete” applications that embody domain-specific
object structures and functionality. Frameworks can be classified by various means, such as
the blackbox and whitebox distinctions described in Sidebar 1 (page 6).
6 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Sidebar 1: Overview of Whitebox and Blackbox Frameworks

Frameworks can be classified in terms of the techniques used to extend them, which
range along a continuum from whitebox frameworks to blackbox frameworks [HJE95],
as described below:
• Whitebox frameworks. Extensibility is achieved in a whitebox framework via
object-oriented language features, such as inheritance and dynamic binding.
Existing functionality can be reused and customized by inheriting from frame-
work base classes and overriding predefined hook methods [Pre95] using pat-
terns such as Template Method [GoF], which defines an algorithm with some
steps supplied by a derived class. To extend a whitebox framework, application
developers must have some knowledge of its internal structure.
• Blackbox frameworks. Extensibility is achieved in a blackbox framework by
defining interfaces that allow objects to be plugged into the framework via
composition and delegation. Existing functionality can be reused by defin-
ing classes that conform to a particular interface and then integrating these
classes into the framework using patterns such as Function Object [Kuh97],
Bridge/Strategy [GoF], and Pluggable Factory [Vli98b, Vli99, Cul99], which pro-
vide a blackbox abstraction for selecting one of many implementations. Black-
box frameworks can be easier to use than whitebox frameworks since appli-
cation developers need less knowledge of the framework’s internal structure.
Blackbox frameworks can also be harder to design, however, since framework
developers must define crisp interfaces that anticipate a range of use cases.

Another way that class libraries differ from frameworks is that the classes in a library
are typically passive since they perform their processing by borrowing the thread from so-
called self-directed applications that invoke their methods. As a result, developers must
continually rewrite much of the control logic needed to bind the reusable classes together to
form complete networked applications. In contrast, frameworks are active since they direct
the flow of control within an application via various callback-driven event handling patterns,
such as Reactor [POSA2] and Observer [GoF]. These patterns invert the application’s flow
of control using the Hollywood Principle: “Don’t call us, we’ll call you” [Vli98a]. Since
frameworks are active and manage the application’s control flow, they can perform a broader
range of activities on behalf of applications than is possible with passive class libraries.
Frameworks and class libraries are complementary technologies in practice. Frame-
works provide a foundational structure to applications. Since frameworks are focused on a
specific domain, however, they aren’t expected to satisfy the broadest range of application
development needs. Class libraries are therefore often used in conjunction within frame-
works and applications to implement commonly needed code artifacts, such as strings, files,
and time/date classes.
Section 1.2 Comparing Software Development and Reuse Techniques 7

Deployment and
configuration
metadata

Trading Logging
Deployment and component component
configuration
mechanism
Containers Containers

Application server

Figure 1.3: A Component Architecture

For example, the ACE frameworks use the ACE wrapper facade classes to ensure their
portability. Likewise, applications can use the ACE container classes described in [HJS]
to help implement their event handlers. Whereas the ACE container classes and wrapper
facades are passive, the ACE frameworks are active and provide inversion of control at run
time. The ACE toolkit provides both frameworks and a library of classes to help program-
mers address a range of challenges that arise when developing networked applications.

1.2.2 Comparing Frameworks and Components

A component is an encapsulated part of a software system that implements a specific ser-
vice or set of services. A component has one or more interfaces that provide access to its
services. Components serve as building blocks for the structure of an application and can
be reused based solely upon knowledge of their interface protocols.
Components are a third-generation development technique [Szy98] that are widely used
by developers of multitier enterprise applications. Common examples of components in-
clude ActiveX controls [Egr98] and COM objects [Box98], .NET web services [TL01], En-
terprise JavaBeans [MH01], and the CORBA Component Model (CCM) [Obj01a]. Com-
ponents can be plugged together or scripted to form complete applications, as shown in
Figure 1.3.
Figure 1.3 also shows how a component implements the business application logic in
the context of a container. A container allows its component to access resources and ser-
vices provided by an underlying middleware platform. In addition, this figure shows how
generic application servers can be used to instantiate and manage containers and execute
the components configured into them. Metadata associated with components provide in-
structions that application servers use to configure and connect components.
8 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Many interdependent components in enterprise applications can reside in multiple—

possibly distributed—application servers. Each application server consists of some number
of components that implement certain services for clients. These components in turn may
include other collocated or remote services. In general, components help developers reduce
their initial software development effort by integrating custom application components with
reusable off-the-shelf components into generic application server frameworks. Moreover, as
the requirements of applications change, components can help make it easier to migrate and
redistribute certain services to adapt to new environments, while preserving key application
properties, such as security and availability.
Components are generally less lexically and spatially coupled than frameworks. For
example, applications can reuse components without having to subclass them from existing
base classes. In addition, by applying common patterns, such as Proxy [GoF] and Bro-
ker [POSA1], components can be distributed to servers throughout a network and accessed
by clients remotely. Modern application servers, such as JBoss and BEA Systems’s Web-
Logic Server, use these types of patterns to facilitate an application’s use of components.
The relationship between frameworks and components is highly synergistic, with nei-
ther subordinate to the other [Joh97]. For example, the ACE frameworks can be used to de-
velop higher-level application components, whose interfaces then provide a facade [GoF]
for the internal class structure of the frameworks. Likewise, components can be used as
pluggable strategies in blackbox frameworks [HJE95]. Frameworks are often used to sim-
plify the development of middleware component models [TL01, MH01, Obj01a], whereas
components are often used to simplify the development and configuration of networked
application software.

1.2.3 Comparing Frameworks and Patterns

Developers of networked applications must address design challenges related to complex
topics, such as connection management, service initialization, distribution, concurrency
control, flow control, error handling, event loop integration, and dependability. Since these
challenges are often independent of specific application requirements, developers can re-
solve them by applying the following types of patterns [POSA1]:
• Design patterns provide a scheme for refining the elements of a software system and
the relationships between them, and describe a common structure of communicating
elements that solves a general design problem within a particular context.
• Architectural patterns express the fundamental, overall structural organization of
software systems and provide a set of predefined subsystems, specify their responsi-
bilities, and include guidelines for organizing the relationships between them.
• Pattern languages define a vocabulary for talking about software development prob-
lems and provide a process for the orderly resolution of these problems.
Section 1.2 Comparing Software Development and Reuse Techniques 9

Traditionally, patterns and pattern languages have been locked in the heads of expert
developers or buried deep within the source code of software applications and systems.
Allowing this valuable information to reside only in these locations is risky and expensive.
Explicitly capturing and documenting patterns for networked applications helps to
• Preserve important design information for programmers who enhance and main-
tain existing software. This information will be lost if it isn’t documented, which can
increase software entropy and decrease software maintainability and quality.
• Guide design choices for developers who are building new applications. Since pat-
terns document the common traps and pitfalls in their domain, they help developers
to select suitable architectures, protocols, algorithms, and platform features without
wasting time and effort (re)implementing solutions that are known to be inefficient
or error prone.

Knowledge of patterns and pattern languages helps to reduce development effort and
maintenance costs. Reuse of patterns alone, however, does not create flexible and effi-
cient software. Although patterns enable reuse of abstract design and architecture knowl-
edge, software abstractions documented as patterns don’t directly yield reusable code. It’s
therefore essential to augment the study of patterns with the creation and use of frame-
works. Frameworks help developers avoid costly reinvention of standard software artifacts
by reifying common patterns and pattern languages and by refactoring common implemen-
tation roles.
ACE users can write networked applications quickly because the frameworks in ACE
implement the core patterns associated with service access, event handling, concurrency,
and synchronization [POSA2]. This knowledge transfer makes ACE more accessible and
directly applicable compared to many other common knowledge transfer activities, such as
seminars, conferences, or design and code reviews. Although these other activities are use-
ful, they are limited because participants must learn from past work of others, and then try
to apply it to their current and future projects. In comparison, ACE provides direct knowl-
edge transfer by embodying framework usage patterns in a powerful toolkit containing both
networked application domain experience and working code.
For example, JAWS [HS99] is a high-performance, open-source, adaptive Web server
built using the ACE frameworks. Figure 1.4 (page 10) illustrates how the JAWS Web server
is structured as a set of collaborating frameworks whose design is guided by the patterns
listed along the borders of the figure. These patterns help resolve common design challenges
that arise when developing concurrent servers, including encapsulating low-level operating
system APIs, decoupling event demultiplexing and connection management from protocol
processing, scaling up server performance via multithreading, minimizing server threading
overhead, using asynchronous I/O effectively, and enhancing server configurability. More
information on the patterns and design of JAWS appears in Chapter 1 of POSA2.
10 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Reactor/Proactor Strategy Singleton

I/O strategy Cached virtual

State
Memento
framework filesystem

Asynchronous completion token Tilde ~

expander

Component configurator
/home/...
Protocol
Event dispatcher

Acceptor
handler

Protocol
filter

Pipes and filters

Adapter

Concurrency

State
Protocol pipeline strategy
framework framework

Component configurator Active object Strategy

Figure 1.4: Patterns Forming the Architecture of JAWS

1.2.4 Comparing Frameworks and Model-Integrated Computing

Model-integrated computing (MIC) [SK97] is an emerging development paradigm that
uses domain-specific modeling languages to systematically engineer software ranging from
small-scale real-time embedded systems to large-scale enterprise applications. MIC devel-
opment environments include domain-specific model analysis and model-based program
synthesis tools. MIC models can capture the essence of a class of applications, as well as
focus on a single, custom application. MIC also allows the modeling languages and en-
vironments themselves to be modeled by so-called meta-models [SKLN01], which help to
synthesize domain-specific modeling languages that can capture subtle insights about the
domains they are designed to model, making this knowledge available for reuse.
Popular examples of MIC being used today include the Generic Modeling Environment
(GME) [LBM+ 01] and Ptolemy [BHLM94] (which are used primarily in the real-time and
embedded domain) and UML/XML tools based on the OMG Model Driven Architecture
(MDA) [Obj01b] (which are used primarily in the business domain thus far). When imple-
mented properly, these MIC technologies help to

• Free application developers from dependencies on particular software APIs, which

ensures that the models can be reused for a long time, even as existing software APIs
are obsoleted by newer ones.
Section 1.2 Comparing Software Development and Reuse Techniques 11

Integrated
model

Model Platform-
interpreter Executable specific Application
and code specifications code code
synthesizer Integrate and Synthesize
generator
generate

System
constraints

Figure 1.5: Steps in the Model-Integrated Computing Development Process

• Provide correctness proofs for various algorithms by analyzing the models automati-
cally and offering refinements to satisfy various constraints.
• Generate code that’s highly dependable and robust since the modeling tools them-
selves can be synthesized from meta-models using provably correct technologies.
• Rapidly prototype new concepts and applications that can be modeled quickly using
this paradigm, compared to the effort required to prototype them manually.
• Reuse domain-specific modeling insights, saving significant amounts of time and
effort, while also reducing application time-to-market and improving consistency and
quality.
As shown in Figure 1.5, the MIC development process uses a set of tools to analyze
the interdependent features of the application captured in a model and determine the feasi-
bility of supporting different QoS requirements in the context of the specified constraints.
Another set of tools then translates models into executable specifications that capture the
platform behavior, constraints, and interactions with the environment. These executable
specifications in turn can be used to synthesize application software.
Earlier efforts at model-based development and code synthesis attempted by CASE
tools generally failed to deliver on their potential for the following reasons [All02]:
• They attempted to generate entire applications, including the infrastructure and the
application logic, which led to inefficient, bloated code that was hard to optimize,
validate, evolve, or integrate with existing code.
• Due to the lack of sophisticated domain-specific languages and associated modeling
tools, it was hard to achieve round-trip engineering, that is, moving back and forth
seamlessly between model representations and the synthesized code.
• Since CASE tools and early modeling languages dealt primarily with a restricted
set of platforms (such as mainframes) and legacy programming languages (such as
COBOL), they did not adapt well to the distributed computing paradigm that arose
12 CHAPTER 1 Object-Oriented Frameworks for Network Programming

from advances in PC and Internet technology and newer object-oriented program-

ming languages, such as Java, C++, and C#.

Many of the limitations with model-integrated computing outlined above can be over-
come by integrating MIC tools and processes with object-oriented frameworks [GSNW02].
This integration helps to overcome problems with earlier-generation CASE tools since it
does not require the modeling tools to generate all the code. Instead, large portions of ap-
plications can be composed from reusable, prevalidated framework classes. Likewise, inte-
grating MIC with frameworks helps address environments where application requirements
and functionality change at a rapid pace by synthesizing and assembling newer extended
framework classes and automating the configuration of many QoS-critical aspects, such as
concurrency, distribution, transactions, security, and dependability.
The combination of model-integrated computing with frameworks, components, and
patterns is an area of active research [Bay02]. In the DOC group, for example, there are
R&D efforts underway to develop a MIC tool suite called the Component Synthesis with
Model-Integrated Computing (CoSMIC) [GSNW02]. CoSMIC extends the popular GME
modeling and synthesis tools [LBM+ 01] and the ACE ORB (TAO) [SLM98] to support
the development, assembly, and deployment of QoS-enabled networked applications. To
ensure the QoS requirements can be realized in the middleware layer, CoSMIC’s model-
integrated computing tools can specify and analyze the QoS requirements of application
components in their accompanying metadata.

1.3 Applying Frameworks to Network Programming

One reason why it’s hard to write robust, extensible, and efficient networked applications is
that developers must master many complex networking programming concepts and mecha-
nisms, including
• Network addressing and service identification/discovery
• Presentation layer conversions, such as marshaling, demarshaling, and encryption, to
handle heterogeneous hosts with alternative processor byte orderings
• Local and remote interprocess communication (IPC) mechanisms
• Event demultiplexing and event handler dispatching
• Process/thread lifetime management and synchronization
Application programming interfaces (APIs) and tools have evolved over the years to sim-
plify the development of networked applications and middleware. Figure 1.6 illustrates the
IPC APIs available on OS platforms ranging from UNIX to many real-time operating sys-
tems. This figure shows how applications can access networking APIs for local and remote
IPC at several levels of abstraction. We briefly discuss each level of abstraction below,
starting from the lower-level kernel APIs to the native OS user-level networking APIs and
the host infrastructure middleware.
Section 1.3 Applying Frameworks to Network Programming 13

HI
HOST INFRASTRUCTURE MIDDLEWARE
SOCKETS & TLI
LEVEL OF USER
open()/close()/putmsg()/getmsg() SPACE
ABSTRACTION

TPI
STREAMS NPI
KERNEL
SPACE
LO FRAMEWORK DLPI
Figure 1.6: Levels of Abstraction for Network Programming

Kernel-level networking APIs. Lower-level networking APIs are available in an OS ker-

nel’s I/O subsystem. For example, the UNIX putmsg() and getmsg() system func-
tions can be used to access the Transport Provider Interface (TPI) [OSI92b] and the Data
Link Provider Interface (DLPI) [OSI92a] available in System V STREAMS [Rit84]. It’s
also possible to develop network services, such as routers [KMC+ 00], network file sys-
tems [WLS+ 85], or even Web servers [JKN+ 01], that reside entirely within an OS kernel.
Programming directly to kernel-level networking APIs is rarely portable between dif-
ferent OS platforms, however. It’s often not even portable across different versions of the
same OS! Since kernel-level programming isn’t used in most networked applications, we
don’t cover it any further in this book. See [Rag93], [SW95, MBKQ96], and [SR00] for
coverage of these topics in the context of System V UNIX, BSD UNIX, and Windows 2000,
respectively.
User-level networking APIs. Networking protocol stacks in modern commercial oper-
ating systems reside within the protected address space of the OS kernel. Applications
running in user space access protocol stacks in the OS kernel via IPC APIs, such as the
Socket or TLI APIs. These APIs collaborate with an OS kernel to provide the capabilities
shown in the following table:

Capability Description
Local endpoint Create and destroy local communication endpoints, allowing ac-
management cess to available networking facilities.
Connection establishment Enable applications to establish connections actively or passively
and connection with remote peers and to shutdown all or part of the connections
termination when transmissions are complete.
Options management Negotiate and enable/disable protocol and endpoint options.
Data transfer mechanisms Exchange data with peer applications.
Name/address translation Convert human-readable names to low-level network addresses
and vice versa.

These capabilities are covered in Chapter 2 of C++NPv1 in the context of the Socket API.
14 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Many IPC APIs are modeled loosely on the UNIX file I/O API, which defines the
open(), read(), write(), close(), ioctl(), lseek(), and select() func-
tions [Rit84]. Due to syntactic and semantic differences between file I/O and network I/O,
however, networking APIs provide additional functionality that’s not supported directly by
the standard UNIX file I/O APIs. For example, the pathnames used to identify files on a
UNIX system aren’t globally unique across hosts in a heterogeneous distributed environ-
ment. Different naming schemes, such as IP host addresses and TCP / UDP port numbers,
have therefore been devised to uniquely identify communication endpoints used by net-
worked applications.
Host infrastructure middleware frameworks. Many networked applications exchange
messages using synchronous and/or asynchronous request/response protocols in conjunc-
tion with host infrastructure middleware frameworks. Host infrastructure middleware en-
capsulates OS concurrency and IPC mechanisms to automate many low-level aspects of
networked application development, including
• Connection management and event handler initialization
• Event detection, demultiplexing, and event handler dispatching
• Message framing atop bytestream protocols, such as TCP
• Presentation conversion issues involving network byte ordering and parameter mar-
shaling and demarshaling
• Concurrency models and synchronization of concurrent operations
• Networked application composition from dynamically configured services
• Hierarchical structuring of layered networked applications and services
• Management of quality of service (QoS) properties, such as scheduling access to
processors, networks, and memory
The increasing availability and popularity of high-quality and affordable host infrastructure
middleware is helping to raise the level of abstraction at which developers of networked
applications can work effectively. For example, [C++NPv1, SS02] present an overview of
higher-level distributed object computing middleware, such as CORBA [Obj02] and The
ACE ORB (TAO) [SLM98], which is an implementation of CORBA built using the frame-
works and classes in ACE. It’s still useful, however, to understand how lower level IPC
mechanisms work to fully comprehend the challenges that arise when designing, porting,
and optimizing networked applications.

1.4 A Tour through the ACE Frameworks

1.4.1 An Overview of ACE
ACE is a highly portable, widely used, open-source host infrastructure middleware toolkit.
The source code is freely available from http://ace.ece.uci.edu/ or http://
Section 1.4 A Tour through the ACE Frameworks 15

NETWORKED
JAWS ADAPTIVE
SERVICE WEB SERVER
COMPONENTS STANDARDS-BASED MIDDLEWARE
LAYER THE ACE ORB
(TAO)
TOKEN GATEWAY
SERVER SERVER

LOGGING NAME TIME

SERVER SERVER SERVER

FRAMEWORK SERVICE CORBA

LAYER HANDLER ACCEPTOR CONNECTOR
HANDLER

C++ PROCESS/
STREAMS LOG
WRAPPER THREAD
SERVICE
SHARED
MALLOC
MANAGERS MSG
FACADE REACTOR/
CONFIGU-
RATOR
LAYER SYNCH SPIPE SOCK SAP/ FIFO PROACTOR MEM FILE
WRAPPERS SAP TLI SAP SAP MAP SAP

OS ADAPTATION LAYER
C
PROCESSES/
WIN32 NAMED
SOCKETS/ UNIX SELECT/ DYNAMIC SHARED FILE SYS
APIs THREADS
PIPES & UNIX
TLI FIFOS IO COMP LINKING MEMORY APIS
STREAM PIPES

PROCESS/THREAD COMMUNICATION VIRTUAL MEMORY & FILE

SUBSYSTEM SUBSYSTEM SUBSYSTEM
GENERAL OPERATING SYSTEM SERVICES

Figure 1.7: The Layered Architecture of ACE

www.riverace.com/. The core ACE library contains roughly a quarter million lines
of C++ code that comprises approximately 500 classes. Many of these classes cooperate to
form ACE’s major frameworks. The ACE toolkit also includes higher-level components, as
well as a large set of examples and an extensive automated regression test suite.
To separate concerns, reduce complexity, and permit functional subsetting, ACE is de-
signed using a layered architecture [POSA1], shown in Figure 1.7. The capabilities pro-
vided by ACE span the session, presentation, and application layers in the OSI reference
model [Bla91]. The foundation of the ACE toolkit is its combination of an OS adaptation
layer and C++ wrapper facades, which together encapsulate core OS network programming
mechanisms to run portably on all the OS platforms shown in Sidebar 2 (page 16). The
higher layers of ACE build on this foundation to provide reusable frameworks, networked
service components, and standards-based middleware.

1.4.2 A Synopsis of the ACE Frameworks

The ACE frameworks are an integrated set of classes that can be instantiated and customized
to provide complete networked applications and service components. These frameworks
help to transfer decades of accumulated knowledge directly from the ACE developers to
16 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Acceptor-
Reactor Proactor
Connector

Service
Streams Task
Configurator

Figure 1.8: The Key Frameworks in ACE

Sidebar 2: OS Platforms Supported by ACE

ACE runs on a wide range of operating systems, including:

• PCs, for example, Windows (32- and 64-bit versions), WinCE, and Macintosh
OS X
• Most versions of UNIX, for example, SunOS/Solaris, IRIX, HP-UX, Tru64
UNIX (Digital UNIX), AIX, DG/UX, Linux (Redhat, Debian, and SuSE), SCO
OpenServer, UnixWare, NetBSD, and FreeBSD
• Real-time operating systems, for example, VxWorks, ChorusOS, LynxOS, Phar-
lap TNT, QNX Neutrino and RTP, RTEMS, and pSoS
• Large enterprise systems, for example, OpenVMS, MVS OpenEdition, Tandem
NonStop-UX, and Cray UNICOS.
ACE can be used with all of the major C++ compilers on these platforms. The ACE
Web site at http://ace.ece.uci.edu contains a complete, up-to-date list of plat-
forms, along with instructions for downloading and building ACE.

ACE users in the form of expertise embodied in well-tested and reusable C++ software
artifacts. The ACE frameworks implement a pattern language for programming concurrent
object-oriented networked applications. Figure 1.8 illustrates the ACE frameworks. To
illustrate how the ACE frameworks rely on and use each other, the lines between boxes
represent a dependency in the direction of the arrow. Each framework is outlined below.
ACE Reactor and Proactor frameworks. These frameworks implement the Reactor and
Proactor patterns [POSA2], respectively. Both are architectural patterns that allow appli-
cations to be driven by events that are delivered to the application from one or more event
sources, the most important of which are I/O endpoints. The Reactor framework facilitates
a reactive I/O model, with events signaling the ability to begin a synchronous I/O opera-
tion. The Proactor framework is designed for a proactive I/O model where one or more
asynchronous I/O operations are initiated and the completion of each operation triggers an
event. Proactive I/O models can achieve the performance benefits of concurrency without
incurring many of its liabilities. The Reactor and Proactor frameworks automate the detec-
tion, demultiplexing, and dispatching of application-defined handlers in response to many
Section 1.4 A Tour through the ACE Frameworks 17

types of events. Chapters 3 and 4 describe the ACE Reactor framework and Chapter 8
describes the ACE Proactor framework.

ACE Service Configurator framework. This framework implements the Component

Configurator pattern [POSA2], which is a design pattern that allows an application to link
and unlink its component implementations without having to modify, recompile, or relink
the application statically. The ACE Service Configurator framework supports the config-
uration of applications whose services can be assembled late in the design cycle, such as
at installation time and/or run time. Applications with high availability requirements, such
as mission-critical systems that perform online transaction processing or real-time indus-
trial process automation, often require such flexible configuration capabilities. Chapter 2
describes the design dimensions associated with configuring networked services and Chap-
ter 5 describes the ACE Service Configurator framework.

ACE Task framework. This framework implements various concurrency patterns, such
as Active Object and Half-Sync/Half-Async [POSA2]. Active Object is a design pattern
that decouples the thread that executes a method from the thread that invoked it. Its purpose
is to enhance concurrency and simplify synchronized access to objects that reside in their
own threads of control. Half-Sync/Half-Async is an architectural pattern that decouples
asynchronous and synchronous processing in concurrent systems, to simplify programming
without reducing performance unduly. This pattern incorporates two intercommunicating
layers, one for asynchronous and one for synchronous service processing. A queueing
layer mediates communication between services in the asynchronous and synchronous lay-
ers. Chapter 5 of C++NPv1 describes the design dimensions associated with concurrent
networked applications and Chapter 6 of this book describes the ACE Task framework.

ACE Acceptor-Connector framework. This framework leverages the Reactor frame-

work and reifies the Acceptor-Connector pattern [POSA2]. This design pattern decou-
ples the connection and initialization of cooperating peer services in a networked system
from the processing they perform once connected and initialized. The Acceptor-Connector
framework decouples the active and passive initialization roles from application-defined
service processing performed by communicating peer services after initialization is com-
plete. Chapter 7 describes this framework.

ACE Streams framework. This framework implements the Pipes and Filters pattern,
which is an architectural pattern that provides a structure for systems that process a stream
of data [POSA1]. The ACE Streams framework simplifies the development and compo-
sition of hierarchically layered services, such as user-level protocol stacks and network
management agents [SS94]. Chapter 9 describes this framework.
When used together, the ACE frameworks outlined above enable the development of
networked applications that can be updated and extended without the need to modify, re-
18 CHAPTER 1 Object-Oriented Frameworks for Network Programming

ACE task Event

loop

ADT Callbacks ACE

classes
streams

Application-specific
Local
event handler Callbacks
invocations
functionality
Event
IPC loop
classes Callbacks

Event
ACE reactor
loop

Figure 1.9: Applying Class Libraries to Develop and Use ACE Frameworks

compile, relink, or restart running applications. ACE achieves this unprecedented flexibility
and extensibility by combining
• OS mechanisms, such as event demultiplexing, IPC, dynamic linking, multithread-
ing, multiprocessing, and synchronization [Ste99]
• C++ language features, such as templates, inheritance, and dynamic binding [Bja00]
• Patterns, such as Component Configurator [POSA2], Strategy [GoF], and Han-
dler/Callback [Ber95]

The ACE frameworks provide inversion of control via callbacks, as shown below:

ACE Framework Inversion of Control

Reactor and Calls back to application-supplied event handlers to perform process-
Proactor ing when events occur synchronously and asynchronously.
Service Configurator Calls back to application-supplied service objects to initialize, suspend,
resume, and finalize them.
Task Calls back to an application-supplied hook method to perform process-
ing in one or more threads of control.
Acceptor-Connector Calls back to service handlers to initialize them after they’re connected.
Streams Calls back to initialize and finalize tasks when they are pushed and
popped from a stream.

The callback methods in ACE’s framework classes are defined as C++ virtual methods.
This use of dynamic binding allows networked applications to freely implement and extend
interface methods without modifying or rebuilding existing framework classes. In contrast,
the ACE wrapper facades rarely use callbacks or virtual methods, so they aren’t as exten-
sible as the ACE frameworks. The ACE wrapper facades do support a broad range of use
Section 1.5 Example: A Networked Logging Service 19

cases, however, and can be integrated together via generic programming [Ale01] techniques
based on the C++ traits and traits classes idioms outlined in Sidebar 40 (page 165).
Figure 1.9 illustrates how the class libraries and frameworks in ACE are complemen-
tary technologies. The ACE toolkit simplifies the implementation of its frameworks via
its class libraries of containers, which include lists, queues, hash tables, strings, and other
reusable data structures. Likewise, application-defined code invoked by event handlers in
the ACE Reactor framework can use the ACE wrapper facades and the C++ standard library
classes [Jos99] to perform IPC, synchronization, file management, and string processing op-
erations. Sidebar 3 describes how to build the ACE library so that you can experiment with
the examples we present in this book.

Sidebar 3: Building ACE and Programs that Use ACE

ACE is open-source software that you can download from http://ace.ece.uci.

edu or http://www.riverace.com and build yourself. These sites contain a
wealth of other material on ACE, such as tutorials, technical papers, and an overview
of other ACE wrapper facades and frameworks that aren’t covered in this book. You
can also purchase a prebuilt version of ACE from Riverace at a nominal cost. See
http://www.riverace.com for a list of the prebuilt compiler and OS platforms
supported by Riverace.
If you want to build ACE yourself, you should download and unpack the ACE dis-
tribution into an empty directory. The top-level directory in the distribution is named
ACE_wrappers. We refer to this top-level directory as “ACE_ROOT.” You should cre-
ate an environment variable by that name containing the full path to the top-level ACE
directory. The ACE source and header files reside in $ACE_ROOT/ace.
The $ACE_ROOT/ACE-INSTALL.html file has complete instructions for building
ACE, including how to configure it for your OS and compiler. This book’s networked
logging service example source and header files reside in $ACE_ROOT/examples/
C++NPv2 and are ready to build on all platforms that ACE supports. To build your
own programs, the $ACE_ROOT directory must be added to your compiler’s file include
path. For command-line compilers, this can be done with the -I or /I compiler option.
Graphical IDEs provide similar options, such as MSVC++’s “Preprocessor, Additional
include directories” section of the C/C++ tab on the Project Settings dialog box.

1.5 Example: A Networked Logging Service

It’s been our experience that the principles, methods, and skills required to develop and
use reusable networked application software cannot be learned solely by generalities or
toy examples. Instead, programmers must learn concrete technical skills and gain hands-
on experience by developing and using real frameworks and applications. We therefore
20 CHAPTER 1 Object-Oriented Frameworks for Network Programming

illustrate key points and ACE capabilities throughout this book by extending and enhancing
the networked logging service example introduced in C++NPv1, which collects and records
diagnostic information sent from one or more client applications.
The logging service in C++NPv1 used many of ACE’s wrapper facades in a two-tier
client/server architecture. This book’s logging service examples use a more powerful archi-
tecture that illustrates a broader complement of capabilities and patterns, and demonstrates
how ACE’s frameworks can help achieve efficient, predictable, and scalable networked ap-
plications. This service also helps to demonstrate key design and implementation consider-
ations and solutions that will arise when you develop your own concurrent object-oriented
networked applications.
Figure 1.10 illustrates the application processes and daemons in our networked logging
service, which we outline below.
Client application processes (such as P1 , P2 , and P3 ) run on client hosts and generate log
records ranging from debugging messages to critical error messages. The logging informa-
tion sent by a client application contains the time the log record was created, the process
identifier of the application, the priority level of the log record, and a variable-sized string
containing the log record text message. Client applications send these log records to a client
logging daemon running on their local host.
Client logging daemons run on every host machine participating in the networked logging
service. Each client logging daemon receives log records from that host’s client applications
via some form of local IPC mechanism, such as shared memory, pipes, or sockets. The
client logging daemon uses a remote IPC mechanism, such as TCP / IP, to forward log records
to a server logging daemon running on a designated host.
Server logging daemons collect and output the incoming log records they receive from
client applications via client logging daemons. A server logging daemon 2 can determine
which client host sent each message by using addressing information it obtains from the
underlying Socket API. There’s generally one server logging daemon per system configu-
ration, though they could be replicated to avoid a single point of failure.
Figure 1.11 (page 22) shows the progression of networked application servers that we’ll
develop and use in this book. These client and server logging daemons will illustrate how
to use the ACE frameworks and wrapper facades with the following concurrency models.
Concurrency Model Section
Reactive 3.5, 4.2, 5.4
Thread pool 4.3, 4.4, 6.3
Thread-per-connection 7.2, 7.3
Producer/consumer 6.2, 7.4, 9.2
Proactive 8.2 – 8.5

2
We use the terms server logging daemon and logging server interchangeably throughout this book.
Section 1.6 Summary 21

Oct 31 14:48:13 [email protected]@38491@7@client::unable to fork in function spawn

Oct 31 14:50:28 [email protected]@18352@2@drwho::sending request to server tango

Console
Server logging Storage
Local IPC daemon device
Client TCP connection
P1 logging Tango Mambo
daemon

P2

TCP connection
P3
Printer
Server

Network

Client Client P1
Tango logging
daemon
P2

Local IPC P3
Client Mambo
int spawn (void){
if (ACE_OS::fork()==-1)
ACE_ERROR((LM_ERROR, if (Options::instance()->debug())
"unable to fork in function spawn")); ACE_DEBUG((LM_DEBUG,
"sending request to server%s",
server_host));

Figure 1.10: Processes and Daemons in the Networked Logging Service

1.6 Summary
Networked application software has been developed manually from scratch for decades.
The continual rediscovery and reinvention of core concepts and capabilities associated with
this process has kept the costs of engineering and evolving networked applications too high
for too long. Improving the quality and quantity of systematic software reuse is essential to
resolve this problem.
Middleware is a class of software that’s particularly effective at providing systemati-
cally reusable artifacts for networked applications. Developing and using middleware is
therefore an important way to increase reuse. There are many technical and nontechni-
cal challenges that make middleware development and reuse hard, however. This chapter
described how object-oriented frameworks can be applied to overcome many of these chal-
22 CHAPTER 1 Object-Oriented Frameworks for Network Programming

Reactive
Logging Server
(Chapters 3, 4)

Thread Pool Dynamically Half-Sync/ Thread-per-

Reactive Configured Half-Async Connection
Logging Logging Logging Logging
Server Server Server Server
(Chapter 4) (Chapter 5) (Chapter 6) (Chapter 7)

Acceptor- Producer/
Proactive
Connector Consumer
Client
Client Client
Logging
Logging Logging
Daemon
Daemon Daemon
(Chapter 8)
(Chapter 7) (Chapter 6)

Figure 1.11: Logging Server Examples

lenges. To make the most appropriate choice of software development technologies, we

also described the differences between frameworks and class libraries, components, pat-
terns, and model-integrated computing. Each technology plays a part in reducing software
development costs and life cycles and increasing software quality, functionality, and perfor-
mance.
The result of applying framework development principles and patterns to the domain
of networked applications has yielded the ACE frameworks. These frameworks handle
common network programming tasks and can be customized via C++ language features
to produce complete networked applications. When used together, the ACE frameworks
simplify the creation, composition, configuration, and porting of networked applications
without incurring significant performance overhead. The rest of this book explains how and
why the ACE frameworks were developed and shows many examples of how ACE uses
C++ features to achieve its goals.
An intangible, but valuable, benefit of ACE is its transfer of decades of accumulated
knowledge from ACE framework developers to ACE framework users in the form of ex-
pertise embodied in well-tested C++ classes that implement time-proven networked appli-
cation software development strategies. These frameworks took scores of person-years to
develop, optimize, and mature. Fortunately, you can take advantage of the expertise embod-
ied in these frameworks without having to independently rediscover or reinvent the patterns
and classes that underlie them.
C HAPTER 2

Service and Configuration Design

Dimensions

C HAPTER S YNOPSIS
A service is a set of functionality offered to a client by a server. Common services available
on the Internet today include

• Web content retrieval services, such as Apache and Google

• Software distribution services, such as Castanet, Citrix, or Softricity
• Electronic mail and network news transfer services
• File access on remote machines
• Network time synchronization
• Payment processing services
• Streaming audio/video services, such as RealPlayer and QuickTime

Networked applications can be created by configuring their constituent services together at

various points of time, such as compile time, static link time, installation time, or run time.
Chapters 1 and 5 of C++NPv1 provided a domain analysis of the communication pro-
tocols and mechanisms and the concurrency architectures used by networked applications.
This chapter expands that coverage to analyze other design dimensions that address key
networked application properties. These properties include service duration and structure,
how networked services are identified, and the time at which they are bound together to form
complete applications. These design dimensions are important in any networked applica-
tion, and of particular importance to the ACE Service Configurator framework (Chapter 5).
If you’re already familiar with these design dimensions, however, you may want to skip
ahead to Chapter 3, which begins the coverage of the ACE frameworks.

23
24 CHAPTER 2 Service and Configuration Design Dimensions

2.1 Service and Server Design Dimensions

When designing networked applications, it’s important to recognize the difference between
a service, which is a capability offered to clients, and a server, which is the mechanism by
which the service is offered. The design decisions regarding services and servers are easily
confused, but should be considered separately. This section covers the following service
and server design dimensions:
• Short- versus long-duration services
• Internal versus external services
• Stateful versus stateless services
• Layered/modular versus monolithic services
• Single- versus multiservice servers
• One-shot versus standing servers

2.1.1 Short-Duration versus Long-Duration Services

The services offered by network servers can be classified as short duration or long duration.
These time durations reflect how long a service holds system resources. The primary trade-
off in this design dimension involves holding system resources when they may be better
used elsewhere versus the overhead of restarting a service when it’s needed. In a networked
application, this dimension is closely related to protocol selection because setup require-
ments for different protocols can vary significantly.
Short-duration services execute in brief, often fixed, amounts of time and usually handle a
single request at a time. Examples of short-duration services include computing the current
time of day, resolving the Ethernet number of an IP address, and retrieving a disk block
from the cache of a network file server. To minimize the amount of time spent setting up a
connection, short-duration services are often implemented using connectionless protocols,
such as UDP / IP [Ste94].
Long-duration services run for extended, often variable, lengths of time and may han-
dle numerous requests during their lifetime. Examples of long-duration services include
transferring large software releases via FTP, downloading MP3 files from a Web server us-
ing HTTP, streaming audio and video from a server using RTSP, accessing host resources
remotely via TELNET, and performing remote file system backups over a network. Ser-
vices that run for longer durations allow more flexibility in protocol selection. For ex-
ample, to improve efficiency and reliability, these services are often implemented with
connection-oriented protocols, such as TCP / IP [Ste94], or session-oriented protocols, such
as RTSP [SRL98] or SCTP [SX01].
Section 2.1 Service and Server Design Dimensions 25

Dispatcher process Dispatcher process

select()
SVC1 SVC2 SVC3

SVC1 SVC2
select()

(1) Internal services (2) External services

Figure 2.1: Internal versus External Services

Logging service ⇒ From the standpoint of an individual log record, our server logging
daemon seems like a short-duration service. Each log record is limited to a maximum length
of 4K bytes, though in practice most are much smaller. The actual time spent handling
a log record is relatively short. Since a client may transmit many log records, however,
we optimize performance by designing client logging daemons to establish connections
with their peer server logging daemons. We then reuse these connections for subsequent
logging requests. It would be wasteful and time consuming to set up and tear down a socket
connection for each logging request, particularly when small requests are sent frequently.
We therefore model our client and server logging daemons as long-duration services.

2.1.2 Internal versus External Services

Services can be classified as internal or external. The primary tradeoffs in this dimension
are service initialization time, isolation of one service from another, and simplicity.
Internal services execute in the same address space as the server that receives the request,
as shown in Figure 2.1 (1). As described in Chapter 5 of C++NPv1, an internal service
can run iteratively, concurrently, or reactively in relation to other internal services. Internal
services usually have low initialization latency and their context switch time is generally
shorter than that of services residing in separate processes.
Internal services may also reduce application robustness, however, since separate ser-
vices within a process aren’t protected from one another. One faulty service can therefore
corrupt data shared with other internal services in the process, which may produce incor-
rect results, crash the process, or cause the process to hang indefinitely. As a result, internal
services should be reserved for code that can be trusted to operate correctly when run in the
context of other services in an application’s address space.
External services execute in different process address spaces. For instance, Figure 2.1 (2)
illustrates a master service process that monitors a set of network ports. When a connection
request arrives from a client, the master accepts the connection and then spawns a new pro-
26 CHAPTER 2 Service and Configuration Design Dimensions

cess to perform the requested service externally. External services may be more robust than
internal services since the failure of one need not cause the failure of another. To increase
robustness, therefore, mission-critical application services are often isolated in separate pro-
cesses. The price for this robustness, however, can be a reduction in performance due to
process management and IPC overhead.
Some server frameworks support both internal and external services. For example, the
INETD superserver [Ste98] is a daemon that listens for connection requests or messages on
certain ports and runs programs to perform the services associated with those ports. System
administrators can choose between internal and external services in INETD by modifying
the inetd.conf configuration file as follows:
• INETD can be configured to execute short-duration services, such as ECHO and DAY-
TIME , internally via calls to statically linked functions in the INETD program.
• INETD can also be configured to run longer-duration services, such as FTP and TEL -
NET , externally by spawning separate processes.

Sidebar 4 (page 31) describes this and other service provisioning mechanisms that use both
internal and external services.
Logging service ⇒ All logging server implementations in this book are designed as inter-
nal services. As long as only one type of service is configured into our logging server, we
needn’t isolate it from harmful side effects of other services. There are valid reasons to pro-
tect the processing of different client sessions from each other, however, particularly if ser-
vices are linked dynamically using the Component Configurator pattern [POSA2]. Chapter
8 in C++NPv1 therefore illustrates how to implement a logging server as an external service
using the ACE_Process and ACE_Process_Manager classes.

2.1.3 Stateful versus Stateless Services

Services can be classified as stateful or stateless. The amount of state, or context, that a
service maintains between requests impacts the complexity and resource consumption of
clients and servers. Stateful and stateless services trade off efficiency for reliability, with
the right choice depending on a variety of factors, such as the probability and impact of host
and network failures.
Stateful services cache certain information, such as session state, authentication keys, iden-
tification numbers, and I/O handles, in a server to reduce communication and computation
overhead. For instance, Web cookies enable a Web server to preserve state across multiple
page requests.
Stateless services retain no volatile state within a server. For example, the Network File
System (NFS) [Ste94] provides distributed data storage and retrieval services that don’t
maintain volatile state information within a server’s address space. Each request sent from
Another Random Scribd Document
with Unrelated Content
created, constituted, or delegated by the Christian people, nor does
it in any sense hold from them. Church power or authority comes
immediately from God to the central see, and from that see
radiates through the whole body; for the author agrees with us that
the church is an organism. Hence we recognize the Council of
Constance as a General Council only after it was convoked by
Gregory XII., who was, in our judgment, the true Bishop of the
Apostolic See, and hold legal only the acts confirmed by Martin V.
The disorganization of the church is, then, its dissolution or death.
It has no power to raise itself from the dead. If the central see
could really fail, the whole organism would fail. The church is
indefectible through the indefectibility of the Holy See, and that is
indefectible because it is Peter's See, and our Lord promised Peter
that, however Satan might try him, his faith should not fail: "Satan
has desired to sift thee as wheat; but I have prayed the Father that
thy faith fail not." The prayer of Christ cannot be unanswered, and
is a promise. The attacks on the Holy See have been violent and
continual, but they have never been successful. Our Lord's prayer
has been effective, and Peter's faith has never failed. No doubt
there is the full authority to teach and to govern in the church; but
this authority is not derived from the faithful nor distributed equally
among them, but resides primarily and in its plenitude in the Holy
See, and therefore in the bishop of that see, or the pope, Peter's
successor, in whom Peter lives and continues to teach and govern
the whole church. All Catholic bishops depend on him, and receive
from him their jurisdiction, and by authority from God through him
govern their respective dioceses. The church is papal in its essential
constitution, not simply episcopal; for we have seen that it is an
organism, and can be an organism only as proceeding from an
organic centre, or central see, on which its unity and catholicity
depend. The Apostolic See cannot be separated from the Sedens;
for without him it is empty, incapable of thinking, speaking, or
acting. It is, then, it seems to us, as utterly impossible to assert the
church as really one and catholic, without asserting the pope, or
supreme pastor, as it would be to assert an organism without
asserting a central organic cell. The failure of the pope would be
the failure of the whole church organism, with no power of
reorganization or reconstruction left—an important point in which
the church and the nation differ. The overlooking of this point of
difference is the reason why our catholicizing Anglicans suppose
that the church, though disorganized, is able to reorganize herself.
The reorganized church, if effected, would be a human
organization, not a divine organism as created by our Lord himself.
But the church, as we have seen, has never been disorganized, and
could not be without ceasing to exist, and cease to exist it cannot,
if catholic. The organic centre from which the whole organism is
evolved and directed has remained at Rome ever since Peter
transferred thither his chair from Antioch, and the particular
churches holding from it and continuing subject to it are integral
elements of the catholic organism, which is as perfect, as complete,
and as entire as it was when the Oriental churches acknowledged
and submitted to the supremacy of the successor of Peter, or when
the church in England was in full communion with the Apostolic See
of Rome. The separation of these from the Roman communion,
though it destroyed their unity and catholicity, did not break the
unity and catholicity of the organism; it only placed them outside of
that organism, and cut them off from the central see, the source of
all organic church life. The revolt of the Anglo-American Colonies
from Great Britain, in 1776, and the Declaration of their
Independence of the mother-country, did not break her unity or
authority as a nation, and indeed did not even deprive her of any
of her rights over them, though it enfeebled her power to govern
them, till she herself acknowledged them to be independent states.

The author seems to suppose that the Greeks and Anglicans, in

separating from Rome, broke the unity of the church, and carried
away with them each a portion of her catholicity, so that there now
can be no One Catholic Church existing in organic unity and
catholicity, save in reminiscence and in potentia, unless these two
bodies are reunited with Rome in one and the same communion.
But the Greeks and Anglicans had both for centuries recognized the
authority of the Apostolic See, as the centre of unity and source of
jurisdiction. When the Greeks separated from that see and refused
to obey it, they took from it neither its organic unity nor its
catholicity; they only cut themselves off and deprived themselves of
both. The same may be said of Anglicans in separating from Rome
and declaring themselves independent. They deprived themselves
of unity and catholicity, but left the original church organism in all
its integrity, and only placed themselves outside of it. The
separation in both cases deprived the church of a portion of her
population, and diminished her external power and grandeur, but
took away none of her rights and prerogatives, and in no respect
impaired, as we have already said, the unity or catholicity of her
internal organism. All that can be said is that the separated Oriental
churches and Anglicans, not the church, have lost unity and
catholicity, and have ceased to be Catholics, even when agreeing
with the church in her dogmas and her external rites and
ceremonies.

There is, then, on the side of the church, no broken unity to be

healed, or lost catholicity to be restored. If the Oriental churches
wish to regain unity and catholicity, all they have to do is to submit
to the jurisdiction of the Apostolic See, and renew their communion
with the mother and mistress of all the churches. Not having lost
their church organization, and having retained valid orders or the
priesthood and the august sacrifice, they can return in their
corporate capacity. There is in their case only a schism to be
healed. The Anglicans and Episcopalians stand on a different
footing; for they have not even so much as a schismatic church,
since the Episcopalians hold from the Anglicans, and the Anglicans
from the state. They have no orders, no priesthood, no sacrifice, no
sacraments—except baptism, and even pagans can administer that
—no church character at all, if we look at the facts in the case, and
therefore, like all Protestants, can be admitted to the unity of the
church only on individual profession and submission. There is much
for those out of unity to do to recover it and to effect the union in
the Catholic communion of all who profess to be Christians, but
nothing to render the church herself one and catholic. Her unity
and catholicity are already established and unalterable, and so are
the terms of communion and the conditions of church life. No
grand combination, then, is needed to restore a divided and
disorganized church.

But if the church were disorganized and a restoration needed, no

possible combination of the several disorganized parts would or
could effect it. The disorganization could not take place without the
loss by the whole organism of the organic centre, and that, once
lost, can be recovered only by an original creation, by our Lord, of
a new church organism, which, even if it were done, would not
restore us the Catholic Church; for it would not be a church
existing uninterruptedly from the beginning, and there would be a
time since the Incarnation when it did not subsist, and when there
was no church. The author assumes that unity may be evolved
from "multiplexity," which is Protestant, not Catholic philosophy;
without unity there can be, as we have said, no multiplicity, as
without the universal there can be no particulars. The universal
precedes the particulars and generates them, and when it goes
they go with it. Unity precedes multiplicity, and produces, sustains,
and directs it. This is implied in every argument used, or that can
be used, by philosophers and theologians, to prove the existence of
God and his providence. Atheism results from the assumption that
multiplicity may exist by itself, independently of unity; pantheism,
from the assumption that unity is a dead, abstract unity, like that of
the old Eleatics, not a concrete, living, and effective unity, and the
denial that unity creates multiplicity. Physiology is refuting both by
its discoveries, confirming what has always been affirmed in
principle by traditional science, the fact that there is no organism or
living body, in either the animal or vegetable world, without the
original central cell, born of ancestors, which creates or generates
and directs all the vital phenomena, normal or abnormal, of the
organism, as has already been stated, thus placing science and the
teaching of the church in harmony.
Dr. Ewer probably does not in his own mind absolutely deny the
present unity on which depends the catholicity of the church; but
he supposes it is in some way involved in multiplicity, so that it
needs not to be created, but to be evoked from the existing
"multiplexity" which now obscures it and prevents its effectiveness.
But this we have shown is not and cannot be the case, because the
unity not only produces, but directs or governs the manifold
phenomena of the church, and must therefore be always distinct
from and independent of them. Also, because so long as unity and
catholicity remain, no disorganization or confusion requiring their
evolution can take place, except in the parts exuded or thrown off
by the organism, severed or exscinded from it, that is, only in what
is outside of the One Catholic Church, and forming no part of the
catholic body. That schismatics and heretics lack unity and
catholicity, is, of course, true; but they cannot obtain either by an
evolution from such organization as they may have retained when
the separation took place, or may have subsequently formed for
themselves, but must do it, if at all, by a return to the organic
centre, where both are and have never ceased to be, on the terms
and conditions the Holy See prescribes.

Dr. Ewer maintains that the Catholic Church is restricted to the

Roman, Greek, and Anglican Churches, and consists in what these
have in common or agree in holding. These, he maintains, have all
failed, have taught and still teach grievous errors, set up false
claims, and one or more of them at least have fallen into
superstitious practices; yet he contends that the universal church
has not failed. But as the universal church has no organic existence
independent of these, has no organs of speech or action, no
personality but in them, how, if he is right in his theory, can he
maintain that the whole church has not failed? If he held that the
unity and catholicity of the church were in the central or organic
see, he might hold that particular churches have failed, and that
the One Catholic Church has not failed. Then he could assert, as
we do, that the organism remains, acts, teaches, and governs
through its own infallible organs, in its own individuality, or the
supreme pontiff who is its personality or person. But on his theory,
the failure of each of the three parts which comprise the whole
church, it seems to us, must carry with it the failure of the whole.

Dr. Ewer's difficulty would seem to grow out of his wish to be a

Catholic and remain in the Anglican or Episcopalian communion in
which he is a minister, or to return from Protestantism to Catholicity
without any change of position. This would be possible, if holding,
on private judgment, Catholic dogmas, and observing, on no
authority, Catholic forms of worship, constituted one a member of
the Catholic Church. But he should understand that what he wishes
is impracticable, and that all his efforts are labor lost. So long as a
man is in a communion separated from the present actual living
unity of the church, he can become a Catholic only by leaving it, or
by its corporate return to the Holy See and submission to the
supreme pontiff.

A corporate return is practicable in the case of the Eastern

churches; but even in them the individual who is personally aware
of the schismatic character of his church should abandon it for
unity at once without waiting for its corporate return; but in the
case of Anglicans and Episcopalians, as in the case of all Protestant
communions, the return must be individual and personal.

We are surprised at Dr. Ewer's statement that the Greek Church has
no cultus of St. Mary ever-Virgin, as we are that many Anglicans,
like Dr. Pusey, who object to the Roman Church on account of that
cultus, should seek communion with the schismatic Greeks, with
whom that cultus is pushed to an extreme far beyond anything in
the Latin Church. The truth is, that all that offends Protestants in
the Church of Rome, except the papacy, exists in even a more
offensive form in the Greek schismatic Church. The schismatic
Greek bishops exercise over their flocks a tyranny which is
impracticable in the Roman Church, where the papal authority
restricts that of bishops and tempers their administration of their
dioceses.
But it is time to bring our remarks to a close. We are unable to
recognize the Catholicity we profess in that proposed by Dr. Ewer.
The one holy catholic apostolic church he sets forth does not
appear to us to be the church of the fathers, nor the one of which
we are an affectionate if an unworthy son. In our judgment, Dr.
Ewer is still in the Protestant family, and following private judgment
as his rule, though denouncing it. He has not grasped the central,
or, as Dr. Channing would say, the "seminal" principle of the
Catholic Church. Yet he seems to be well disposed, and to be
seeking it, and has made large strides toward it. We think his
discourses are not only brilliant, bold, and energetic, but fitted to
have great influence in turning the public mind toward the Catholic
Church. We have given our reasons why we do not admit that he
has as yet found that church; but still we think his discourses will
help many to find it, though he himself may not find it. He has, as
yet, strong prejudices against the Church of Rome, and is
undeniably anti-papal. But still there can be no doubt that he would
like to feel himself a Catholic, and have done once for all with
Protestantism. Dr. Ewer stands not alone. There are large numbers
in his communion, and other Protestant communions, who think
and feel as he does, who, from the top of Mount Pisgah, have
obtained—if not the clear vision Moses had—at least some
glimpses, more or less confused, of the promised land, and are
attracted and charmed by what they see of it. We have a feeling of
respectful tenderness toward these men, and of great sympathy
with their trials and struggles. While we must tell them the truth in
firm and manly tones, treat them as men, not as children, we
would, on no account, say or do anything to wound their
susceptibilities, or to create an impression that we do not take a
deep and lively interest in their efforts to arrive at Catholicity. The
spirit is working in them to bring them to light and life. They are
not against us, and to some extent are with us. We would, for their
sakes, they were wholly with us, and we never cease to pray God
that they may find the haven of security and rest it has pleased
him that we should find for ourselves. We once were one of their
number, thought and felt with them, struggled with them, and we
can have for them only words of encouragement and hope. In what
we have said we have had only the desire to assist them to find
and understand the One Holy Catholic Apostolic Church.
 The Evening Primrose.

Of twilight and fresh dews,

Most odorous flower, thou art the child;
Adorning evening's pensive hues
With splendors mild.

A vesper acolyte,
Born, but for this one night,
To swing thy golden censer of perfume,
While stars the tranquil firmament illume,
For heaven's delight.

Thy term of service, fleet,

Creative wish doth meet;
A swift existence; but which this rare grace
Of ceaseless worship, filling life's brief space,
Crowns as complete.

Thy blissful vigil keep,

Rapt flower, while others sleep:
Adoring angels claim thee from above
A dear companion in their task of love;
And I would fain present,
With worshipful intent,
Thy dewy blossoms on my evening shrine;
A contrite homage; sighing to repair,
With the accepted incense of thy prayer,
For sloth like mine.
 Heremore-Brandon; Or,
The Fortunes Of A Newsboy.

"Howe'er it be, it seems to me

'Tis only noble to be good;
Kind hearts are more than coronets,
And simple faith than Norman blood."

Chapter I.

Four little boys: two of them had soft fair hair, and were dressed in
the finest cloth; the other two had very bushy heads, and were
dressed in whatever they could get. It was early Christmas
morning, and the two rich boys were sitting by the window of a
handsome brown-stone house, and they had each a stocking plump
full of dainties; the two poor boys were calling the morning papers
on the stone-cold sidewalk, and if they had any stockings at all,
you may be very sure they were plump full of holes.

"An't he funny," remarked the smaller of the two in the house,

looking at the larger of the two in the street; "an't he too funny!"
And between laughing and eating, little Fred came near choking
himself. "See his old coat, Josie, it trails like Aunt Ellie's blue dress!
And such a queer old hat; don't it make you laugh, Josie?"

"I have seen so many of 'em," explained Josie.

"What are you laughing at, Fred," asked their sister Mary, coming
up to them.

"Those newsboys," he answered, and imitated their "Times, 'Erald,

Tribune! Here's the 'Erald, Times, Tribune!" so perfectly that their
father thought it was a real newsboy calling, and cried out to them
from another room to "hurry up and bring a Herald," at which
command the children rushed eagerly into the hall, and tugged
with their united strength to open the doors, each anxious to be
the first to speak to the odd-looking newsboys, and also to be the
fortunate one to take the paper to their father. In the mean time,
the two newsboys had not been unmindful of the faces behind the
plated window.

"I say, Jim," said the big boy, who was about twelve or thirteen
years old, "did you ever see the beat of that young 'un there? Don't
you choke yerself, youngster, f'fear you'd cheat a friend from doing
that same when you're growed up.—Ere's the 'Erald! Tribune!
Times!—George! Jim, I wish to thunder there'd some new papers
come up. An't yer tired allers a hollerin' out them same old tunes?
—Times! 'Erald! Tribune!—How d'ye s'pose a feller'd feel to wake up
some of these yere mornin's in one o' them big houses?"

"Heerd tell of stranger things 'n that, Dick," replied Jim, who read
the weekly papers. "'Turn again, Whittington, Lord-Mayor of
London,' as the cat said! Turned out true, too."

"You'd better get a cat, Jim, you're such a stunnin' feller; shouldn't
wonder if you'd turn out alderman some of these days!" At which,
for no apparent reason, Dick laughed until every rag was fluttering.

"They wants a paper; better 'tend to yer business," answered Jim,

at which the other newsboy instantly grew grave, and, shuffling his
old shoes across the street, mounted the steps where the children
were waiting and calling for him.

"I want a New York Herald," said Fred very grandly.

"Han't got no 'Eralds," answered the newsboy.

Fred rushed into the house saying, "His Heralds are all gone."
"Tribune, then, and don't keep the door open," instructed the rough
voice from some invisible spot. Mary shut the door all but a little
crack. "Papa wanted a Herald," she said; "you ought to have one
when my papa wants it."

"Thought I had, but couldn't help it; 'Erald's got a great speech to-
day, and I've sold 'em all."

"Do you sell papers every day?" Mary asked.

The bushy head made a sort of bow, as the poor newsboy looked
at the fair-haired little girl on the stoop, who condescended to
question him.

"Yes, miss," he answered, "since ever I wasn't bigger'n a

grasshopper."

"An't he funny?" said Fred.

"Don't you get tired?" asked Mary.

"Well, I can't say I doesn't, 'specially sometimes."

"An't you glad it's Christmas?" Josie asked, as questions seemed

the fashion.

"I kinder am," replied the newsboy.

"Did you have many presents?" questioned Mary.

"Me? Bless you, who'd give 'em to me, miss?"

"Didn't you hang up your stocking last night?" Fred asked.

The newsboy seemed much amused at the question; for it was

plain that he could hardly keep from laughing right out.
"Well, no, I didn't," he answered. "Don't think things would stick in
one long, if I did!"

"Do you put your money in a savings bank? By and by you'd have
enough to build a house maybe, if you were careful," said Josie.

"Jim and me likes takin' it out in eatin' best," answered Dick.

"Why don't you bring me that paper?" cried their father's voice.
And the two boys ran hastily into the house.

"You may have my candy," said Mary in a stately way. "I can have
plenty more." And she put her store of dainty French candy into
the boy's hand, and, while he was still looking at her in
amazement, followed her brothers into the house and shut the
door.

"Just you pinch me, Jim," Dick said, joining his companion. "Drive
in hearty, now. An't I asleep?"

"Well, I dunno; what yer got there?"

"She give it to me."

"Who's that?"

"Her on the steps; didn't you see her?"

"You tell that to the marines! Guess you took it."

"No, I didn't," Dick said indignantly. "I never took nothin' as warn't
mine yet."

"Let's have a look," said Jem, reaching out his hand for the
package; but Dick would not let him touch it. "I'm going to keep it
always to remember her," he said.
"Guess you want ter eat it yerself," Jem said. "I wouldn't be so
mean."

"I an't gen'rally called mean," Dick answered with great dignity.

"Don't you wonder, Jim," said Dick, as they made friends and
passed on—"don't it seem curious how some folks is rich and purty
like them there, and others is poor and ugly like me and you, Jim?"

"George! speak for yerself, if ye like. Guess I'd pass in a crowd, if

I'd the fine fixin's!"

"S'posin' me and you had dandified coats and yeller gloves, and the
fixin's to match, s'pose anybody'd know we was newsboys?" Dick
asked thoughtfully.

"I rayther think," said Jim, "we'd be a deal sight handsomer'n

some of them chaps as has 'em now."

"Let's save our money and try it, Jim."

"'Nuff said," answered Jim, laughing. And the newsboys in their

queer garments, and with their light hearts, passed out of sight of
Mr. Brandon's brown-stone house and fair-haired children.

But not out of all remembrance. The children had a party that
Christmas afternoon; and when they were tired of romping, and
were seated around the room, the girls playing with their dolls; the
Catholic ones telling the others in low voices about the flowers and
lights, and the wonderful manger which they had seen at Mass that
morning; and the boys eagerly listening to the stories of faraway
lands, which one of the older people was telling, little Mary knelt in
an arm-chair, and looked out of the window at the people hurrying
through the driving rain and snow, and at the street-lamps glaring
through the wet and cold. Her kind little heart had been very light,
and a strange joyousness had surrounded her all day, making her
more gentle than ever, so that she had not spoken one hasty word,
or once hesitated to take the lowest part in any of the plays.
Though she did not know it, the little infant Jesus had smiled on
her that morning when she was kind to the poor, homeless
newsboy; and now she understood—for charity had enlarged her
mind—more distinctly than she ever had before, that there were
many cold and desolate children for whom there were no earthly
glad tidings that day, yet who were as much God's own as the little
ones grouped around her father's pleasant parlors. Then, just as
she did the best she could, and prayed in her heart for the children
of the poor, she thought she saw the newsboy to whom she had
spoken in the morning standing close to the railing by the window;
but before she could be sure of it, the servant lighted the gas; she
heard the children calling her for a new game, and she ran lightly
away. But there was one crouched in the cold outside, who
wondered at the sudden light and glow within; and as the
bewildered newsboy saw her dancing past the lighted windows, it
seemed to him that it was not so far, after all, to the heaven and
the angels of whom he had heard; for the "glad tidings" had come
to Dick, even Dick, and they woke up the good, the will to do right,
which is in every heart, and which did not sleep again in him, even
when the little, uncared-for, outcast head rested on the stone steps
that Christmas night.

Chapter II.

Very little idea had poor Dick of right or wrong. No fond mother
took him to her heart when he was a toddling wee one, just big
enough to half understand, and between her kisses told him of
angels and saints, of heroes and martyrs, and of that Queen
Mother up in heaven, dearer than them all, who never forgot those
who once had loved her, and of the beautiful world with its flowers
and fruits, its great rivers and high mountains, its delicious green
and its glorious blue, which a good Father had given to men for
their enjoyment. No loving sister, with bright eyes and tender voice,
tossed him in her strong young arms, and sang to him how knights
and warriors, the great and good of earth, and loved of heaven,
had all been children once like him, only never half so sweet and
dear. No noble father, true in the midst of trials, ever watched with
anxious care that those little feet should walk only in the straight
and narrow path. So it was a hard thing for poor Dick, when he
rubbed his brown hands through his bushy, uncombed hair the next
morning, and pushed the worn old hat over his still sleepy eyes, to
know just what to do to find the temple of Fortune. At times,
though, he had followed the crowd of noisy boys and girls whom
you may see around the doors of any Catholic church at about nine
o'clock on Sunday mornings, and had listened with a critical air, and
slightly supercilious, from some dark corner near the door, to the
talking and the prayers which he did not wholly understand, but
portions of which he did once or twice take into his "inner
consciousness" and fully approve. In some way, he then seemed to
feel that which made him less rough in all his answers, readier in
all his responses to the call for papers, not always gently called for;
and, though he knew not why, there were fewer wicked words on
his lips that day than for many a day before.

It happened that he kept his eyes open and grew thoughtful, and
did not forget his wish to be better; so that, from being a newsboy,
he became an errand-boy in a book-store, where he learned to be
honest and to tell the truth, which was a rapid advance in his
education; for you know it is more than some people have learned
who have lived to be six times Dick's age. Sometimes a little lady
came to that very store to choose her picture-books and Christmas
stories; and it was his place to open the door for her; or perhaps
some one would call out, "Dick, a chair for this lady," and then he
was as happy as a prince. Sometimes he would be sent home with
her purchases, and mounted the steps, entered her father's house,
and always felt "good" again; for always the same picture of a little
girl in blue, with fair hair, and her hands full of dainty French
candy, and a ragged newsboy, dirty and amazed, would be there
before him.
Christmas had come and gone more than once, and it was coming
again, when Dick turned up the gas in a mere closet of a room
very high up in a dingy boarding-house, and made a ghost of a fire
in an old rusty stove. It wouldn't seem to us a very enlivening
prospect; for the room was but slightly furnished, and the stove
smoked, while the wind beat at the not overclean windows, on
which there were no curtains to shut out the dark and cold. But
Dick seemed to think it something very luxurious; for he rubbed his
hands before the blue apology for a flame, and sat down on the
broken wooden stool, with as much zest as that with which I have
seen grand people sink into a great arm-chair after a walk.

"Christmas eve again," he said to the fire, for it was his only
companion. "Let me look at you, Mr. Coals, and see what pictures
you have for me to-night. How many nights, worse nights than this,
I have been glad to crouch under an old shed, or in some alley,
and now to think, thanks to the good God, I have a fire of my own!
Poor little bare feet on the icy pavement to-night, I wish I had you
round my jolly old stove. When I am rich, I will!" Then he laughed
at the idea. "But I won't wait until I am rich, or I would never
deserve to have the chance."

"How are you, Dick?" said a cheery voice, though deep and rough,
at the door. And a man came into the room, which either his figure,
or his coat, or his voice, or the flute under his arm, seemed to fill
to such an extent that the very corners were crowded.

"How are you, Dick? It's blowing a hurricane outside, and you're as
cold as Greenland here. It may do for you, but not for me; old
blood is thin, my boy, old blood is thin." At which Dick laughed
heartily, while putting more coal on the fire; for Carl Stoffs was in
the prime of life, hale and hearty, weighing at least two hundred
pounds, I am sure, and with a round face, very red, but also very
solemn, for Carl Stoffs was a German every inch of him. The stove
grew very red also under his vigorous hands; but whether from
anger or by reflection I will not attempt to say. "And now," he said,
seating himself on the wooden chair, Dick having given it up to his
guest, while he occupied a box instead—"and now, how are you,
boy? Ready for merry Christmas, eh? You'll come to us to-morrow,
so says my wife. In America, you all do mind your wives; mine tells
me to bring you."

"Then I must, I know," Dick said, looking at the other, who was
near three times his size. "I would have a poor chance in opposing
you!" But Carl Stoffs knew well how gratefully the friendless boy
accepted the thoughtful invitation.

"Now, shall we have some music," said he, as he drew out his
flute, and, without waiting an answer, put it to his mouth, and
brought forth such rich, full tones from the instrument that Dick, as
he stared at the now bright fire, seemed to be in a land of
enchantment.

"You are the only man, from the queen of England down, whom I
really envy," said Dick, in one of the pauses. "You can have music
whenever you wish it; I am only a beggar, grateful for every note
thrown in my way. Were you out, last night?"

"Yes, all night in Fourteenth street, at the rich Brandons. Madam is

very gay, this winter."

"I wish I were a musician," said Dick. "It must be jolly to see all
the dancing and the bright dresses!"

"And the pretty ladies, eh? who don't mind you no more than if you
were a stick or a stone. Indeed, my boy, you'd soon get tired of it;
it seems so grand at first, the beautiful picture all in motion; but
your eyes—they ache after a little. Too much light, my boy, too
much light." And the musician went long journeys up and down his
wonderful flute before he spoke again. "They'll go music-mad over
some fool at the piano; but you play until your own music makes
you wild, and never one thinks or cares about you. Last night, I
played only for one. She was always dancing, and she seemed to
go on the wings of the music just as it said to her go. I was not
tired last night."

Awaiting no answer, he turned again to his flute, and all through

the dingy, crowded house rang a joyous "Gloria in excelsis."
Rough captives of labor heard it, and answered to it, knowing well
the glad tidings, the most glorious ever sung, and yet sung to kings
and shepherds alike. The old sinners heard it, and thought of the
strange days when even they were young and innocent.

"Finis," cried the German, rising slowly, and putting on his shaggy
overcoat. "I promised my wife that I would be home at nine, and,
as do all the people here, I mind my wife; but it is one
inconvenient thing. You will come to us after Mass, to-morrow?"

"You are too good to me. When I am rich, perhaps I shall know
how to thank you."

"You should think yourself rich now. You are young; there is no
riches like that."

"I wish I were older, though," sighed Dick.

"Never say that, never, never. The poorest youth is better than the
richest age," said the German earnestly. I shouldn't wonder if Mr.
Stoffs had just found his first gray hair, and was speaking under its
influence. At all events, he did not convince Dick, who said, with
equal earnestness and more quickness:

"I must say it: every day seems too long, every hour goes too
slowly, until I can get at my life's work. This waiting for it kills me."

"My friend, do you call this waiting?" laughed the German. "Was it
waiting and doing nothing that changed you from—"

"But think," interrupted Dick, "of what ought to have-been. Some

day—some day I will get my hand to the plough, you'll see! At
least," a little ashamed of the seeming conceit, "I hope you will."

"And what makes you care?"

"I think it's born in us all to like to be active—to be doing

something. Indeed, it's about the only legacy my poor parents left
me. It may be, for I know nothing of them, that they were just the
same as other people, out of whom bitter poverty has taken all
pride and ambition; but I can't think it, somehow."

"Do you really know nothing of them?"

"Nothing. I have a little sealed box, with an injunction on the

outside of it that I am not to open it until I am of age. I don't
know where I first got it, nor from whom it came. It may be some
trick to tease me for years, and disappoint me at last, for all I
know; and still I have always kept it, for it is all I have. And I think
it came from them."

"It may tell you something wonderful," said his visitor, laughing. For
it was easy for him to understand that some young mother, who
even in her poverty had found the means of reading and believing
stones of princes in disguise, and countesses in cellars, disowned
and disinherited, all for true love's sake, had made a mystery of
leaving a lock of her hair, and perhaps a cheap wedding-ring, to
her boy; and he could not forbear a little ridicule of the folly. "It
may tell you something wonderful. If it gives you possession of half
of New York, don't forget your friends, will you, Dick?" And then,
buttoned up to his chin, and with his cap covering half his face,
and looking just like Santa Claus, Carl Stoffs bundled his cherished
flute under his arm, and obediently went home to his wife.

Dick lingered a moment, after he left, before closing the door. The
room was not wholly his own; but his companion had a father and
a mother in New Jersey, and he had gone home to them, with
something in his pockets for the children's Christmas; so for that
night Dick was in undisputed possession. The passages were dark
and cold; the snow had got through some of the broken windows,
and lay in several little hills on the entry floor; the sash rattled, and
Dick shivered, as he stood irresolute at the door of his room. But
the irresolution did not last long. He bundled up, as well as his
scanty wardrobe permitted, closed the door firmly behind him, and
went down the creaking, broken stairs, and through the dreary
passages, where he could see the snow huddling up to the dark
window-panes, as if it were a white bird trying to get in and
beating its wings against the dirty glass. Dick had not far to walk,
after leaving the house, before he found that which he had come
out to find—somebody without a shelter from the storm. And I
should not wonder if any night, however bitter and cold, that you
or I should take a notion to go out on the same errand, we should
not have to go far for equal success, and that even if we started
from the most delightful dwelling-place in all New York.

Under the remains of some broken steps, or more truly by the side
of them, for they were too broken to shelter a kitten, two dark
figures were lying close together. In one of the pauses of the
storm, when the street-lamp had a chance to shine a little, Dick
could see that the figures were those of two boys asleep. He did
not wait long to rouse them. One woke up at once, cross, and, if I
must tell the truth, with some very wicked words on his lips.

"Get up, and come with me," said Dick.

"What yer want 'long o' me? I an't doing nothin'," he muttered.

"I know that; but I will give you a better place to sleep in. Come."

Bad words again. "I an't done nothin' to you. Le' me 'lone."

"I want you to come home with me. Did you ever hear of a
newsboy called Big Dick? That's me."

"I an't afeard o' nothin'. H'ere goes!" And the poor little fellow, still
believing the other was "chaffing," got on his feet. "Do you want
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com