Nonnegative Matrix and Tensor Factorizations

Applications to Exploratory Multi way Data

Analysis and Blind Source Separation 1st Edition
Andrzej Cichocki download
https://ebookultra.com/download/nonnegative-matrix-and-tensor-
factorizations-applications-to-exploratory-multi-way-data-
analysis-and-blind-source-separation-1st-edition-andrzej-
cichocki/

Explore and download more ebooks or textbooks

at ebookultra.com
 We believe these products will be a great fit for you. Click
the link to download now, or visit ebookultra.com
to discover even more!

Multi way Analysis Applications in the Chemical Sciences

1st Edition Age Smilde

https://ebookultra.com/download/multi-way-analysis-applications-in-
the-chemical-sciences-1st-edition-age-smilde/

Making Sense of Data I A Practical Guide to Exploratory

Data Analysis and Data Mining 2nd Edition Glenn J. Myatt

https://ebookultra.com/download/making-sense-of-data-i-a-practical-
guide-to-exploratory-data-analysis-and-data-mining-2nd-edition-glenn-
j-myatt/

Exploratory Data Analysis with MATLAB 1st Edition Wendy L.

Martinez

https://ebookultra.com/download/exploratory-data-analysis-with-
matlab-1st-edition-wendy-l-martinez/

Exploratory Data Mining and Data Cleaning 1st Edition

Tamraparni Dasu

https://ebookultra.com/download/exploratory-data-mining-and-data-
cleaning-1st-edition-tamraparni-dasu/
Technologies for Migration and Commuting Analysis Spatial
Interaction Data Applications Premier Reference Source
First Edition John Stillwell
https://ebookultra.com/download/technologies-for-migration-and-
commuting-analysis-spatial-interaction-data-applications-premier-
reference-source-first-edition-john-stillwell/

Tensor Analysis 1st Edition Heinz Schade

https://ebookultra.com/download/tensor-analysis-1st-edition-heinz-
schade/

Tensor Operators and their Applications 1st Edition Arif

Salimov

https://ebookultra.com/download/tensor-operators-and-their-
applications-1st-edition-arif-salimov/

Data Analysis and Applications 2 1st edition Edition

Bozeman

https://ebookultra.com/download/data-analysis-and-applications-2-1st-
edition-edition-bozeman/

Fundamentals of Matrix Analysis with Applications 1st

Edition Edward Barry Saff

https://ebookultra.com/download/fundamentals-of-matrix-analysis-with-
applications-1st-edition-edward-barry-saff/
Nonnegative Matrix and Tensor Factorizations
Applications to Exploratory Multi way Data Analysis and
Blind Source Separation 1st Edition Andrzej Cichocki
Digital Instant Download
Author(s): Andrzej Cichocki, Rafal Zdunek, Anh Huy Phan, Shun-ichi Amari
ISBN(s): 9780470746660, 0470746661
Edition: 1
File Details: PDF, 14.94 MB
Year: 2009
Language: english
NONNEGATIVE MATRIX
AND TENSOR
FACTORIZATIONS
NONNEGATIVE
MATRIX AND
TENSOR
FACTORIZATIONS
APPLICATIONS TO EXPLORATORY
MULTI-WAY DATA ANALYSIS AND
BLIND SOURCE SEPARATION

Andrzej Cichocki
Laboratory for Advanced Brain Signal Processing, Riken Brain Science Institute, Japan; and Warsaw
University of Technology and Systems Research Institute, PAN, Poland

Rafal Zdunek
Institute of Telecommunications, Teleinformatics and Acoustics, Wroclaw University of Technology,
Poland; and Riken Brain Science Institute, Japan

Anh Huy Phan

Laboratory for Advanced Brain Signal Processing, Riken Brain Science Institute, Japan

Shun-ichi Amari
Research Unit for Mathematical Neuroscience, Riken Brain Science Institute, Japan
This edition first published 2009
© 2009 John Wiley & Sons, Ltd

Registered office
John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply for permission to
reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright,
Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any
form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK
Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available
in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and
product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective
owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed
to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding
that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is
required, the services of a competent professional should be sought.

MATLAB® MATLAB and any associated trademarks used in this book are the registered trademarks of The
MathWorks, Inc.

For MATLAB® product information, please contact:

The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA, 01760-2098 USA
Tel: 508-647-7000
Fax: 508-647-7001
E-mail: [email protected]
Web: www.mathworks.com

Library of Congress Cataloging-in-Publication Data

Nonnegative matrix and tensor factorizations : applications to exploratory multiway data analysis
and blind source separation / Andrzej Cichocki . . . [et al.].
p. cm.
Includes bibliographical references and index.
ISBN 978-0-470-74666-0 (cloth)
1. Computer algorithms. 2. Data mining. 3. Machine learning 4. Data structures (Computer science)
I. Cichocki, Andrzej.
QA76.9.A43N65 2009
005.1–dc22 2009016049

A catalogue record for this book is available from the British Library.

ISBN: 978-0-470-74666-0

Set in 9/11 pt Times by Thomson Digital, Noida, India

Printed in Singapore by Fabulous
Contents

Preface xi

Acknowledgments xv

Glossary of Symbols and Abbreviations xvii

1 Introduction – Problem Statements and Models 1

1.1 Blind Source Separation and Linear Generalized Component Analysis 2
1.2 Matrix Factorization Models with Nonnegativity and Sparsity Constraints 7
1.2.1 Why Nonnegativity and Sparsity Constraints? 7
1.2.2 Basic NMF Model 8
1.2.3 Symmetric NMF 9
1.2.4 Semi-Orthogonal NMF 10
1.2.5 Semi-NMF and Nonnegative Factorization of Arbitrary Matrix 10
1.2.6 Three-factor NMF 10
1.2.7 NMF with Offset (Affine NMF) 13
1.2.8 Multi-layer NMF 14
1.2.9 Simultaneous NMF 14
1.2.10 Projective and Convex NMF 15
1.2.11 Kernel NMF 16
1.2.12 Convolutive NMF 16
1.2.13 Overlapping NMF 17
1.3 Basic Approaches to Estimate Parameters of Standard NMF 18
1.3.1 Large-scale NMF 21
1.3.2 Non-uniqueness of NMF and Techniques to Alleviate the Ambiguity Problem 22
1.3.3 Initialization of NMF 24
1.3.4 Stopping Criteria 25
1.4 Tensor Properties and Basis of Tensor Algebra 26
1.4.1 Tensors (Multi-way Arrays) – Preliminaries 26
1.4.2 Subarrays, Tubes and Slices 27
1.4.3 Unfolding – Matricization 28
1.4.4 Vectorization 31
1.4.5 Outer, Kronecker, Khatri-Rao and Hadamard Products 32
1.4.6 Mode-n Multiplication of Tensor by Matrix and Tensor
by Vector, Contracted Tensor Product 34
1.4.7 Special Forms of Tensors 38
1.5 Tensor Decompositions and Factorizations 39
1.5.1 Why Multi-way Array Decompositions and Factorizations? 40
1.5.2 PARAFAC and Nonnegative Tensor Factorization 42
1.5.3 NTF1 Model 47
vi Contents

1.5.4 NTF2 Model 49

1.5.5 Individual Differences in Scaling (INDSCAL) and Implicit Slice Canonical
Decomposition Model (IMCAND) 52
1.5.6 Shifted PARAFAC and Convolutive NTF 53
1.5.7 Nonnegative Tucker Decompositions 55
1.5.8 Block Component Decompositions 59
1.5.9 Block-Oriented Decompositions 62
1.5.10 PARATUCK2 and DEDICOM Models 63
1.5.11 Hierarchical Tensor Decomposition 65
1.6 Discussion and Conclusions 66
Appendix 1.A: Uniqueness Conditions for Three-way Tensor Factorizations 66
Appendix 1.B: Singular Value Decomposition (SVD) and Principal Component
Analysis (PCA) with Sparsity and/or Nonnegativity Constraints 67
1.B.1 Standard SVD and PCA 68
1.B.2 Sparse PCA 70
1.B.3 Nonnegative PCA 71
Appendix 1.C: Determining a True Number of Components 71
Appendix 1.D: Nonnegative Rank Factorization Using Wedderborn Theorem – Estimation
of the Number of Components 74
References 75

2 Similarity Measures and Generalized Divergences 81

2.1 Error-induced Distance and Robust Regression Techniques 82
2.2 Robust Estimation 84
2.3 Csiszár Divergences 90
2.4 Bregman Divergence 96
2.4.1 Bregman Matrix Divergences 103
2.5 Alpha-Divergences 104
2.5.1 Asymmetric Alpha-Divergences 104
2.5.2 Symmetric Alpha-Divergences 110
2.6 Beta-Divergences 112
2.7 Gamma-Divergences 116
2.8 Divergences Derived from Tsallis and Rényi Entropy 118
2.8.1 Concluding Remarks 119
Appendix 2.A: Information Geometry, Canonical Divergence, and Projection 120
2.A.1 Space of Probability Distributions 120
2.A.2 Geometry of Space of Positive Measures 123
Appendix 2.B: Probability Density Functions for Various Distributions 125
References 127

3 Multiplicative Iterative Algorithms for NMF with Sparsity Constraints 131

3.1 Extended ISRA and EMML Algorithms: Regularization and Sparsity 132
3.1.1 Multiplicative NMF Algorithms Based on the Squared Euclidean Distance 132
3.1.2 Multiplicative NMF Algorithms Based on Kullback-Leibler I-Divergence 139
3.2 Multiplicative Algorithms Based on Alpha-Divergence 143
3.2.1 Multiplicative Alpha NMF Algorithm 143
3.2.2 Generalized Multiplicative Alpha NMF Algorithms 147
3.3 Alternating SMART: Simultaneous Multiplicative Algebraic Reconstruction Technique 148
3.3.1 Alpha SMART Algorithm 148
3.3.2 Generalized SMART Algorithms 150
Contents vii

3.4 Multiplicative NMF Algorithms Based on Beta-Divergence 151

3.4.1 Multiplicative Beta NMF Algorithm 151
3.4.2 Multiplicative Algorithm Based on the Itakura-Saito Distance 156
3.4.3 Generalized Multiplicative Beta Algorithm for NMF 156
3.5 Algorithms for Semi-orthogonal NMF and Orthogonal Three-Factor NMF 157
3.6 Multiplicative Algorithms for Affine NMF 159
3.7 Multiplicative Algorithms for Convolutive NMF 160
3.7.1 Multiplicative Algorithm for Convolutive NMF Based on Alpha-Divergence 162
3.7.2 Multiplicative Algorithm for Convolutive NMF Based on Beta-Divergence 162
3.7.3 Efficient Implementation of CNMF Algorithm 165
3.8 Simulation Examples for Standard NMF 166
3.9 Examples for Affine NMF 170
3.10 Music Analysis and Decomposition Using Convolutive NMF 176
3.11 Discussion and Conclusions 184
Appendix 3.A: Fast Algorithms for Large-scale Data 187
3.A.1 Random Block-wise Processing Approach – Large-scale NMF 187
3.A.2 Multi-layer Procedure 188
3.A.3 Parallel Processing 188
Appendix 3.B: Performance Evaluation 188
3.B.1 Signal-to-Interference-Ratio - SIR 188
3.B.2 Peak Signal-to-Noise-Ratio (PSNR) 190
Appendix 3.C: Convergence Analysis of the Multiplicative Alpha NMF Algorithm 191
Appendix 3.D: MATLAB Implementation of the Multiplicative NMF Algorithms 193
3.D.1 Alpha Algorithm 193
3.D.2 SMART Algorithm 195
3.D.3 ISRA Algorithm for NMF 197
Appendix 3.E: Additional MATLAB Functions 198
3.E.1 Multi-layer NMF 198
3.E.2 MC Analysis with Distributed Computing Tool 199
References 199

4 Alternating Least Squares and Related Algorithms for NMF and SCA Problems 203
4.1 Standard ALS Algorithm 203
4.1.1 Multiple Linear Regression – Vectorized Version of ALS Update Formulas 206
4.1.2 Weighted ALS 206
4.2 Methods for Improving Performance and Convergence Speed of ALS Algorithms 207
4.2.1 ALS Algorithm for Very Large-scale NMF 207
4.2.2 ALS Algorithm with Line-Search 208
4.2.3 Acceleration of ALS Algorithm via Simple Regularization 208
4.3 ALS Algorithm with Flexible and Generalized Regularization Terms 209
4.3.1 ALS with Tikhonov Type Regularization Terms 210
4.3.2 ALS Algorithms with Sparsity Control and Decorrelation 211
4.4 Combined Generalized Regularized ALS Algorithms 212
4.5 Wang-Hancewicz Modified ALS Algorithm 213
4.6 Implementation of Regularized ALS Algorithms for NMF 213
4.7 HALS Algorithm and its Extensions 214
4.7.1 Projected Gradient Local Hierarchical Alternating Least Squares (HALS)
Algorithm 214
4.7.2 Extensions and Implementations of the HALS Algorithm 216
4.7.3 Fast HALS NMF Algorithm for Large-scale Problems 217
viii Contents

4.7.4 HALS NMF Algorithm with Sparsity, Smoothness and Uncorrelatedness

Constraints 220
4.7.5 HALS Algorithm for Sparse Component Analysis and Flexible
Component Analysis 222
4.7.6 Simplified HALS Algorithm for Distributed and Multi-task Compressed Sensing 227
4.7.7 Generalized HALS-CS Algorithm 231
4.7.8 Generalized HALS Algorithms Using Alpha-Divergence 233
4.7.9 Generalized HALS Algorithms Using Beta-Divergence 234
4.8 Simulation Results 236
4.8.1 Underdetermined Blind Source Separation Examples 236
4.8.2 NMF with Sparseness, Orthogonality and Smoothness Constraints 237
4.8.3 Simulations for Large-scale NMF 239
4.8.4 Illustrative Examples for Compressed Sensing 241
4.9 Discussion and Conclusions 249
Appendix 4.A: MATLAB Source Code for ALS Algorithm 252
Appendix 4.B: MATLAB Source Code for Regularized ALS Algorithms 253
Appendix 4.C: MATLAB Source Code for Mixed ALS-HALS Algorithms 256
Appendix 4.D: MATLAB Source Code for HALS CS Algorithm 259
Appendix 4.E: Additional MATLAB Functions 261
References 264

5 Projected Gradient Algorithms 267

5.1 Oblique Projected Landweber (OPL) Method 268
5.2 Lin’s Projected Gradient (LPG) Algorithm with Armijo Rule 270
5.3 Barzilai-Borwein Gradient Projection for Sparse Reconstruction (GPSR-BB) 271
5.4 Projected Sequential Subspace Optimization (PSESOP) 273
5.5 Interior Point Gradient (IPG) Algorithm 275
5.6 Interior Point Newton (IPN) Algorithm 276
5.7 Regularized Minimal Residual Norm Steepest Descent Algorithm (RMRNSD) 279
5.8 Sequential Coordinate-Wise Algorithm (SCWA) 281
5.9 Simulations 283
5.10 Discussions 289
Appendix 5.A: Stopping Criteria 290
Appendix 5.B: MATLAB Source Code for Lin’s PG Algorithm 292
References 293

6 Quasi-Newton Algorithms for Nonnegative Matrix Factorization 295

6.1 Projected Quasi-Newton Optimization 296
6.1.1 Projected Quasi-Newton for Frobenius Norm 296
6.1.2 Projected Quasi-Newton for Alpha-Divergence 298
6.1.3 Projected Quasi-Newton for Beta-Divergence 303
6.1.4 Practical Implementation 305
6.2 Gradient Projection Conjugate Gradient 305
6.3 FNMA algorithm 308
6.4 NMF with Quadratic Programming 310
6.4.1 Nonlinear Programming 311
6.4.2 Quadratic Programming 312
6.4.3 Trust-region Subproblem 314
6.4.4 Updates for A 316
6.5 Hybrid Updates 318
Contents ix

6.6 Numerical Results 319

6.7 Discussions 323
Appendix 6.A: Gradient and Hessian of Cost Functions 324
Appendix 6.B: MATLAB Source Codes 325
References 333

7 Multi-Way Array (Tensor) Factorizations and Decompositions 337

7.1 Learning Rules for the Extended Three-way NTF1 Problem 337
7.1.1 Basic Approaches for the Extended NTF1 Model 338
7.1.2 ALS Algorithms for NTF1 340
7.1.3 Multiplicative Alpha and Beta Algorithms for the NTF1 Model 341
7.1.4 Multi-layer NTF1 Strategy 343
7.2 Algorithms for Three-way Standard and Super Symmetric Nonnegative Tensor
Factorization 344
7.2.1 Multiplicative NTF Algorithms Based on Alpha- and Beta-Divergences 345
7.2.2 Simple Alternative Approaches for NTF and SSNTF 350
7.3 Nonnegative Tensor Factorizations for Higher-Order Arrays 351
7.3.1 Alpha NTF Algorithm 353
7.3.2 Beta NTF Algorithm 355
7.3.3 Fast HALS NTF Algorithm Using Squared Euclidean Distance 355
7.3.4 Generalized HALS NTF Algorithms Using Alpha- and Beta-Divergences 358
7.3.5 Tensor Factorization with Additional Constraints 360
7.4 Algorithms for Nonnegative and Semi-Nonnegative Tucker Decompositions 361
7.4.1 Higher Order SVD (HOSVD) and Higher Order Orthogonal Iteration (HOOI)
Algorithms 362
7.4.2 ALS Algorithm for Nonnegative Tucker Decomposition 365
7.4.3 HOSVD, HOOI and ALS Algorithms as Initialization Tools for Nonnegative
Tensor Decomposition 366
7.4.4 Multiplicative Alpha Algorithms for Nonnegative Tucker Decomposition 366
7.4.5 Beta NTD Algorithm 370
7.4.6 Local ALS Algorithms for Nonnegative TUCKER Decompositions 370
7.4.7 Semi-Nonnegative Tucker Decomposition 374
7.5 Nonnegative Block-Oriented Decomposition 375
7.5.1 Multiplicative Algorithms for NBOD 376
7.6 Multi-level Nonnegative Tensor Decomposition - High Accuracy Compression
and Approximation 377
7.7 Simulations and Illustrative Examples 378
7.7.1 Experiments for Nonnegative Tensor Factorizations 378
7.7.2 Experiments for Nonnegative Tucker Decomposition 384
7.7.3 Experiments for Nonnegative Block-Oriented Decomposition 392
7.7.4 Multi-Way Analysis of High Density Array EEG – Classification of Event
Related Potentials 395
7.7.5 Application of Tensor Decompositions in Brain Computer Interface –
Classification of Motor Imagery Tasks 404
7.7.6 Image and Video Applications 409
7.8 Discussion and Conclusions 412
Appendix 7.A: Evaluation of Interactions and Relationships Among Hidden
Components for NTD model 415
Appendix 7.B: Computation of a Reference Tensor 416
Appendix 7.C: Trilinear and Direct Trilinear Decompositions for Efficient Initialization 418
x Contents

Appendix 7.D: MATLAB Source Code for Alpha NTD Algorithm 420
Appendix 7.E: MATLAB Source Code for Beta NTD Algorithm 421
Appendix 7.F: MATLAB Source Code for HALS NTD Algorithm 423
Appendix 7.G: MATLAB Source Code for ALS NTF1 Algorithm 425
Appendix 7.H: MATLAB Source Code for ISRA BOD Algorithm 426
Appendix 7.I: Additional MATLAB functions 427
References 429

8 Selected Applications 433

8.1 Clustering 433
8.1.1 Semi-Binary NMF 434
8.1.2 NMF vs. Spectral Clustering 435
8.1.3 Clustering with Convex NMF 436
8.1.4 Application of NMF to Text Mining 438
8.1.5 Email Surveillance 440
8.2 Classification 442
8.2.1 Musical Instrument Classification 442
8.2.2 Image Classification 443
8.3 Spectroscopy 447
8.3.1 Raman Spectroscopy 447
8.3.2 Fluorescence Spectroscopy 449
8.3.3 Hyperspectral Imaging 450
8.3.4 Chemical Shift Imaging 452
8.4 Application of NMF for Analyzing Microarray Data 455
8.4.1 Gene Expression Classification 455
8.4.2 Analysis of Time Course Microarray Data 459
References 467

Index 473
Preface

Signal processing, data analysis and data mining are pervasive throughout science and engineering. Extracting
an interesting knowledge from experimental raw datasets, measurements, observations and understanding
complex data has become an important challenge and objective. Often datasets collected from complex
phenomena represent the integrated result of several inter-related variables or they are combinations of
underlying latent components or factors. Such datasets can be first decomposed or separated into the compo-
nents that underlie them in order to discover structures and extract hidden information. In many situations,
the measurements are gathered and stored as data matrices or multi-way arrays (tensors), and described by
linear or multi-linear models.
Approximative low-rank matrix and tensor factorizations or decompositions play a fundamental role in
enhancing the data and extracting latent components. A common thread in various approaches for noise
removal, model reduction, feasibility reconstruction, and Blind Source Separation (BSS) is to replace the
original data by a lower dimensional approximate representation obtained via a matrix or multi-way array
factorization or decomposition. The notion of a matrix factorization arises in a wide range of important ap-
plications and each matrix factorization makes a different assumption regarding component (factor) matrices
and their underlying structures, so choosing the appropriate one is critical in each application domain. Very
often the data, signals or images to be analyzed are nonnegative (or partially nonnegative), and sometimes
they also have sparse or smooth representation. For such data, it is preferable to take these constraints into
account in the analysis to extract nonnegative and sparse/smooth components or factors with physical mean-
ing or reasonable interpretation, and thereby avoid absurd or unpredictable results. Classical tools cannot
guarantee to maintain nonnegativity.
In this book, we provide a wide survey of models and algorithmic aspects of Nonnegative Matrix Factori-
zation (NMF), and its various extensions and modifications, especially Nonnegative Tensor Factorization
(NTF) and Nonnegative Tucker Decomposition (NTD). In NTF and NTD approaches, high-dimensional
data, such as hyper-spectral or medical images, are factored or decomposed directly and approximated by
a sum of rank-one nonnegative tensors. The motivation behind NMF, NTF and NTD is that besides the
dimensionality reduction sought in many applications, the underlying data ensemble is nonnegative and can
be better modeled and interpreted by means of nonnegative and, ideally, sparse or smooth components.
The notions of NMF, NTF and NTD play a major role in a wide range of important applications, including
bioinformatics, micro-array analysis, neuroscience, text mining, image understanding, air pollution research,
chemometrics, and spectral data analysis. Other applications include linear sparse coding, image classifica-
tion, clustering, neural learning process, sound recognition, remote sensing, and object characterization. For
example, NMF/NTF processing permits the detection of alternative or context-dependent patterns of gene
expression in complex biological systems and especially the recovery of meaningful biological informa-
tion from cancer-related microarray data. We believe that a potential impact of NMF and its extensions on
scientific advancements might be as great as the Independent Component Analysis (ICA), or the Singular
Value Decomposition (SVD) and Principal Component Analysis (PCA). In contrast to ICA or SVD/PCA
approaches, NMF/NTF and NTD techniques, if successively realized, may improve interpretability and
visualization of large-scale data while maintaining the physical feasibility more closely.
Researchers from various research fields are interested in different, usually very diverse, aspects of NMF
and NTF. For example, neuroscientists and biologists need reliable methods and techniques which can
xii Preface

extract or separate useful information from superimposed biomedical data corrupted by a large level of
noise and interference; for example, by using non-invasive recordings of human brain activities targeted
at understanding the ability of the brain to sense, recognize, store and recall patterns, and comprehending
crucial elements of learning: association, abstraction and generalization. A second group of researchers,
engineers and computer scientists, are fundamentally interested in developing and implementing flexible
and efficient algorithms for specific practical engineering and scientific applications. A third group of re-
searchers, mathematicians and physicists, have an interest in the development of a fundamental theory to
understand mechanisms, properties and abilities of the developed algorithms, and their generalizations to
more complex and sophisticated models. Interactions between such groups permit real progress to be made
in the very interdisciplinary research devoted to NMF/NTF and NTD, and each group benefits from the
others.
The theory built up around NMF, NTF and NTD is so extensive and the applications so numerous that we
are, of course, not able to cover all of them. Our selection and treatment of material reflects our background,
and our own research interests and results in this fascinating area over the last five years.
The book provides wide coverage of the models and algorithms for nonnegative matrix factorizations and
tensor decompositions both from a theoretical and practical point of view. The main objective is to derive
and implement, in MATLAB, efficient and relatively simple iterative algorithms that work well in practice
for real-world data. In fact, almost all of the algorithms presented in the book have been implemented in
MATLAB and extensively tested. We have attempted to present the concepts, models and algorithms in
general or flexible forms to stimulate the reader to be creative in visualizing new approaches and adopting
methods or algorithms for their specific needs and applications.
In Chapter 1 we describe the basic NMF models and their extensions, and formulate the fundamental
problems related to the calculation of component (factor) matrices. Special emphasis is given to basic
properties and mathematical operations for multi-way arrays, also called multi-dimensional matrices or
tensors. Chapter 1 introduces the basic linear and multi-linear models for matrix factorizations and tensor
decompositions, and formulates the fundamental analytical framework for the solution of the problems
posed in this book. The workhorse is the NMF model for sparse representations of data, and its extensions,
including the multi-layer NMF, semi-NMF, sparse NMF, tri-NMF, symmetric NMF, orthogonal NMF, non-
smooth NMF (nsNMF), overlapping NMF, convolutive NMF (CNMF), and large-scale NMF. Our particular
emphasis is on the NMF and semi-NMF models and their extensions to multi-way models (i.e., multi-linear
models which perform multi-way array (tensor) decompositions) with nonnegativity and sparsity constraints,
especially a family of Tucker and PARAFAC models.
In Chapter 2, we give an overview and discuss properties of a large family of generalized and flexible
divergences or similarity distances between two nonnegative sequences or patterns. They are formulated for
probability distributions used in the development of novel algorithms for NMF and NTF. Information theory,
convex analysis, and information geometry play key roles in the formulation of the divergences. The scope
of these results is vast since the generalized divergence functions and their variants include quite a large
number of useful loss functions, including those based on relative entropies, generalized Kullback-Leibler
or I-divergence, Hellinger distance, Jensen-Shannon divergence, J-divergence, Pearson and Neyman Chi-
squared divergences, triangular discrimination, and arithmetic-geometric Taneya divergence. Many of these
measures belong to the class of Alpha-divergences and Beta-divergences, and have been applied successfully
in disciplines such as signal processing, pattern recognition, probability distributions, information theory,
finance and economics. In the following chapters we apply such divergences as cost functions (possibly with
additional constraints and regularization terms) to derive novel multiplicative and additive projected gradient
and fixed-point NMF/NTF algorithms. They provide working solutions for the problems where nonnegative
latent (hidden) components can be generally statistically dependent, and satisfy some other conditions or
additional constraints such as sparsity or smoothness.
In Chapter 3, we introduce a wide family of iterative multiplicative algorithms for NMF and related prob-
lems, subject to additional constraints such as sparsity and/or smoothness. Although a standard multiplicative
update rule for NMF achieves a sparse representation of its components, we can impose a control over the
Preface xiii

sparsity of the components by designing a suitable cost function with additional penalty terms. In this chapter
we consider a wide class of cost functions or divergences, leading to generalized multiplicative algorithms
with regularization and/or penalty terms. Such relaxed forms of the multiplicative NMF algorithms usually
provide better performance and convergence speed, and allow us to extract the desired components uniquely
up to the scale and permutation ambiguities. As special cases we introduce the multiplicative algorithms for
Alpha and Beta divergences, squared Hellinger, Pearson’s Chi-squared, and Itakura-Saito distances.
In Chapter 4, we derive and give an overview of the Alternating Least Squares algorithms referred to as
the ALS algorithms for NMF, semi-NMF, and multi-layer NMF. This is important as many existing NMF
techniques are prohibitively slow and inefficient, especially for large-scale problems. For such problems,
a promising approach is to apply the ALS algorithms with the regularization and/or extended line search
techniques. Special emphasis in this chapter is put on various regularization and penalty terms together with
local learning rules. By incorporating the regularization and penalty terms into the weighted Frobenius norm,
we show that it is possible to achieve sparse, orthogonal, or smooth representations, thus helping to obtain
a desired global solution. The main objective of this chapter is to develop efficient and robust Regularized
ALS (RALS) algorithms. For this purpose, we use several approaches from constrained optimization and
regularization theory, and in addition introduce several heuristic algorithms. The proposed algorithms are
characterized by improved efficiency and often very good convergence properties, especially for large-scale
problems.
In Chapter 5, we present a wide class of Projected Gradient (PG) algorithms and compare their per-
formance, especially for large-scale problems. In contrast to the multiplicative NMF algorithms discussed
in Chapter 3, the PG algorithms have additive updates, and provide an approximate solution to the Non-
Negative Least Squares (NNLS) problems. Chapter 5 focuses on the following PG algorithms: Oblique
Projected Landweber (OPL), projected gradients with Armijo rule, Barzilai-Borwein gradient projection,
Projected Sequential Subspace Optimization (PSESOP), Interior-Point Newton (IPN), Minimal Residual
Norm Steepest Descent (MRNSD), and Sequential Coordinate-Wise Algorithm (SCWA).
In Chapter 6, we introduce learning algorithms for NMF, using second-order approximations, that is, the
information about the Hessian and gradient of a cost function. Using the information about the curvature of
the cost function, which is intimately related to second-order derivatives, the convergence can be considerably
accelerated. This, however, also introduces many related practical problems that must be addressed prior to
applying learning algorithms. For example, the Hessian must be positive-definite to ensure the convergence of
approximations to a local minimum of a specific cost function. Unfortunately, this is not guaranteed using the
NMF alternating minimization rule, and we need to resort to some suitable Hessian approximation techniques.
In addition, the Hessian matrix may be very large, and of a severely ill-conditioned nature (in particular for
large-scale problems), which gives rise to many difficult problems related to its inversion. Moreover, the
nonlinear projections may be performed in many ways, similarly to the PG and steepest descent algorithms.
This chapter provides a comprehensive study of the solutions to the above-mentioned problems. We also
give some heuristics on the selection of a cost function and the related regularization terms which restrict
the area of feasible solutions, and help the algorithms to converge to the global minimum of a specific cost
function. In particular, we discuss the simplest approach to the projected quasi-Newton optimization using
the Levenberg-Marquardt regularization of the Hessian. We then extend the discussion to more sophisticated
second-order algorithms that iteratively update only the strictly positive (inactive) variables. The example
includes the Gradient Projection Conjugate Gradient (GPCG). Furthermore, as a special case of the second-
order method, we present one Quadratic Programming (QP) method that solves a QP problem using the
trust-region subproblem algorithm. The QP problem is formulated from the Tikhonov regularized squared
Euclidean cost function extended with a logarithmic barrier function to satisfy nonnegativity constraints.
The BSS experiments demonstrate the high efficiency of the proposed algorithms.
In Chapter 7, we attempt to extend and generalize the results and algorithms from the previous chapters
for the NTF, BOD and NTD models. In fact, almost all the NMF algorithms described in the earlier chapters
can be extended or generalized to the various nonnegative tensor factorizations and decompositions formu-
lated in Chapter 1. However, in this chapter we mainly focus on NTF, that is, PARAFAC with nonnegativity
Random documents with unrelated
content Scribd suggests to you:
 This increased sensibility was for evil as well as good. While the river
seemed even lovelier, if possible, than upon the previous journey, side by
side with the pleasure he had in it, a premonition of evil entered Ralph's
breast. "Something is going to happen," a voice whispered to him. He
sought to laugh it away, but it stuck. He could not but remember the stories
that are told in the North of how men living alone in the woods become
gifted with a prescience of what is to come.

With a vague feeling that escape from the danger lay ahead, he paddled
until ten o'clock that night. Darkness was then falling, and his weary arms
could scarcely lift the paddle. He camped on the river in the spot where they
had dined on the second day of the other journey. He fell asleep with the
premonition like a cold hand on his breast.
 "An instant later a long dugout swept into view, with four men in it"

In the morning it awakened him all of a piece. He abruptly sat up to

listen. There was no sound. "What is the matter with me?" he thought
wonderingly. "Something is upon you," that still voice seemed to whisper.
He looked to his gun. His heart failed him a little, he was so terribly alone.
Inside him he offered up an unspoken prayer that whatever was coming
might come quickly, before fear of the unknown should unman him.

Hastily cooking his breakfast, he never ceased to listen; therefore he was

scarcely surprised when he finally heard the most startling sound in the
wilderness—human voices. An instant later a long dugout swept into view
upstream with four men in it. Courage warmed Ralph's breast again; to be
sure it was bad enough, but it was real.

At sight of Ralph the men in the dugout set up a shout. Arriving abreast
of his camp they swung around and beached their craft below. In the bow
was a white man strange to Ralph, Joe Mixer and Stack sat amidships,
while the stern paddle was wielded by a handsome, muscular young half-
breed. They all got out. Ralph awaited them on the top of the bank. Burly
Joe approached with an anticipatory, cynical grin; little Stack kept partly
behind him.

"Hello, pardner!" cried Joe.

Ralph, seeing that he actually expected to keep up the fiction of

friendliness, smiled grimly. "What do you want?" he asked.

Ralph's warning of danger had served him well. Joe, seeing him cool and
prepared, was completely disconcerted. "What do I want?" he repeated,
falling back with a scowl. "That's a hell of a nice good-morning to hand out
to a man!"

"What were you looking for?" asked Ralph, "an address of welcome?"

Joe turned purple, and shook his fist. "I'll show you!" he cried.
 Little Stack stepped from behind Joe. Physical terror gave his face a
greenish cast, but his chagrin at seeing his careful plans about to be
destroyed was stronger still. It emboldened him to put himself in front of
Joe. "Wait!" he implored. "You mustn't quarrel! Let me explain!"

Joe turned aside with a muttered oath.

A fawning note crept into Stack's voice. "We've taken the Doctor by
surprise," he said. "He thinks we're spying on him. You can't hardly blame
him."

"You're a good guesser, Stack," said Ralph grimly.

"It's nothing of the kind!" cried Stack virtuously. "You must remember I
told you long ago I wanted to take a trip through the wilds if I could get a
chance. Mr. Mixer was willing to go, so I engaged him and these men to
guide me."

"Why explain?" said Ralph. "It's nothing to me. The river is free to all."

"I didn't expect this from you," said Stack, with an aggrieved air. "I
thought we were friends. What have you got against me?"

"Nothing," said Ralph; "but you're in bad company."

Joe could no longer hold himself in. His face was purple. "Who the hell
do you think you are?" he cried thickly. "You stinking dude! You smooth-
face poisoner! You rah-rah college boy. It makes my stomach turn to hear
you lisping! What are you doing in a man's country? Go home to your pink
teas and your toe-dancing!"

Ralph could not help but smile at the style of Joe's invective. The smile
maddened Joe. The foulest dregs of English speech were fished up to
express his feelings. The other white man laughed obsequiously. He was in
Joe's pay. The half-breed pitched pebbles into the stream, handsome and
unconcerned. Ralph took it all steely eyed and smiling still.
 "You stand there like a little Gorramighty!" cried Joe, with a string of
oaths. "What can you do against the four of us? We've got you where we
want you now, and you know it! You'll be singing another tune before we're
done with you!"

"Now you're talking!" cried Ralph, bright-eyed. "The truth is coming out
at last!"

Stack all but wrung his hands at the turn things were taking.
"Gentlemen! Gentlemen!" he implored.

"Ahh! shut your head," snarled Joe. "You hate him as much as me!"

Stack turned paler still, and darted a furtive look at Ralph, and cringed
and tried to smile indulgently. "Don't listen to him," he said to Ralph.
"You've made him mad. He don't mean what he says. It wasn't half an hour
ago he said to me, 'Won't it be sport to surprise the Doctor?' There's no need
for you to quarrel like this. We don't want to intrude upon your privacy.
Come to our camp to supper to-night, and talk things over quiet, and shake
hands on it."

Ralph preferred Joe's honest obscenity to this. He made no answer.

"Ah! come on!" said Joe. "I'm sick of your palaver!"

He pulled the smaller man back to the dugout. Stack got in, nodding and
smiling over his shoulder in a comic and pitiable attempt to propitiate the
grim Ralph. They pushed off. As the dugout disappeared around the first
bend below, Stack actually had the effrontery to wave his hand to Ralph.

Ralph sat down to do some hard thinking. His charming dreams were
rudely shattered, and like every man suddenly roused to action, he felt a
little ashamed at having been caught dreaming. He remembered precautions
he might have taken had he been wide awake. When his anger cooled—in
spite of the smile he had been no less angry than Joe Mixer—he was a little
appalled by his situation. Four against one is heavy odds. If he had had even
so much as a dog to keep watch while he slept! How could he venture to
sleep and leave himself open to a night attack? He resolutely put that
unnerving thought out of his head. "I shall travel exactly as if they had not
come!" he decided.

The more he thought, the greater loomed his difficulties. In a manner of

speaking he was trapped in the river just the same as if they had him on a
road between high and unscalable walls. He could not go back against the
current, and he could not leave the river. With his clumsy boat and one
paddle, against their dugout and four, there was not the slightest possibility
of his escaping them downstream. They were free to follow him at their
leisure, and play with him like cat and mouse.

Ralph was amazed, as any open-hearted man might be, at the suddenness
of the discovery that he had active and malignant enemies. Joe Mixer's
hatred he instinctively understood, and returned. Those two had been
formed to hate each other. He likewise understood now that the evil fire
Nahnya had lighted in Joe's breast was no mere ephemeral flame. It was
clear that Joe hoped to reach Nahnya through him. "I'll lead him a chase,"
Ralph thought grimly. This brought up the thought that Joe might be the
means of keeping him from returning to Nahnya. Ralph ground his teeth at
that, and understood the desire to murder that is born in men's breasts.

In Stack Ralph realized he had a more dangerous enemy than Joe. In

vain he threshed his brain to discover a reason for Stack's being in Joe's
galley. He had never laid eyes on the little man until they took their places
in the stage together. It was true he had never thought much of the little
Jackstraw, but there had never been anything but friendly exchanges
between them. There was a mystery here that tantalized him.

The upshot of his cogitations was, Ralph decided to accept Stack's

invitation to visit their camp that night—not to eat with them, Ralph's gorge
rose at the idea, but to go after supper. "It'll surprise 'em," he thought
grimly. "Nothing like bearding them in their own den. I'm bound to find out
something. One man's strength isn't enough against four. I've got to use all
the wits I have, too. I've got to meet them on their own ground, lie for lie.
Beastly crooks! I'll go further than lying if necessary to keep them out!"

All day they remained ahead of him in the river, About nine o'clock,
while it was still fully light, he came upon their camp in the accustomed
camping-place where Nahnya had stopped on the second night of the
previous journey; the spot where Nahnya and Ralph had effected their
midnight reconciliation. There was the little grassy shelf in the bank where
she had lain! The coarse voices of the men above profaned the scene
horribly.

Ralph's face as he climbed the bank was serene. His greeting was as
bland and off-hand as a schoolboy's. The four men were sitting on the
ground playing "jackpot." As Ralph had pleasurably anticipated, their jaws
dropped upon his appearance. Only Stack answered his greeting. Cards in
hand, the little man jumped up obsequiously, but Joe Mixer barked at him,
and he sat down abruptly. Joe scowled at his cards like a hangman. The
game proceeded as if Ralph were not there.

Ralph's cheeks began to burn at the implied insult, but he clapped his
anger under hatches. He saw clearly enough that Joe was waiting for him to
make an opening for a quarrel. Drawing closer, he coolly overlooked the
game. They had a folded blanket between them to play the cards upon. In
lieu of chips they used matches. The half-breed was winning. He was a fine
specimen of physical manhood a year or two younger than Ralph, with a
bold, conceited face. He scarcely took pains to hide his contempt for the
three white men of his party, and Ralph observed that even Joe was inclined
to truckle to him like a bully to one whose strength he has not measured.
Stack was obsequious all around. In the third white man Ralph recognized
Crusoe Campbell, a disreputable character well known up and down the
river of that name. He had the reputation of being not quite right in his
head, which he traded upon to his advantage. His wits were good enough to
play a crafty game of poker.

So much for Ralph's observations. "A rum outfit!" he thought grimly.

When the cards were collected for a fresh deal Ralph asked coolly:
"What are the stakes?"

"Nickel a match," answered Crusoe Campbell.

"Give me the worth of that," he said, throwing a five-dollar bill on the

blanket. "You," he said, indicating the half-breed, "what's your name?"
 "Philippe Boisvert," the breed announced swaggeringly.

Crusoe Campbell and Philippe made room between them and Ralph sat
down. All looked covertly at Joe to see how he would take it. Joe, still
scowling, kept his eyes down and said nothing. The game went on. Ralph's
bluff was as yet uncalled.

Outwardly as cool as the ideal poker-player, Ralph was on the qui vive
for an explosion. Under stress of excitement, his spirits soared like a bird
taking wing. The corners of his lips twitched provokingly, and the shine of a
hidden fire glowed in his dark eyes. He bet recklessly, winning and losing
with equal good humour. His good humour communicated itself to three of
the other players. All men love a good gambler. The ill-assorted game
became almost jolly. Only Joe grew more and more morose. His face turned
an ugly brownish red, and a vein stood out ominously on his forehead.

When the explosion took place it was not directed at Ralph. Stack,
carried away by the appearance of general good feeling, during a pause
while the cards were being shuffled had the misfortune to say, addressing
Joe and Ralph: "You two ought to shake hands and let bygones be
bygones."

Joe Mixer broke out on him so violently as to be almost comic. "You

sneaking little two-faced informer!" he shouted with a whole string of
oaths. "Keep your lip out of my affairs, will you? I'll learn you to talk to
your betters! You make me sick with your lying palaver! Get the hell out of
this game anyway! You ain't man enough to play poker!"

Stack hastily retreated from the circle. The breed laughed. Crusoe
Campbell quietly confiscated Stack's matches.

"Give me another box of cigarettes out of your bag," the breed said
curtly.

"A half-breed issuing orders to a white man and being obeyed!" thought
Ralph.

"Bring up a pail of water from the river," commanded Crusoe.

 The little man had already become the camp drudge, it appeared.

Stack sat down at a little distance from the game with a childish
assumption of injured dignity. During the deals Joe alternately chaffed and
reviled him coarsely. Ralph could not find it in his heart to feel very sorry
for the little man. "He is a sneak," he thought. He kept his ears open for any
word that might throw light on this obscure and curious situation.

After a while Stack said humbly: "Doctor Cowdray, if you please I'd like
to have a word with you before you go."

"I'm damned if you do!" cried Joe. "You'd like to play him off against
me, wouldn't you; and me against him, and get your private pickings off the
both of us! Me and Cowdray we ain't got no use for each other. We don't
make no pretences. But you! You snide! you want to square yourself with
him, don't you? After telling me you trailed him all the way from the coast!"

"I have nothing to say to you!" cried Stack, with a display of childish
fury that caused all three of his mates to shout with laughter.

A light broke on Ralph. Trailed all the way from the coast! To learn this
was worth having come for! But why anybody should want to trail him was
more of a mystery than ever. He determined to find out.

Meanwhile the game went on with four players. The fortune of the cards
changed, and Joe Mixer began to win, principally from Ralph. His good
humour was restored. This was as good a way to get square as any. As
Ralph's pile of matches melted away, Joe triumphed insolently. He doubled
and trebled the ante whenever it came to him. Finally he said:

"A dollar to draw and two to play. Does that scare you off, Doc?"

"Not at all," said Ralph coolly. "This is mild beside the play in New York
clubs."

"Well, it ain't hard to win all you've got," snarled Joe.

"Three cards," said Ralph to the dealer. "This is my last hand."

 He had been dealt a pair of aces. He drew another ace with a pair of
sixes, and a comfortable little satisfaction warmed his breast. His face was
like the Sphinx's. Joe Mixer drew two cards. Ralph, watching him narrowly,
saw a tiny spark of satisfaction light his eye when he looked at them, and
guessed that he held three and had drawn a pair. Revenge was as sweet to
Ralph as anybody.

Joe bet in a small way, and Ralph raised him modestly. The others had
dropped out. Joe raised again, and Ralph followed suit. Joe, seeing that he
was not to be shaken off, began to plunge. Ralph's matches were exhausted
long ago, and he threw the money on the blanket, raising Joe a dollar each
time. Joe began to breathe hard and his face became as pale as a butcher's
face may, except his ears, which remained a furious crimson. He raised
Ralph five, and finally ten dollars at a time, hoping to bluff him out. Ralph
covered his bets with a smile, and each time raised him one. A respectable
little hill of greenbacks grew on the blanket. Crusoe and the breed eyed it
hungrily. Finally, when it came to Joe's turn, he stopped. Little beads of
perspiration had sprung out on his forehead.

"What's the matter?" asked Ralph innocently. "Are you scared off?"

"No!" cried Joe with an oath. "Ain't got no more money," he added
sheepishly. "Don't carry it on the trail. Will you take my I.O.U.?"

Ralph shook his head. "A cash game, you said. I'll take back my last
raise and call you instead."

Joe with a great air of bravado laid down three kings and two queens.

Ralph made believe to be dumbfoundered. Joe grinned and reached for

the money with a trembling hand; whereupon Ralph counted out his three
aces and his pair of little ones.

"It's a shame to take all you've got," he said softly.

 XV

THE STANLEY RAPIDS

Joe ardently desired to continue the poker game on borrowed capital, but
Ralph pointed out that he had announced in advance his intention of retiring
from the game. "I've got to sleep," he said.

"Camp here if you like," growled Joe.

Ralph shook his head. "I'll drop down the river a little piece," he said. "I
want to get an early start."

"You'll have to get up early to keep ahead of us in that contraption," said

Crusoe with a laugh. "It's no more than a dunnage bag stretched on a couple
of half hoops!"

"You can't go down the Stanley rapids in her," said the breed. "She all
bus' up."

"Don't expect to go down the Stanley rapids," said Ralph with a great air
of carelessness. "I'm going up the Stanley."

He observed that Stack and Joe were listening attentively.

"You can't track her," the breed said scornfully.

"My partner is waiting for me at the Forks," lied Ralph. "He's got a
dugout."

"Where the hell did you pick up a pardner?" Joe burst out, forgetting
himself.

Ralph opened his eyes wide in affected surprise. "Well, say, give me
time," he drawled, "and I'll tell you all my private business!"

The laugh was fairly on Joe. He flung away with a muttered curse.
 Ralph, embarking, paddled no farther than around the first bend. Here he
made his camp on the same side of the river as the others. He thought it
likely Stack would try to communicate with him during the night. Ralph
was highly satisfied with the results of the evening's entertainment. Besides
winning about fifty dollars, he had shown them he was not afraid, and he
had put them, he hoped, on a false scent as to his destination.

He made a little fire, and retired under his shelter, but not to sleep. He
had plenty to occupy his mind. After an hour or so he heard a rustle in the
underbrush, and presently a scared voice whispering:

"Doctor Cowdray! Doctor Cowdray!"

Ralph sprang up.

"Don't shoot! Don't shoot!" cried the voice in terror. "It's only me,
Stack."

Ralph laughed.

The little man drew near, cringing. "Won't you put out the fire?" he
whined. "In case any of them should come."

Ralph scattered the embers.

Stack needed no encouragement to make him speak. It came tumbling

out; truth and lies, complaints and excuses all mixed. "My God! Doctor!
What a terrible position I'm in!" he wailed. "I don't know which way to
turn. I gave Mixer two hundred and fifty dollars to guide me through the
country, and look at the way they treat me! You saw it! I have to wash the
dishes, and wait on the half-breed! Me! with a college education! I'm in
momentary terror of my life. I hired Mixer, thinking no wrong, and now I
find him pursuing some murderous vengeance against you! If you could
hear how he talks about you! Look what a position that puts me in—
travelling with a gang of murderers! What must you think of me?"

Ralph listened to all this, smoking impassively. "What are you making
this trip for?" he asked.
 "Just to see the country," whined Stack. "Didn't I tell you that? I wish to
heaven I was well out of it!"

"That's a lie," said Ralph coolly.

"Oh, Doctor Cowdray, I wouldn't lie to you! I wouldn't do such a thing!"

he protested volubly.

"Did you hire Joe Mixer to bring you after me?" Ralph demanded
imperatively.

"Yes," faltered Stack. "But for a purely legitimate purpose. I swear it!"

"Have you, as Joe said, been trailing me all the way from the coast?"

"Yes," he confessed. "But meaning no harm at all—purely legitimate,

Doctor, purely legitimate!" His voice trailed away.

"Well I'm damned!" said Ralph. There was a silence while he smoked.
"What was your purpose?" he finally demanded to know.

"It's such an improbable story I didn't dare tell you," said Stack. "And I
haven't any proof of it."

"You tell me and I'll decide as to the proof," said Ralph.

Stack took a breath and began with renewed glibness: "I'm a newspaper
reporter—Pacific Herald. The city editor was told you had made a big new
strike up here, and he sent me to follow you in, and get the first story of it
for the Herald. I had to do what I was told," he whined, "or lose my job.
You can't blame me——!"

"Who told him about me?" asked Ralph astonished.

"Don't ask me," said Stack. "I've heard they have the assay office
watched. I don't know."

It was obvious to Ralph from the man's silky, fawning voice that he was
lying still. His gorge rose. Evidently the truth had to be terrified out of such
a creature. They were sitting beside the last faint embers of the fire. Ralph
shot out his hand and gripped Stack by the collar. A faint, gasping cry
escaped the little man, and he went limp in Ralph's grasp.

"I have my revolver in the other hand," Ralph said in a rasping voice.
"The truth now, or I'll crack your skull with it! It was you who watched the
assay office."

"Yes," murmured Stack in accents of honest terror.

"You followed me up here on your own responsibility, hoping to get in

on my strike?"

"Yes."

Ralph dropped him. "Now we know where we stand!" he said.

Stack, like all born liars, had an infinite capacity for swallowing his lies.
Ralph had no sooner dropped him than he unblushingly appropriated the
credit for his confession.

"I had to come and square myself with you," he whined. "I couldn't rest
until I had come and told you the truth!"

"Well, I'm damned!" said Ralph again. "Go on!"

"You're the only friend I've got!"

"Friend!" said Ralph with a snort of scornful amusement. "This is good!

Give it to me straight," he went on curiously. "What did you come here for
to-night?"

Stack's voice rose to a piteous wail. "Any night I may be murdered in my

blankets!"

"Sure," said Ralph coolly. "But what can I do for you?"

"Take me with you in your boat," Stack blurted out.

 "Well, upon my word!" cried Ralph.

"Don't refuse! Don't refuse!" said Stack breathlessly. "They wouldn't

dare touch me if I was with you. They're afraid of you. That was
magnificent of you to come to their camp and sit in the game as if nothing
had happened. It had its effect, I can tell you! Oh! take me with you!" he
went on, stuttering in his eagerness. "I can help you escape from them. Two
heads are better than one. I have a good head for planning when I'm not in
mortal fear of my life!"

"Fine!" said Ralph. "And you get right in on my strike!"

"I wouldn't ask much," said Stack. "I'd be content with whatever you
wanted to let me have. Why can't we work together? You need a
representative outside. You've got to file a lot of dummy claims to cover the
whole field. You've got to form a company. I can attend to all that for you.
It's just my line!"

"Thought you said newspaper reporter?" remarked Ralph.

"That was just making out," said Stack hastily. "I know the mining
business from A to Z. I've got legal training. You need me!"

"Thanks," said Ralph coolly. "I prefer to pick my own company."

"If anything happens to me it'll be on your head," whimpered Stack.

"Aren't you going to take me with you?"

"No!" said Ralph in a tone there was no mistaking.

"What shall I do? What shall I do?" moaned Stack. "If you won't let me
travel with you, tell me where you're going, and if I can escape from them,
I'll try to reach you. In common humanity you can't refuse that!"

Ralph smiled into the darkness. "Is it possible he still thinks I am fool
enough to give away my secret!" he thought. "If he does, all right!" Aloud,
he said carelessly: "I've no objection to telling you that. But I won't
guarantee you a welcome."
 "Anyway, you're not a murderer!" whined Stack.

"It's about twenty-five miles up the Stanley River from the Grand Forks
——"

"Then you were telling the truth?" said Stack with naïve surprise.

"Why not?" said Ralph coolly. "I'm not afraid of them." He bethought
himself of adding a few convincing touches to his lie. "You enter a tributary
that comes in on the right-hand side of the Stanley, and ascend it as far as
you can go into the foothills. There you will find our camp."

"How will I know the mouth of the right tributary?" asked Stack.

"By two pine trees that lean across, one at each side, until their tops
almost meet," said Ralph readily. "My partner and I call it the A River."

"Take me with you!" Stack began all over again. "You need me!"

"Cut it out!" said Ralph impatiently.

"You ought to take me with you," Stack persisted. An indescribable, sly,

cringing threat crept into his whine. "Now that I know where you're going,
if they torture me I might let it out in spite of myself!"

Disgust overmastered Ralph. He sprang up. "You little cur!" he cried.

"Get out of here before I hurt you!"

Stack waited to hear no more.

During the next three days the two boats seesawed on the lakes and
rivers, Ralph now ahead, and now Joe Mixer's party. Ralph kept much
longer working hours, but the others made it up in speed. Whenever they
passed each other it became the occasion for an exchange of half-serious
abuse, which was only prevented from developing into a fight by Ralph's
unshakable, steely smile. Ralph insisted on making out that it was all a joke.
Joe was itching for a fight, but the smile cut the ground from under him.
Meanwhile Ralph gave as good as he got. Stack never took part in these
contests of wit. He sat in the dugout haggard and abstracted, gripping the
gunwales under his skinny knuckles. When he thought Ralph's gaze rested
on him, he did his best to look meek and imploring, but succeeded very ill
in disguising his hatred. Joe Mixer carried a deal of liquor in his baggage as
evinced by their frequent thickness of speech.

At the end of the third day they had travelled far down the Rice River.
By paddling until near dark Ralph succeeded in pitching his camp three
miles in advance of the other party. It was his intention to sleep for four
hours only, and then go on. According to his calculations he was within a
few hours' journey of the Grand Forks, and it was essential to his plan that
he get there first. He meant to watch from some place of concealment on
the shore, to make sure that they turned up the Stanley River instead of
continuing downstream. In case they were not deceived by his false lead,
and did not leave the main stream, he had one more desperate card to play.
The moon was now nearly full again, and he could be sure of a certain light
until dawn.

Ralph pitched his little shelter in an opening among the willows that
thickly lined this part of the bank. His boat was drawn high up on the stones
below, and tied to the willow trunks. He ate a hasty supper and turned in. As
he lay waiting for sleep, once again he was warned by a vague disquiet in
his breast of an impending danger. He remembered this afterward. At the
time he was dog-tired, and the still voice was not insistent enough to cleave
the gathering mists of sleep. He soon became unconscious.

He was awakened immediately, or so it seemed to him, by a sudden

outburst of drunken shouting. At the same moment his shelter collapsed on
top of him. When he succeeded in freeing himself of the entangling
blankets, netting, and canvas, in the dim light he saw four figures reeling
about where his fire had been, kicking his belongings into the bush, and
wreaking what senseless damage they could. A terrible rage nerved him in
every fibre.

"You damned cowards!" he cried.

 Hearing his voice, they made for him simultaneously, but Ralph
retreated silently under the willows, and bided his time, peering through the
branches. They searched for him, stumbling over the roots and shouting
inanely.

During the next two or three minutes the scene was as confused and
incredible as a nightmare. Ralph made out a swollen body swaying on the
edge of the bank, outlined against the moonlight. Rushing him, he hauled
off and struck him on the jaw with a savage satisfaction in the crack of it.
He made to follow up the blow, but Joe was not there. He lay in a heap at
the bottom of the bank. Hearing a sound behind him, in the act of whirling
around, a bludgeon aimed at Ralph's head descended on his shoulder.
Seizing him who had wielded it around the body, Ralph lifted him clear of
the ground and flung him after Joe. This one was Crusoe Campbell. A third
figure scuttled down to the water's edge without waiting to be assisted.
Ralph stood in the ashes of his fire, breathing hard, and glaring around like
a lion for another adversary.

The half-breed stepped from out the shadows of the willows. "Look out,
white man!" he cried boastfully. "I got it in for you! I'll fix you good!"

"Come on!" cried Ralph gladly. At the same time the curious thought
shot through his brain: what could the half-breed have against him? It was
not Joe Mixer's quarrel; there could be no mistaking the note of personal
enmity.

The moon shone down serenely indifferent. A little prize-ring was

illuminated within the encircling willows. In it the two men advanced
toward each other, fists up. They crashed together. This was an adversary
worthy of Ralph; he fought like a white man, and he fought fair. Shrewd
blows were exchanged on either side. Each quickly learned to respect the
other, and thereafter fought more warily. Failing to reach Ralph's head, the
breed punished him about the body. Every one of Ralph's blows was aimed
in the centre of the pale ellipse that denoted the other man's face.

Ralph had an advantage in that the breed's head was somewhat fuddled.
His blows began to go wild. Ralph beat him to his knees, and stood back to
let him rise. As they rushed each other again, Ralph's ankles were grasped
from behind, and he was flung violently to the ground, striking his head.

As from an immense distance he heard the half-breed say: "Dam' little

sneak! Wat for you do that? I want lick 'im myself!"

Then the voices receded. Ralph heard them from the beach; heard a
hoarse guffaw, and afterward the splashing of paddles. He understood that
they had gone.

By this time he had got to his feet. He stood, reeling from the effects of
his fall, and half suffocated with a cold and deadly rage. He made his way
down to the water's edge. His boat was turned upside down on the stones,
and the moonlight revealed several clean slashes in her canvas bottom.

"Oh! the scum!" muttered Ralph in his rage. "Unnatural beasts without
decency or manliness! Malignant, cowardly, sneaking rats!"

In cutting his boat they had not done as serious damage as they doubtless
aimed to do, for Ralph carried spare pieces of canvas in his baggage, and a
can of waterproof gum against emergencies. He instantly set about repairing
the boat, working away in the partial darkness with the pertinacity inspired
by a cold rage. He had no doubt now of what he meant to do.

"They'll be sleeping sound after the booze," he thought grimly. "They

think they've fixed me for a while. They won't be looking for a visit to-
night."

When he had his patches affixed, he built a small fire on the stones, and
held the boat over it to dry the gum.

In less than two hours she was fit to float again. He carried his fire up on
the bank then, and making a blaze, hastily collected his scattered
belongings. This refreshed his rage. In his impatience he flung everything
into his boat higgledy-piggledy, and pushed off. He did not paddle, for fear
of being carried past, but allowed the current to take him, while he searched
both shores with straining eyes. No shadow was allowed to pass
unexplained.
 He had not gone much above a mile when he saw what he so ardently
desired: their dugout drawn up on the stones. A great satisfaction diffused
itself throughout his breast. Softly paddling ashore, he beached his own
boat alongside, and bent his head to listen. A faint snoring from the bank
overhead reassured him. He smiled scornfully. In their drunken carelessness
they had actually left most of their baggage in the dugout. Ralph had no
desire to starve them to death, or to deprive them of the means of ultimate
escape. With suitable precautions of silence he unloaded everything on the
stones. Then untying the rope by which the dugout was fastened to a tree,
he heaved her adrift on the current. He didn't care much whether they heard
that or not. But no alarm was raised.

Embarking in his own boat, Ralph towed the larger craft into midstream.
Picturing the scene that awaited their awakening next morning, he chuckled
grimly, and found his breast eased of its weight of rage. He felt not the
slightest regret for what he had done; indeed he was blaming himself for the
foolish compunctions that had prevented him from doing it earlier. His
enemies were in no pressing danger; they possessed a store of food, also
guns and ammunition. They would eventually build a raft. In the meantime
he would get a start that would put him out of their reach for good. He was
free of them. A great serenity descended on his spirit.

Before he cast off the dugout it occurred to him that it was better fitted to
descend the rapids ahead than his own clumsy coracle. He debated the
matter. An odd quirk of conscience finally prevented him from making the
change. "If I use the thing," he thought, "it's the same as stealing it." On this
fine distinction depended the whole subsequent course of his story. He cast
the dugout adrift. There was no wind to blow it ashore and it was good for a
long journey.

During the rest of the night Ralph paddled and floated with the current
without seeking any further rest. Dawn found him among the islands that
marked the approach of the end of the Rice River. This was where he had
first been blindfolded on the previous journey, and he awaited the
subsequent sights of the river with a stimulated curiosity.

At sun-up, rounding a bend, he beheld the wide expanse of the meeting

of the waters, the Grand Forks of the Spirit River. There could be no
mistaking the place. The two rivers occupied the same valley; one came
down from the north, one from the south; meeting head on they swung
away to the eastward. The green current and the brownish struggled
ceaselessly for possession of the channel. At present the Stanley was in
flood, backing up the waters of the Rice River for several miles. The
division between sweeping brown water and motionless green water was as
sharply defined as between water and land. Poking the nose of his boat into
the current, she swung around and almost rolled awash under the impact.
Ralph instantly remembered the sensation which had so puzzled him while
he lay blindfolded.

Soon after he began to move down on the majestic flood of the

augmented river, the murmur of the great rapids crept on his ears, and his
heart began to beat. This would be the first real test of his paddle. The
murmur increased to a rumble, then to a roar. Finally he could make out the
white-caps leaping below, like the naked arms of a multitude ceaselessly
tossed to the sky in wild excitement. He appreciated the vast difference
between a pretty stream brawling among the stones, and a mighty
watercourse plunging over a barrier of rock.

He landed a little way above the rapids and fortified himself with an
excellent breakfast. Afterward he made his way alongshore to the beginning
of the turmoil to try to spy out the best place to enter it. A close view of its
mightiness made him feel very small. The immeasurable flood of water
swept smoothly over the hidden ledge with an oily streaked surface, moving
faster and faster until it suddenly boiled up madly at the bottom. From shore
to shore, nearly half a mile, the wild, white welter prevailed. Ralph received
a stunning impression of the tearing, resistless might of the down-rushing
water. Its roar was deafening. At the thought of tempting it with his flimsy
coracle, his heart shrunk away to nothing in his breast. But it had to be
done.

At first as far as he could tell one place was as bad as another to descend.
Gradually he made out that by great good fortune he had chosen the right
side of the river. Toward the other bank the white surface was everywhere
pointed with ugly black rocks. He saw that the greatest volume of water
rushed down close to the shore on which he stood. If he could keep his boat
in the middle of it there was no danger of rocks. There remained the danger
of those strange, great billows which curled and rolled and roared without
ever advancing an inch in their paths.

He returned to his boat, fighting his terror of the place. Refusing to think
of it, he worked desperately to make all snug. He got in and clung to a
branch that trailed in the water, while the increasing current sucked at his
little craft. He had fallen out of the habit of articulate prayer; maybe he
prayed in his own way. He let go of the branch, and began to drift toward
the place. He moistened his lips, and drew a long breath, and drove his
paddle into the water. No turning back then.

Then he took the plunge, and was filled with an amazing exhilaration.

The struggle was brief. His boat plunged her nose right under the first
curling white billow and half a ton of water fell aboard. She staggered
drunkenly, and in spite of his desperate paddling swung broadside in the
current. The next billow raked him from stem to stern, rolled his boat
completely under and washed him clear of it. The opposed currents of the
water clutched at him and racked him like whirling machinery. He came to
the surface gasping, only to be flung violently against a rock, striking on his
shoulder. Stunned by the buffeting and the roar, he was carried on down like
a rotten log, now underneath, now on top, the plaything of every wild eddy.

Struggling instinctively, in the end he found himself somehow in still

water. He crawled out on the beach and lay inert, struggling for breath and
for consciousness. Very slowly the realization of his plight was forced on
him. He felt no great concern. It was like something that might have
happened to somebody else. There lay a poor devil cast ashore in the
wilderness hundreds of miles from any fellow-creature. Everything he
possessed, boat, food, matches, axe, blankets, gun and ammunition were at
the bottom of the river. Out of the wreck he had saved only Nahnya's
necklace, which was sewed to his shirt, and his pocketbook with money,
neither article being of the slightest service to keep life in his body.

He sat up, roused by an imperious pain. Looking sideways and down at

himself he was mildly impressed by the extraordinary conformation of his
right shoulder—like somebody else's shoulder. It was dislocated. He could
not lift his right arm. It was a mercy, if but a small one, that his faculties
began to work so slowly. His first articulate thought was:

"Well, thank God! I got a skinful of breakfast before I lost it!"

XVI

THE TWO GIRLS

A traveller might have descended through the Spirit River pass half a
dozen times without suspecting the vicinity of any fellow-creatures in the
hundred miles of mountains. Nevertheless there was a white man's camp at
the foot of Mount Milburn. Milburn is the hoary-headed monarch that
stands guard on the right-hand side of the gateway to the Rockies. It rises
sheer from the river to a height of more than six thousand feet. In the
country it is otherwise called the Mountain of Gold because it has long been
known that one of the buttresses of its base is entirely composed of a metal-
bearing quartz.

The few people of the country knew of course that Jim Sholto had
established himself here with his three children for the purpose of smelting
the ore in a small way, but Jim had built his shacks a quarter of a mile back
from the river to avoid the inconvenient observation of the chance traveller.
Jim and his two sons excavated the ore and burned it in half a dozen little
furnaces of porcelain and brick, the materials for which they had brought in
with immense difficulty. The venture was not highly regarded in the
country. The expense of bringing in supplies was too great. They worked
like beavers, it was said, for a net return no greater than day labourer's
wages. Such unremitting industry accused the easy-going ways of the
North.
 On a brilliant afternoon in July Kitty Sholto was redding up the kitchen
in the larger of the two shacks. There was a cloud on her charming face.
She slapped the enamel-ware plates on the shelf with a malicious
satisfaction in the clatter, and cast the dish-towels over the line, as if they
had individually offended her.

Kitty was twenty years old. In her face were combined elements of
gentleness and piquancy, a rare association and provoking to the other sex.
The piquancy was due to her long eyes, green-gray in colour, and placed a
thought obliquely in her head. Green in eyes is thought of in connection
with feline qualities. There was nothing of that sort about Kitty. All the rest
was gentleness. She had a small, straight nose, and an adorable mouth that
turned up at the corners. Her hair, darkest brown in colour, was of the
crinkly sort that reaches out tendrils. She had a soft voice, with an odd,
hushed thrill in it that was all her own, and a soft and ready laugh. She was
not at all the kind of girl to be given to ill-humours.

Sweeping the crumbs over the door-sill, she stood broom in hand leaning
against the jamb. In one swift cast around she took in the whole scene, the
exquisite, limpid sky, the polished malachite of the deciduous foliage, the
rich bottle-green of the pines, the brook whipping itself white on the stones.
She took it all in, and the line between her dark eyebrows deepened as if the
loveliness of nature were an added affront.

Down the trail from the excavations the four ponies came plodding, each
laden with a double wooden bucket of ore. Bill, the younger of Kitty's two
brothers, walked behind, whistling vociferously, and tickling the rearmost
beast with a switch. Bill was a tall, strong youth of twenty-two, a black
Scotchman with a gleaming smile. Dumping the contents of the buckets on
the little mountain of ore before the other shack, with a flick of his switch
he sent the ponies trotting back one by one for another load.

Bill, pausing to fill his pipe, grinned amiably at his sister. Kitty's
brothers adored her, and teased her remorselessly. "Hello, sis!" he said.
"What's biting you?"

"Nothing!" she said quickly.

 "You look as if the cat was dead and the milk turned," he said in the
humorous style that brothers affect.

"There is no cat and I haven't tasted milk in a year and a half," said Kitty
sharply.

"Take example from me!" sang Bill. "Dog-tooth Bill, the sunshine of
Milburn Gulch!"

"That's all very well!" said Kitty bitterly. "Who wouldn't be gay in your
shoes. You're going away to-morrow. You're going to mix with people; to
see something besides trees; to have some fun! What have I got to look
forward to?"

"Cheer up, sis," said Bill with jocular solicitude. "What can we do about
it? The little iron chest has to be carried out. It's getting too heavy to be left
lying around loose. And there's next year's grub to be brought in."

"Certainly, I know you're obliged to go," said Kitty.

"If you could go in my place you'd be welcome," said Dick. "But it's too
hard a trip both out and in again. You and Dick couldn't do it alone."

"I know it," said Kitty stiffly. "You don't have to explain."

"And we can't take you with us, because the old man can't keep the plant
going, and cook his own grub, too."

"I wouldn't think of leaving him alone," said Kitty indignantly.

Bill began to grin again. "Cheer up, the worst is yet to come!" he cried.
"We'll be back in six weeks with a scow full of good things! What'll I bring
her from town for a present? A silk dress?"

"A lot of good a silk dress would do me!" Kitty said scornfully. "Who do
I ever see from one month to another?"

"Ah, there we have her trouble!" cried Bill. He began to sing and to
caper absurdly:
 "Kitty is mad and I am glad,
For I know how to please her;
A bottle of wine to make her shine
And a nice young man to squeeze her!"

"You're horrid!" cried Kitty, frowning and blushing.

"Give me the specifications," Bill went on, with an air of serious gravity.
"Blond, brunette, or albino? Heavy, welter, or light weight? Kind of
disposition you prefer, and amount of purse to be put up before you enter
the ring? I'll bring the candidate back with me if I have to sandbag him!"

Kitty retired into the house, slamming the door. Bill, with a whoop,
started up the trail after his horses.

When the cabin was put to rights there was nothing more that Kitty was
obliged to do until it was time to start the supper. On such occasions she
was accustomed to help her father in the "works," as they called the other
shack, but the furnaces had been cold for a week now, while all hands
joined to get out enough ore to keep them fed while the boys were away.
There was plenty of work that Kitty might have done, but she was in a
mood to dream and to nourish her grievances. She might have gone up to
the excavation to help, but she dreaded male raillery. She finally turned in
the other direction and followed the path down to the river.

It ended in a little glade that had been a camping-place since time out of
mind. In the middle of the place was a fire-hole, centuries old, maybe.
Upright posts were driven on either side, with a bar across and wooden
hooks of assorted sizes waiting for the bails of the next traveller's pots. In
front of Kitty as she stood beside the fireplace the river stretched its smooth
jade-green flood across to the base of the mountain opposite, and at her left
hand the limpid waters of the creek mingled with the thicker current.

Below the camping-place stretched a bank of fine yellow sand

precipitated by the eddies in times of high water. Partly drawn up on the
sand was a dugout. The Sholtos kept their two boats cached in the creek, but
this one had been got out in preparation for the journey next day. It was the
happy-go-lucky Bill who had left it where it was without tying it, forgetful
of the sudden rises of the river in hot weather.

Kitty got in the dugout, and sat down in the stern, where she might trail
her hands in the water, while she thought things out and dreamed her
dreams. All unwittingly Bill had discovered to her the very source of her
discontent, and she was disturbed and ashamed. It was true that she wanted
a young man! Here she was twenty years old; it was jocularly granted by
her brothers that she was not exactly a fright; yet she had never had a young
man. What was worse there was no young man, at least of her own colour,
within hundreds of miles, and she was doomed to her present imprisonment
for at least another year. Twenty-two loomed ahead like old age itself.
"What chance will I have then!" she thought dejectedly. Behind this was the
hot-cheeked, nagging thought: what business had a nice girl to be desiring a
young man, anyway!

But after a while the lovely afternoon began to have its way with her,
and the disquieting thoughts melted by imperceptible degrees into deceitful,
charming daydreams. She was lying in the bottom of the boat with her arm
on the gunwale, and her head on her arm. Her eyes were bent upstream as
far as she could see. He will come down the river, she dreamed. "Perhaps he
is just around the bend at this moment. I should not be surprised. But what
if he should come when I am not here, and be carried past! That is not
possible! If he is the right one, some power will lead him directly to me!
What is he like? Tall and slender, with round, strong arms, and a wonderful
light in his eyes. He will not be surprised to see me either. He will say: 'I
have found you!' And I will say quite simply: 'I have been waiting for you,'
and everything will be understood."

Following the usual course of day-dreams, Kitty little by little lost the
direction of this beautiful story, and picture began to succeed picture
without any help from her. She found herself climbing the higher slopes of
Mount Milburn hand in hand with the youth whose face was hidden from
her; up into the intoxicating air of the summits. Then presto! without so
much of an effort as the wink of an eyelid they were transported to the busy
streets of town, and looked into the bewildering shop-windows without any
surprise at all. Then they walked between endless rows of silk dresses hung
on hooks, and all the dresses were hers, but she couldn't decide which one
she liked the best, and was much distressed. And he said: "Don't worry; I
have a paper boat to sail down Milburn Creek in." And she answered: "We'll
never get up again," without caring in the least. And then they danced to
delicious music that issued from a row of trees like the pipes of an organ.

With a long sigh Kitty stretched herself luxuriously in the bottom of the
dugout, and ceased to dream. If any young man had come along then and
had seen her thus, her head on her folded arm, her lashes on her cheeks, and
a dream-smile tilting the corners of her mouth, it is safe to say he would
never have been the same again afterward.

She awakened as quietly as she had fallen asleep, and lay for a while
gazing up between the sides of the dugout at the delicate clear sky, which
had not changed while she slept. Gradually she became aware of missing
something; it was the turbulent voice of Milburn Creek, never stilled in her
ears at home. At the same time the dugout rocked gently with her, filling her
with an unexplained fear. She quickly sat up.

The heart in her breast turned cold. She was adrift in midstream. Mount
Milburn had disappeared and the even more familiar limestone face of
Stanhope, opposite their camp. Strange mountain shapes surrounded her,
and unfamiliar shores. Her eyes darted up and down the dugout; there was
no paddle; nothing! The swirling green eddies smiled at her horribly, like
things biding their time. Blank, hideous terror descended on her, scattering
her faculties.

There was worse in store. Sweeping around a bend, she saw far down the
river the white horses leaping in the sunshine. She knew the place, the
Grumbler rapids; up and down river they bore a sinister reputation. She
stared at the place, fascinated with horror. The river was so smiling, sunny,
and beautiful, she could not believe that there was the end of all; the very
white-caps below seemed to be leaping in play. And she herself, twenty
years old, and full of the zest of living—it was not possible! But the ever-
increasing voice of the place warned her, there waited Death, sure and
dreadful. And nothing might stop her deliberate progress between the green
shores. She must sit with her hands in her lap and watch it coming step by
step.
 Kitty's very softness and gentleness shielded her. She could not take in
so much horror. Her eyes widened; she struggled for her breath—and
collapsed in the bottom of the dugout.

When consciousness and sight returned, she found a strange, dark face
bending over her. She was lying on firm ground beside the river. The roar of
the rapids filled the air. Seeing Kitty's eyes open, and the light of reason
return, the face broke into a beautiful and kind smile. Kitty, without
understanding clearly, was immensely reassured. It was a girl not much
older than herself.

"You all right now," the girl said.

"What happened?" asked Kitty faintly.

"You near get in the rapids."

The recollection of her terror rushed back over her almost drowning
Kitty's senses again.

"You all right," the girl repeated in a cheery, matter-of-fact tone that was
just what Kitty needed. "I was working on the shore," she went on, "and I
see a canoe come floating down. I think it is foolish to let a good boat get
broke on the rocks, so I get my boat and paddle for it, but there isn't much
time. I come to it, and I look in. Wah! there is you!"

"Oh, it was horrible! horrible!" murmured Kitty, shaken by strong

shudders.

"Forget it," said the girl. "You all right now."

"How did you get me ashore?" Kitty asked.

"It was not much," the girl said with a shrug. "I was too near the rapids
to save both boats, so I jump in yours and let mine go down. It was pretty
hard paddling," she went on, smiling; "we were on the wrong side for the
deep water. Long time we jus' stand still out there, and not go up or down.
Then we come in slow, slow. There is a tree fallen down beside the water,
and I catch hold just in time."

"You have saved my life!" murmured Kitty.

"Cut it out!" said the dark girl gruffly. "It was worth it for the boat
alone."

"But you lost your boat," said Kitty.

The other shook her head. "It is stuck on the rocks down there," she said.
"I will get it after."

Strength and self-command came back to Kitty, and she sat up. The two
girls measured each other with glances of shy, strong curiosity. Each was a
surprising discovery to the other.

"You are Kitty Sholto," said the dark girl.

"How did you know that?" exclaimed Kitty, opening her eyes.

"There is no other white girl in the country."

"I don't know you," said Kitty.

The other shrugged and smiled a little. "There are plenty red girls," she
said. "I am Annie Crossfox."

"Where do you live?"

Nahnya pointed vaguely downstream. "My people are the Sapi Indians,"
she said.

"But that is way down by the canyon," said Kitty. "Do you travel so far
by yourself?"

"I like travel by myself," Nahnya said deprecatingly. "I hunt and I fish.
People think I am crazy. They say it is like a man!"
 Each thought the other a wonderful creature. Nahnya marvelled at the
colour of Kitty's eyes, green-gray like the Spirit River itself, and her cheeks
like snow—snow with the light of the setting sun upon it. Her delicacy and
gentleness seemed like the qualities of a superior creature. Kitty for her part
was no less admiring of Nahnya's strength and courage. The gentle Kitty
like most girls had often wished that she had been born in one of her
brother's places. To be able to go where one pleased like a man! this stirred
her imagination. Each of these lonely girls was hungry for a woman friend;
therein lay the explanation of their kind and wistful looks upon each other.

Kitty was soon quite herself again. Only at intervals did the recollection
of her terror cause her to catch her breath, and send the colour flying from
her cheeks. A lesser fear succeeded.

"How will I get home?" she said. "Dad and the boys! They will be
frantic, poor things!"

"Have they another boat?" asked Nahnya.

Kitty nodded.

"Then they will come look for you soon," said Nahnya calmly. "It is all
right."

Kitty was much reassured.

By degrees the two girls felt their way toward intimate speech. "I am so
surprise I find a white girl in this country," Nahnya said in her quaint, soft
Mission English. "When I look in your boat I am thinking nothing at all.
And there you are! I am so surprise almost we both go in the rapids!"

Kitty explained how she had been carried off.

"Yes, all day the water rise," said Nahnya.

"If you hadn't been there!" said Kitty, and all her terrors returned.

"We must eat," said Nahnya energetically. "I have tea and bread and
meat across the river. We must track for half a mile before I can cross. You
Welcome to our website – the ideal destination for book lovers and
knowledge seekers. With a mission to inspire endlessly, we offer a
vast collection of books, ranging from classic literary works to
specialized publications, self-development books, and children's
literature. Each book is a new journey of discovery, expanding
knowledge and enriching the soul of the reade

Our website is not just a platform for buying books, but a bridge
connecting readers to the timeless values of culture and wisdom. With
an elegant, user-friendly interface and an intelligent search system,
we are committed to providing a quick and convenient shopping
experience. Additionally, our special promotions and home delivery
services ensure that you save time and fully enjoy the joy of reading.

Let us accompany you on the journey of exploring knowledge and

personal growth!

ebookultra.com