STATISTICS

Statistics is a branch of science that deals with the collection,

organisation, analysis of data and drawing of inferences from the
samples to the whole population. This requires a proper design of the
study, an appropriate selection of the study sample and choice of a
suitable statistical test. An adequate knowledge of statistics is
necessary for proper designing of an epidemiological study or a
clinical trial. Improper statistical methods may result in erroneous
conclusions which may lead to unethical practice.
R
R is an extremely flexible statistics programming language and environment
that is Open Source and freely available for all mainstream operating
systems
Because of R’s Open Source structure and a community of users dedicated
to making R of the highest quality, the computer code on which the
methods are based is openly critiqued and improved.
R is an object-oriented language and environment where objects, whether
they be a single number, data set, or model output, are stored within an R
session/workspace.
The flexibility of R is arguably unmatched by any other statistics program,
as its object-oriented programming language allows for the creation of
functions that perform customized procedures and/or the automation of
tasks that are commonly performed.

Statistical Methods and Analysis Techniques:

Statistical methods and analysis techniques play a crucial role in extracting insights
from data. These methods help researchers make sense of complex datasets and
draw valid conclusions. Some common statistical methods and techniques include:

1.Descriptive Statistics:
Central Tendency Measures:
 • Mean: The average value of a variable.
Use mean(data$variable_name) where data is your data frame
and variable_name is the specific variable you're analyzing.
• Median: The middle value in a sorted dataset.
Use median(data$variable_name).
• Mode: The most frequent value. Use table(data$variable_name)$Mode.

Dispersion (Spread) Measures:

• Standard Deviation: Measures the average distance from the mean.

Use sd(data$variable_name).
• Variance: The squared standard deviation. Use var(data$variable_name).
• Interquartile Range (IQR): The difference between the 75th and 25th
percentiles, indicating the spread of the middle half of the data.
Use IQR(data$variable_name).

Frequency Distributions:

• Table: Provides the number of occurrences for each unique value.

Use table(data$variable_name).
• Histogram: Visualizes the distribution of data points across intervals.
Use hist(data$variable_name).
Measures of Shape:
• Skewness: A measure of the asymmetry of the distribution.
• Kurtosis: A measure of the "peakedness" or "flatness" of the distribution.

2. Hypothesis Testing:
• Formulating Hypotheses: Define the null hypothesis (no significant
difference) and alternative hypothesis (a significant difference exists) based
on your research question.
• Choosing Tests: Select the appropriate statistical test based on the type of
data (categorical or continuous) and the number of variables being compared.
o Categorical Data: Chi-square tests (e.g., Chi-square test of
independence) are used to assess relationships between two
categorical variables.
o Continuous Data:
▪ One Sample: T-tests (e.g., one-sample t-test) compare the
sample mean to a specific value.
▪ Two Samples: T-tests (e.g., two-sample t-test, paired t-test) or
ANOVA (Analysis of Variance) are used to compare means
between groups.
• Conducting Tests: R offers various functions like chisq.test, t.test,
and aov for performing specific hypothesis tests.
• Interpreting Results: Evaluate the p-value (probability of observing the data
under the null hypothesis) to assess the level of significance (typically alpha =
0.05). Reject the null hypothesis if the p-value is less than alpha.
Common hypothesis tests: t-tests, z-tests, chi-square tests, ANOVA, etc.

Regression Analysis:

• Linear Regression: This method estimates the relationship between a

dependent variable (e.g., user engagement) and one or more independent
variables (e.g., number of library visits, resource type used). Use
the lm function to fit a linear model.
• Generalized Linear Models (GLMs): These extend linear regression for
situations where the dependent variable is not normally distributed. Use
packages like glmmTMB for various GLMs.
• Model Evaluation: Assess the model's fit and performance using metrics like
R-squared (coefficient of determination) and adjusted R-squared.

Confidence Intervals:
• Confidence intervals provide a range of values within which the population
parameter is likely to lie with a certain level of confidence (e.g., 95%
confidence interval).
Non-parametric Methods:
• Non-parametric tests are used when the assumptions of parametric tests are
violated or when data are not normally distributed.
• When data is skewed or has outliers, non-parametric tests offer a more
reliable alternative to parametric tests.

Common Non-Parametric Tests in R:

• Wilcoxon signed-rank test: This test compares two related samples (e.g.,
user satisfaction before and after using a new library feature). Use
the wilcox.test function.
• Mann-Whitney U test: This test compares two independent samples (e.g.,
user borrowing rates for different membership types). Use
the wilcox.test function with paired = FALSE.
• Kruskal-Wallis test: This test compares three or more independent samples
(e.g., resource usage across different academic disciplines). Use
the kruskal.test function.

Exploratory Data Analysis (EDA)

Exploratory data analysis (EDA) is a crucial step in understanding and summarizing
your digital library data before diving into further analysis or modeling. Here's how
you can use R for EDA:
1. Data Cleaning and Preparation:

• Import libraries: Use dplyr and tidyr for data manipulation and wrangling
tasks.
• Load data: Use read.csv or relevant functions to read your data from its
source (e.g., CSV file).
• Check data structure: Use str(data) to understand variable types,
dimensions, and potential missing values.
• Handle missing values: Employ appropriate methods (e.g., removal,
imputation) to address missing values if necessary.

• Univariate Analysis: This involves the examination of a single variable in

isolation. It aims to understand the distribution, central tendency, and
variability of that variable. Techniques used in univariate analysis include
histograms, box plots, and summary statistics like mean, median, and mode.
• Bivariate Analysis: Bivariate analysis explores the relationship between two
variables. It helps understand how one variable changes concerning another.
Techniques used in bivariate analysis include scatter plots, correlation analysis,
and contingency tables.
• Multivariate Analysis: Multivariate analysis involves the simultaneous
examination of three or more variables to understand complex relationships
among them. Techniques used in multivariate analysis include principal
component analysis (PCA), factor analysis, and cluster analysis.

Time series analysis

Time series analysis plays a crucial role in studying data collected over time,
like in digital libraries. R provides diverse tools for analyzing and modeling
such data, allowing you to gain insights into user behavior, resource usage
patterns, and other trends.

• ARIMA (Autoregressive Integrated Moving Average) model: This popular

model uses past values and residuals to predict future values.
The forecast package offers functionalities for fitting and evaluating ARIMA
models.

Survival Analysis
• Survival analysis focuses on analyzing the time until an event of interest
occurs and the factors influencing the occurrence
Data Preparation:

1.Ensure your data includes:

o Time variable (e.g., time to unsubscribe)

o Event indicator (e.g., whether the event occurred, typically coded as 0
for censored and 1 for event)
o Additional variables potentially influencing the event (e.g., user
demographics, resource type)

2. Kaplan-Meier (KM) Estimator:

• This non-parametric method estimates the probability of surviving (not

experiencing the event) over time.
• Use the survfit function from the survival package to calculate the KM
estimate and visualize the survival curve.

3. Cox Proportional Hazards Model:

• This is a popular semi-parametric model used to assess the relationship

between explanatory variables and the hazard function (the probability of
experiencing the event at a specific time).
• Use the coxph function from the survival package to fit the model and
interpret the results.

4. Interpretation:

• KM curve: The y-axis represents the estimated probability of surviving, and

the x-axis represents time. A steeper downward slope indicates a higher
chance of experiencing the event earlier.
• Cox model: The model provides coefficients for each explanatory variable. A
positive coefficient indicates an increased hazard (higher chance of the event)
with increasing values of that variable, while a negative coefficient suggests
the opposite.

Machine Learning Techniques:

• Decision Trees (R package: rpart): These tree-based models make
predictions by following a series of decision rules based on feature values.
They are interpretable and work well with various data types.
• Random Forests (R package: randomForest): These ensemble models
combine multiple decision trees, leading to improved performance and
robustness compared to individual trees.
• Support Vector Machines (SVM) (R package: e1071): These algorithms find
the optimal hyperplane separating data points into different classes. They are
effective for high-dimensional data and specific classification problems.
• Neural Networks (R packages: nnet, keras): These flexible models are
inspired by the human brain and can learn complex relationships between
features and the target variable. They can be powerful but require careful
optimization and are often less interpretable than other techniques.
• K-Nearest Neighbors (KNN) (R package: class): These algorithms classify
data points based on the majority vote of their K nearest neighbors in the
training data. They are simple to implement but may not perform well in high-
dimensional settings.

Datasets related to Digital Libraries:

• PubMed: PubMed provides access to biomedical literature. You can

use the rentrez package to access PubMed data.
• ArXiv: ArXiv is a repository of research papers in various fields,
including computer science, mathematics, physics, etc.
• Digital Public Library of America (DPLA): DPLA provides access to
millions of photographs, manuscripts, books, and more from libraries,
archives, and museums across the United States. They offer APIs for
accessing their data.
• Kaggle: Kaggle is a platform that hosts datasets related to various
domains. You may find datasets related to digital libraries through
Kaggle competitions or datasets uploaded by users.