Unit 1 - Metric
Selection
Defining Metrics for success, Creating work plan
Assembling data and expert sources,
Selecting modelling approaches, Validating and Verifying analytical results
Communicating and presenting results to clients and driving organizational
change and assessing impact
Data Science Metrics
• Metric selection in data analytics refers to the process of choosing the most
appropriate quantitative measurements to assess and evaluate specific aspects of
data or business performance. It involves identifying and defining the key metrics
that align with the objectives and goals of the analysis. The selected metrics
serve as the basis for analyzing data, tracking progress, and making informed
decisions.
• Data Science Metrics is to measures and report for evaluative purpose
• Metrics, by themselves, may not be very useful, but when used in the right
context, they provide meaning to any kind of data analytics activity and aid
decision-making.
• In the context of data science for business intelligence, metrics refer to
quantifiable measures used to evaluate and analyze data-related aspects of a
business. These metrics are typically derived from data and provide insights into
various performance indicators, trends, and patterns within the organization.
Common Metrics
Revenue Metrics These metrics assess the financial performance of a business. Examples include total revenue, profit margin,
average revenue per customer, or customer lifetime value.
Customer Metrics These metrics focus on understanding and analyzing customer behavior and satisfaction. Examples include
customer acquisition rate, customer churn rate, customer retention rate, or Net Promoter Score (NPS).
Operational Metrics These metrics evaluate the efficiency and effectiveness of operational processes. Examples include
production yield, order fulfillment time, inventory turnover, or customer support response time.
Marketing Metrics These metrics measure the effectiveness of marketing campaigns and initiatives. Examples include
conversion rate, click-through rate (CTR), cost per acquisition (CPA), or return on ad spend (ROAS).
Product Metrics These metrics assess the performance and usage of products or services. Examples include product adoption
rate, average usage time, customer satisfaction score (CSAT), or product defect rate.
Financial Metrics These metrics provide insights into financial health and stability. Examples include return on investment
(ROI), gross margin, cash flow, or debt-to-equity ratio.
Web Analytics Metrics These metrics focus on analyzing website or app performance and user engagement. Examples include page
views, bounce rate, conversion rate, average session duration, or click-through rate (CTR).
Forecasting Metrics These metrics involve predicting future trends and outcomes based on historical data and statistical models.
Examples include sales forecasts, demand forecasts, or customer lifetime value (CLV) predictions.
Metric Selection
Identify business goals
Understand the context
Determine relevant dimensions
Define performance indicators
Consider leading and lagging indicators
Data availability and reliability
Align with stakeholders
Keep it manageable
Document and communicate
Monitor and iterate
Strategically aligned metrics are called Key Performance indicator
Targets
strategy, planning,
Strategy or budgeting
KPI embody a sessions
strategic objective
Benchmarks
Ranges results often serve as a
e.g., above, on, or KPI benchmark, but arbitrary
numbers or external
below target
benchmarks may also be
used
Encodings
Ranges are encoded in Time Frames
software, enabling the
visual display of
performance
performance mileposts
Metrics - Classification
• Outcome KPIs-sometimes known as lagging Lagging indicators
indicators-measure the output of past Leading indicators
activity (e.g., revenues).
• They are often financial in nature, but not
always. Driver KPIs-sometimes known as Operational metrics
leading indicators or value drivers-measure Performance metrics
activities that have a significant impact on
outcome KPIs (e.g., sales leads)
• In business organization wide range of
operational metrics are collected. Therefore,
as the name implies, these metrics deals with
operational and performance of a company.
Examples for metrics in operational area
Customer performance. Metrics for Service performance. Metrics for service-
customer satisfaction, speed and accuracy of call resolution rates, service renewal rates,
issue resolution, and customer retention. service-level agreements, delivery
performance, and return rates.
Sales operations. New pipeline accounts, Sales plan/forecast. Metrics for price-to-
sales meetings secured, conversion of purchase accuracy, purchase order-to
inquiries to leads, and average call closure fulfillment ratio, quantity earned, forecast-to-
time. plan ratio, and total closed contracts.
Business Process Management –
Different names
Corporate Performance [Link]
Management
Strategic Enterprise [Link]
Management
Enterprise Performance [Link]
Management
Understanding data Types
• Length of an arm – Numerical data
• Mode of Transport – Categorical data
• Time travel to office
• Money spent at restaurants last year
• City of residence / hometown
• Postal code
• Hours slept at night
• Duration of movie
• Favourite sport
• Date of birth
Types of Data
Nominal
Categoric
al
Ordinal
Data Type
Discrete
Interval
Numerical
Continuou
s Ratio
Categorical data
Nominal Data:–
Nominal data is a type of categorical data that does not have any inherent order or ranking. In this
type of data, the categories are mutually exclusive, and there is no logical order or sequence
between them. In nominal data, you can count the frequency of each category, but you cannot
perform mathematical operations or make comparisons based on the values. Examples of nominal
data include:
• Colors: Red, blue, green, yellow.
• Marital status: Married, single, divorced, widowed.
• Types of vehicles: Car, truck, motorcycle, bicycle.
Ordinal Data:
• Ordinal data is a type of categorical data that has a natural order or ranking between categories.
The categories in ordinal data are ordered, and the intervals between them may not be uniform.
Examples of ordinal data include:
• Educational attainment: High school diploma, bachelor's degree, master's degree, Ph.D.
• Rating scales: Excellent, good, fair, poor.
• Survey responses: Strongly agree, agree, neutral, disagree, strongly disagree.
• In ordinal data, you can not only count the frequency but also compare the values and determine
the relative order or rank between categories. However, you cannot perform mathematical
operations on the values or assume that the differences between the categories are equal.
Numerical Data
• Interval Data: • Ratio Data:
Interval data is a type of numerical data that has Ratio data is a type of numerical data that has
meaningful intervals or differences between values. In meaningful intervals between values and a true
this type of data, the numerical values represent zero point. In this type of data, the numerical
quantities or measurements where the difference
between any two values is consistent and meaningful. values represent quantities or measurements
However, interval data does not have a true zero point where the difference between values is consistent
or absolute starting point. Examples of interval data and meaningful, and there is an absolute starting
include: point or true zero. Examples of ratio data include:
Temperature in Celsius or Fahrenheit: The difference Height: A person with a height of 0 cm has no
between 10°C and 20°C is the same as the difference
between 20°C and 30°C, but 0°C does not represent an height.
absence of temperature. Weight: A weight of 0 kg represents no weight.
Calendar dates: The difference between January 1st and Time: A time duration of 0 seconds means no time
January 15th is the same as the difference between has passed.
January 15th and January 30th, but 0 does not represent In ratio data, you can perform all mathematical
an absence of time. operations, including addition, subtraction,
In interval data, you can perform mathematical multiplication, and division. You can calculate
operations such as addition and subtraction, calculate
differences, and calculate measures like means and
ratios, proportions, percentages, and meaningful
standard deviations. However, you cannot multiply or comparisons between values.
divide the values or make meaningful statements about
Numerical Data – Other Classification
• DISCRETE DATA CONTINUOUS DATA:
Discrete data is a type of numerical data that Continuous data is a type of numerical data that can
can only take on specific, separate values. take on any value within a certain range or interval.
These values are typically whole numbers or Continuous data is characterized by a smooth,
counts and cannot be subdivided further. unbroken range of possible values, and there can be
Discrete data is characterized by gaps or jumps infinitely many values between any two observed
between values, and there is no meaningful data points. Examples of continuous data include:
interpretation of values between the observed
Height: Height can vary continuously from the
data points. Examples of discrete data include:
shortest to the tallest person, with infinite possible
Number of siblings: You can have 0, 1, 2, 3, and values in between.
so on siblings, but you cannot have 1.5 siblings. Temperature: Temperature can vary continuously
Number of cars in a parking lot: You can have 0, from the coldest to the hottest point, with infinite
1, 2, 3, and so on cars, but you cannot have 2.5 possible values in between.
cars. Weight: Weight can vary continuously from the
Number of students in a class: You can have 20, lightest to the heaviest point, with infinite possible
21, 22, and so on students, but you cannot values in between.
have 20.5 students. Continuous data is often represented using line
Discrete data is often represented using bar graphs, histograms, or scatter plots. Statistical
charts or frequency tables. Statistical analyses analyses of continuous data involve measures such
of discrete data involve counting frequencies, as means, medians, standard deviations, and
calculating probabilities, and performing correlations. Continuous data can also be subjected
specific discrete data analysis techniques. to mathematical operations like addition,
Data Collection Methods - Data collection methods refer to
the techniques and procedures used to gather data for
research or analysis purposes. Here are some common data
collection methods :
Surveys: Surveys involve collecting data wby asking questions to Interviews: Interviews involve direct communication with
individuals or groups of people. Surveys can be conducted through participants to collect data. Interviews can be structured (where a
various mediums, such as online surveys, paper questionnaires, predefined set of questions is asked) or unstructured (where the
face-to-face interviews, or telephone interviews. conversation is more open-ended). Interviews can be conducted in
person, over the phone, or through video calls.
Experiments: Experiments involve manipulating variables in a controlled Existing Data Sources: Researchers can collect data from existing sources
environment to observe the effects and collect data. Participants are such as government databases, organizational records, public archives, or
assigned to different groups (control group and experimental group) to previously conducted studies. This method is known as secondary data
compare the outcomes. Data collection in experiments can involve various collection.
methods such as surveys, observations, or measurements.
Focus Groups: Focus groups involve a small group of participants who Document Analysis: Document analysis involves collecting data by
engage in a guided discussion led by a facilitator. The goal is to collect examining and analyzing existing documents, such as reports, letters,
qualitative data by exploring opinions, attitudes, and experiences of the diaries, emails, or historical records. This method is often used in
participants on a specific topic. qualitative research to gain insights into social, cultural, or historical
phenomena
Case Studies: Case studies involve in-depth analysis of a particular Observations: Observations involve systematically watching and recording
individual, group, organization, or event. Data collection methods in case behaviors, events, or processes. Observations can be done in a natural
studies can include interviews, observations, document analysis, and setting (naturalistic observation) or a controlled environment (controlled
reviewing existing records. observation). Researchers can be passive observers or actively participate
in the situation being observed.
Types of data collection
Primary data Collection method Secondary Data collection methie
(1) sources. Researchers collect primary data specifically Secondary Data Collection:
for their own research purposes. This method provides Secondary data collection involves using existing data
researchers with control over the data collection process that has been collected by other researchers,
and allows them to tailor data collection methods to suit organizations, or sources for purposes other than the
their research objectives. Examples of primary data current research project. Researchers analyze and utilize
collection methods include: this previously collected data to address their research
Surveys: Researchers design and distribute questions. Examples of secondary data collection
questionnaires or conduct interviews to collect methods include:
responses directly from participants. Literature Review: Researchers review and analyze
Experiments: Researchers manipulate variables and published studies, articles, books, and other written
collect data from participants in a controlled sources that contain relevant data.
environment to observe outcomes and effects. Government Databases: Researchers access and
Observations: Researchers directly observe and record analyze data available in government databases, such
behaviors, events, or processes to gather data. as census data, economic indicators, or health statistics.
Focus Groups: Researchers facilitate group discussions Organizational Records: Researchers examine existing
to collect qualitative data on specific topics. records maintained by organizations, such as sales
Primary data collection methods offer the advantage of reports, financial statements, or customer data.
providing data that is tailored to the research needs. Secondary data collection methods offer the advantage
However, they can be time-consuming and resource- of time and cost efficiency, as the data already exist and
intensiv are readily accessible. However, researchers have
limited control over the quality and scope of the data
and must carefully evaluate its relevance and reliability.
Performance Measurement System
• Performance measurement and management are 2
different aspects.
