Maximizing Patron Outcomes
Gather and Use Community Data to Deliver High Value
Technology Services
Mike Crandall, Samantha Becker, Stacey Wedlake
University of Washington Information School
Agenda
 Introduction & orientation
 What do you need to know?
 Gathering data
 Analysis & reflection
 Putting results to use
Why do needs assessment?
 Align library resources and services to
community needs.
 Give voice to community members who
you might not hear from at the library.
 Show your commitment to being
responsive to emerging needs.
 Discover strengths and areas for
improvement.
Basic steps for needs assessment
1. Assemble your working team
2. Figure out what you need to know
3. Gather existing data
4. Define data collection methods
5. Collect data
6. Analyze data
7. Reflection
8. Reporting
Words of encouragement
Our Case Study
Altamont Public Library
Altamont, Catawba
Let’s get started!
1. Assemble your working team
 Library managers
 Library board/friends
 Local government staff
 Peer agency staff
 Patrons
People who can help you:
– Understand the needs of your
community
– Understand expectations in your
political environment
– Roll up their sleeves and do the work
2. What do you need to know?
Big picture:
 What are the most important issues facing your
community?
 What do your community members need to
have, know, or do in order to be successful?
 What kind of community do you aspire to be?
2. What do you need to know?
Close up:
 What kinds of programs can meet community
needs?
 How well do our programs meet community
needs?
 Who aren’t we serving as well as we could?
What does Altamont Public Library
need to know?
Gathering existing data
Existing data
What existing data can
tell you:
 Who lives in your
service area
 What kinds of lives
they lead
 What they might need
 How patrons are
using the library
Data sources:
 Census/ACS
 Broadband USA
 Community indicators
 City/county surveys
 Education agency
 Employment agency
 Library records
 Other research
Compile into a community profile
 What are the important demographics in your
community that may drive library needs?
0%
10%
20%
30%
40%
50%
60%
70%
80%
1 2 3 4 5 6
National vs. Community
Compare to other research/surveys
My community is:
 Older than average
 More low-income
 More single-parent
households
Research says:
 Older/low-income
people are less
likely to use
technology or have
technology at home.
Could mean: Higher use of library technology
More need for one-on-one help
More need for early literacy programs
Let’s do it!
A good start for thinking about
library services
But how do I know they’re the right ones?
Needing, doing, being
 Frame it together as a speculative theory of
change (you’ll do data collection to verify)
 The end point reflects the mission of your library,
the priorities of the community, and the needs of
patrons (through which community priorities are
achieved).
Theory of change
Need
Books
Reference
Public
technology
Programs
Ability to
do
Potential
to
become
What do you do here…
to help your patrons get here?
And why do you think it helps?
Now you can test your theory
 Are the needs identified the most pressing in
your community?
 Are there other needs and wants in your
community that you aren’t addressing?
 Do the resources and services you offer meet
those needs and contribute to the change?
 Are there better ways to meet those needs?
Data collection methods
Community forums
Focus Groups
Interviews
Surveys
The voices approach
The goal is to hear from a variety of voices who
can:
 Confirm/refute your theory of change
 Provide a more nuanced understanding of how
your services affect change in your patrons and
community
 Create a shared vision and solutions to
community problems
 (and while you’re there, get some good stories)
Focus groups and
interviews
Creating the atmosphere for sharing
1. Fear not!
2. Practice
3. Engage
4. Warm up
5. Ask for outcomes
6. Give space
7. Follow along
8. Drill down
9. Wrap up
10. Reconnect
Focus groups
Best for:
 Interactions with young people or special
populations
 Understanding values and opinions
 Getting reactions about proposed resources or
services
 Generating dialogue and new ideas
 Understanding how different people interact with
the library
Focus groups
Pros:
 Opportunity to drill
down
 Space for idea
generation
 Less time
consuming than
one-on-one
interviews
Cons:
 Need to manage
group dynamics
 Need to hold more
than one
 Usually need an
outside facilitator for
open feedback
Focus groups
 Plan ahead with specific questions and learning
goals
 Invite 5-7 people for each focus group
 Choose a comfortable location
 Have a facilitator and a note taker
 Start with a structured activity
 Keep conversation moving along, encourage
quiet participants
 Transcribe notes immediately
Let’s do it!
Interviews
Good for:
 In-depth understanding/deep diving
 Exhausting potential opinions
 Asking sensitive questions
*Can be structured or open-ended
Interviews
Pros:
 Very rich information
 Ability to follow-
up/probe
 Can get at outcomes
and collect stories
for advocacy
Cons:
 Usually need
outside interviewers
 Time consuming
 More difficult to
analyze results
Interviews
 Create interview guide based on what you need
to know
 Record interviews if at all possible & take notes
 Give space for responses, then give more space
 Follow-up with increasingly detailed questions
 Transition between topics
 Continue to interview new subjects until new
views are exhausted
What to do with the data?
Qualitative analysis
 Impressions of results from forums, focus
groups, and interviews can be misleading
 Approach analysis systematically
 Write up brief case summaries
 Categorize beliefs, opinions, outcomes
 Look for prevailing themes
 Look for exceptions and alternate views
 Connect stories to your evaluation questions
 Validate findings with other methods
Which brings us to
surveys
Surveys (aren’t just for satisfaction)
Best for:
 Validating findings from qualitative data
 Understanding the extent of the phenomena
 Learning about outcomes
Surveys
Pros:
 Low-cost on the
front end for web
survey
 Able to reach/hear
from more people
 Can be “piggy-
backed” on other
community surveys
Cons:
 Phone surveys very
expensive
 Good survey
questions require
some expertise
 Analysis can be
difficult and time
consuming
You have to work for responses
 Try a combo of web and paper survey
 Send out links to email list
 Mail paper survey with return envelope
 Include link to web survey
 Make community aware of survey
 Newspaper/radio announcement
 Social media
 Enlist partner organizations
Survey questions
 Focus on factual information
 Open-ended questions can be treated like interviews
 Avoid compound questions
 Multiple-choice questions should have complete,
mutually-exclusive response choices
 Be aware of “social desirability bias” in designing
questions
 Always pre-test your surveys
Survey questions
Get the most you can out of the fewest number of
questions:
 Instead of “Have you visited the library in the
past year?” Ask “How frequently have you visited
the library in the past year?”
 Ask multi-check questions (top priorities)
 Use other organizations’ survey questions that
have already been tested
Two new tools to help
Where People Connect,
Communities Achieve
http://www.libraryedge.org/http://impactsurvey.org/
ImpactSurvey.org
What is the Impact Survey?
A survey tool that helps libraries:
 Gather information about how patrons use their
technology services
 Analyze collected data to inform internal planning and
benchmarking
 Present findings to key stakeholders to advocate for
technology services
ImpactSurvey.org
Why use Impact Survey?
 No need to develop survey questions on your own.
Impact Survey was developed by library researchers and
has been repeatedly validated.
 No need to program or pay for Survey Monkey.
Impact Survey is ready for your use as soon as you sign
up.
 No staff time spent on analyzing results and
formatting them into reports. Impact Survey provides
all of your results in easy to read reports that can be
immediately put to use internally and externally.
ImpactSurvey.org
The survey asks about use and
activities in core outcome areas
Civic engagement eGovernment eBusiness Employment
eCommerce Health & wellness Education Social inclusion
ImpactSurvey.org
Simple setup to professional-looking
reports in just a few steps
1. Create an account
2. Install the survey link on your website
3. Run the survey and invite
the community to respond
4. Get results in customized
reports the next day
What is Edge for Libraries?
 Assess current public access technology and
how it’s used
 Identify ways to strengthen or enhance public
access technology
 Engage with key leaders about the role for the
public library in improving communities
Where People Connect,
Communities Achieve
BENCHMARKS ASSESSMENT
TOOLS
RESOURCE GUIDE
& CASE STUDIES
TRAINING
REPORTING &
PRESENTATION
TOOLS
What’s included in the Edge toolkit?
Where People Connect,
Communities Achieve
COMMUNITY
ASSESSMENT
TECHNOLOGY
MANAGEMENT
ADVOCACY AND
OUTREACH
LIBRARY
LEADERSHIP
Training opportunities
Where People Connect,
Communities Achieve
How do Impact Survey and Edge work together?
libraryedge.orgimpactsurvey.org
Advancing communities through high
quality and sustainable digital inclusion
resources
Advocacy
Support
Planning
and
resource
allocation
Community
Needs
Assessment
-Create an action plan
to deliver the right
services
-Measure the change in
patron outcomes as a
result
-Find out how patrons
are using technology
resources and what
they get out of them.
-Gather information
from the community
about policy areas of
strategic importance
-Use advocacy tools to
communicate the value of
public access to the community
and gain support.
-Use executive tools to show
city managers that the library is
accountable for results
Putting the results to work
Analysis and reflection
 Connect your results with the community profile
 Answer your questions
 Think through proposed initiatives
 Create value
 Operationally feasible
 Attracts support
 Share your results
Make the ask
 Frame in terms of your community needs
 Connect with expressed priorities of leaders
 Show your evidence of need
 Make the case for your solution
 Tell a story of how it will make a difference
 Put a price tag on it
Let’s do it!
ImpactSurvey.org
Thank you!
libraryedge.org
impactsurvey.org
More information:
impact.ischool.uw.edu

Community needs assessment.pla_2014.handout

  • 1.
    Maximizing Patron Outcomes Gatherand Use Community Data to Deliver High Value Technology Services Mike Crandall, Samantha Becker, Stacey Wedlake University of Washington Information School
  • 2.
    Agenda  Introduction &orientation  What do you need to know?  Gathering data  Analysis & reflection  Putting results to use
  • 3.
    Why do needsassessment?  Align library resources and services to community needs.  Give voice to community members who you might not hear from at the library.  Show your commitment to being responsive to emerging needs.  Discover strengths and areas for improvement.
  • 4.
    Basic steps forneeds assessment 1. Assemble your working team 2. Figure out what you need to know 3. Gather existing data 4. Define data collection methods 5. Collect data 6. Analyze data 7. Reflection 8. Reporting
  • 5.
  • 6.
  • 7.
  • 8.
  • 9.
    1. Assemble yourworking team  Library managers  Library board/friends  Local government staff  Peer agency staff  Patrons People who can help you: – Understand the needs of your community – Understand expectations in your political environment – Roll up their sleeves and do the work
  • 10.
    2. What doyou need to know? Big picture:  What are the most important issues facing your community?  What do your community members need to have, know, or do in order to be successful?  What kind of community do you aspire to be?
  • 11.
    2. What doyou need to know? Close up:  What kinds of programs can meet community needs?  How well do our programs meet community needs?  Who aren’t we serving as well as we could?
  • 12.
    What does AltamontPublic Library need to know?
  • 13.
  • 14.
    Existing data What existingdata can tell you:  Who lives in your service area  What kinds of lives they lead  What they might need  How patrons are using the library Data sources:  Census/ACS  Broadband USA  Community indicators  City/county surveys  Education agency  Employment agency  Library records  Other research
  • 15.
    Compile into acommunity profile  What are the important demographics in your community that may drive library needs? 0% 10% 20% 30% 40% 50% 60% 70% 80% 1 2 3 4 5 6 National vs. Community
  • 16.
    Compare to otherresearch/surveys My community is:  Older than average  More low-income  More single-parent households Research says:  Older/low-income people are less likely to use technology or have technology at home. Could mean: Higher use of library technology More need for one-on-one help More need for early literacy programs
  • 17.
  • 18.
    A good startfor thinking about library services But how do I know they’re the right ones?
  • 19.
    Needing, doing, being Frame it together as a speculative theory of change (you’ll do data collection to verify)  The end point reflects the mission of your library, the priorities of the community, and the needs of patrons (through which community priorities are achieved).
  • 20.
    Theory of change Need Books Reference Public technology Programs Abilityto do Potential to become What do you do here… to help your patrons get here? And why do you think it helps?
  • 21.
    Now you cantest your theory  Are the needs identified the most pressing in your community?  Are there other needs and wants in your community that you aren’t addressing?  Do the resources and services you offer meet those needs and contribute to the change?  Are there better ways to meet those needs?
  • 22.
    Data collection methods Communityforums Focus Groups Interviews Surveys
  • 23.
    The voices approach Thegoal is to hear from a variety of voices who can:  Confirm/refute your theory of change  Provide a more nuanced understanding of how your services affect change in your patrons and community  Create a shared vision and solutions to community problems  (and while you’re there, get some good stories)
  • 24.
  • 25.
    Creating the atmospherefor sharing 1. Fear not! 2. Practice 3. Engage 4. Warm up 5. Ask for outcomes 6. Give space 7. Follow along 8. Drill down 9. Wrap up 10. Reconnect
  • 26.
    Focus groups Best for: Interactions with young people or special populations  Understanding values and opinions  Getting reactions about proposed resources or services  Generating dialogue and new ideas  Understanding how different people interact with the library
  • 27.
    Focus groups Pros:  Opportunityto drill down  Space for idea generation  Less time consuming than one-on-one interviews Cons:  Need to manage group dynamics  Need to hold more than one  Usually need an outside facilitator for open feedback
  • 28.
    Focus groups  Planahead with specific questions and learning goals  Invite 5-7 people for each focus group  Choose a comfortable location  Have a facilitator and a note taker  Start with a structured activity  Keep conversation moving along, encourage quiet participants  Transcribe notes immediately
  • 29.
  • 30.
    Interviews Good for:  In-depthunderstanding/deep diving  Exhausting potential opinions  Asking sensitive questions *Can be structured or open-ended
  • 31.
    Interviews Pros:  Very richinformation  Ability to follow- up/probe  Can get at outcomes and collect stories for advocacy Cons:  Usually need outside interviewers  Time consuming  More difficult to analyze results
  • 32.
    Interviews  Create interviewguide based on what you need to know  Record interviews if at all possible & take notes  Give space for responses, then give more space  Follow-up with increasingly detailed questions  Transition between topics  Continue to interview new subjects until new views are exhausted
  • 33.
    What to dowith the data?
  • 34.
    Qualitative analysis  Impressionsof results from forums, focus groups, and interviews can be misleading  Approach analysis systematically  Write up brief case summaries  Categorize beliefs, opinions, outcomes  Look for prevailing themes  Look for exceptions and alternate views  Connect stories to your evaluation questions  Validate findings with other methods
  • 35.
    Which brings usto surveys
  • 36.
    Surveys (aren’t justfor satisfaction) Best for:  Validating findings from qualitative data  Understanding the extent of the phenomena  Learning about outcomes
  • 37.
    Surveys Pros:  Low-cost onthe front end for web survey  Able to reach/hear from more people  Can be “piggy- backed” on other community surveys Cons:  Phone surveys very expensive  Good survey questions require some expertise  Analysis can be difficult and time consuming
  • 38.
    You have towork for responses  Try a combo of web and paper survey  Send out links to email list  Mail paper survey with return envelope  Include link to web survey  Make community aware of survey  Newspaper/radio announcement  Social media  Enlist partner organizations
  • 39.
    Survey questions  Focuson factual information  Open-ended questions can be treated like interviews  Avoid compound questions  Multiple-choice questions should have complete, mutually-exclusive response choices  Be aware of “social desirability bias” in designing questions  Always pre-test your surveys
  • 40.
    Survey questions Get themost you can out of the fewest number of questions:  Instead of “Have you visited the library in the past year?” Ask “How frequently have you visited the library in the past year?”  Ask multi-check questions (top priorities)  Use other organizations’ survey questions that have already been tested
  • 41.
    Two new toolsto help Where People Connect, Communities Achieve http://www.libraryedge.org/http://impactsurvey.org/
  • 42.
    ImpactSurvey.org What is theImpact Survey? A survey tool that helps libraries:  Gather information about how patrons use their technology services  Analyze collected data to inform internal planning and benchmarking  Present findings to key stakeholders to advocate for technology services
  • 43.
    ImpactSurvey.org Why use ImpactSurvey?  No need to develop survey questions on your own. Impact Survey was developed by library researchers and has been repeatedly validated.  No need to program or pay for Survey Monkey. Impact Survey is ready for your use as soon as you sign up.  No staff time spent on analyzing results and formatting them into reports. Impact Survey provides all of your results in easy to read reports that can be immediately put to use internally and externally.
  • 44.
    ImpactSurvey.org The survey asksabout use and activities in core outcome areas Civic engagement eGovernment eBusiness Employment eCommerce Health & wellness Education Social inclusion
  • 45.
    ImpactSurvey.org Simple setup toprofessional-looking reports in just a few steps 1. Create an account 2. Install the survey link on your website 3. Run the survey and invite the community to respond 4. Get results in customized reports the next day
  • 46.
    What is Edgefor Libraries?  Assess current public access technology and how it’s used  Identify ways to strengthen or enhance public access technology  Engage with key leaders about the role for the public library in improving communities Where People Connect, Communities Achieve
  • 47.
    BENCHMARKS ASSESSMENT TOOLS RESOURCE GUIDE &CASE STUDIES TRAINING REPORTING & PRESENTATION TOOLS What’s included in the Edge toolkit? Where People Connect, Communities Achieve
  • 48.
  • 49.
    How do ImpactSurvey and Edge work together? libraryedge.orgimpactsurvey.org Advancing communities through high quality and sustainable digital inclusion resources Advocacy Support Planning and resource allocation Community Needs Assessment -Create an action plan to deliver the right services -Measure the change in patron outcomes as a result -Find out how patrons are using technology resources and what they get out of them. -Gather information from the community about policy areas of strategic importance -Use advocacy tools to communicate the value of public access to the community and gain support. -Use executive tools to show city managers that the library is accountable for results
  • 50.
  • 51.
    Analysis and reflection Connect your results with the community profile  Answer your questions  Think through proposed initiatives  Create value  Operationally feasible  Attracts support  Share your results
  • 52.
    Make the ask Frame in terms of your community needs  Connect with expressed priorities of leaders  Show your evidence of need  Make the case for your solution  Tell a story of how it will make a difference  Put a price tag on it
  • 53.
  • 54.