Downloaded 11 times
Uploaded byALATechSource
How to Improve Your Library Instruction: Assessment in Five Minutes (November 2017)
This document discusses how to quickly assess student learning in one-shot library instruction sessions through the use of learning outcomes and short assessment instruments. It recommends drafting 2-5 learning outcomes focused on what students must accomplish and comprehend. Outcomes should be measurable and action-oriented. Assessment questions can then be created based on the outcomes. Several assessment tools are presented, like polls, games, and questionnaires that can be used within a 5 minute time frame to gauge student understanding. Reviewing responses and reflecting on results with students is also encouraged.
More Related Content
Similar to How to Improve Your Library Instruction: Assessment in Five Minutes (November 2017)
How to Improve Your Library Instruction: Assessment in Five Minutes (November 2017)
- 1. How to ImproveYour Library Instruction: Assessment in Five Minutes Sarah Steiner November 15, 2017
- 2. During Today’s Session,We Will… • Review the purpose and benefits of quick assessment in one-shot instruction sessions. • Draft learning outcomes for a session. • Compose assessment instruments based on our learning outcomes. • Select appropriate assessment tools based on your instruction goals, time, and population. http://www.flickr.com/photos/topgold/4858921901/
- 3. What Can QuickClassroom Assessment Do? • It can identify instructional gaps or disconnects. • It can help you determine how you spend class time. • It can build your confidence. • It can provide evidence of efficacy and impact. • It can inspire and motivate your learners. • It can (& should) be a basis for change. http://www.flickr.com/photos/sercasey/324341982/
- 4. What Will WeAssess? • To assess: – Comprehension of integral concepts – Achievement of learning outcomes – Retention • Not to assess: – Your personal shortcomings – Your speaking mistakes – Your hairstyle http://www.flickr.com/photos/toniblay/52445415/
- 5. Have you everwritten learning objectives or outcomes? • Nope, not yet • Yes, one or two times • Yes, regularly • I’m not sure http://www.flickr.com/photos/dadaistic_fantastic/5089525186/
- 6.
- 7. Learning Outcomes “A LearningOutcome is a statement of what the student should understand and be able to do as a result of what she has learned ... ‘the essential and enduring knowledge, abilities, and attitudes or dispositions’ that enable a learner to practice and apply her learning in the real world.” -Valencia Community College http://www.flickr.com/photos/gblakeley/5583354276/
- 10. Outcomes • Global: large,meant to excite and inspire long- term vision • Educational: mid-level, meant to describe how a student will interact with a topic or content. • Instructional: specific, narrow, focus on day-to-day “slices of learning” -Anderson & Krathwohl https://flic.kr/p/4oEedh
- 11. A Good LearningOutcome Will… • Identify the audience. • Set a time frame and a context. • Be jargon-free. • Be measurable. • Be action-oriented. • Be brief. • Be linked to learner needs. • Be appropriately narrow. http://www.flickr.com/photos/auntiep/4310267/
- 12. Determine the ClassPriorities http://www.flickr.com/photos/paloetic/4795592340/ Choose two to five. – What must the students accomplish? – What must they comprehend in order to accomplish it? – What is likely to trip them up? – What do they already know/find obvious?
- 13. Write the LearningOutcomes Stem + Verb + Product/Outcome http://www.flickr.com/photos/jjpacres/3293117576/
- 14. Stem By the endof this class, you will be able to… http://www.flickr.com/photos/47108884@N07/4594962925/
- 15.
- 17. Add Product orOutcome What do the students need? • A thesis statement • Keyword search strategies (basic or advanced) • Understanding of the research process • Ability to critically analyze a source What is the context? • A class assignment • A real-world scenario http://www.flickr.com/photos/fdecomite/2907794565/
- 18. All Together Now! Stem+ Verb + Product/Outcome By the end of this class, you will be ready to construct a thesis statement for your paper. This session will equip you with the tools you need to critically analyze whether or not a source is credible. By the end of class, you will be able locate scholarly articles on your topic and will choose to seek help from your librarian, if you struggle. http://www.flickr.com/photos/revlimit/2186765034/
- 19. What’s Wrong Here? TodayI’ll talk about… 1. The library website. 2. Database searching for peer-reviewed journals using boolean logic and the subject heading thesaurus. 3. Where you click to search the catalog, search the databases, and place an ILL request, and every step you’ll take to accomplish these processes. http://www.flickr.com/photos/mythoto/2604818537/
- 20.
- 21. Have you everassessed student comprehension or learning in class? • Nope, not yet • Yes, but not regularly • Yes, regularly • I’m not sure http://www.flickr.com/photos/dadaistic_fantastic/5089525186/
- 22. Let’s Assess! • Convertyour learning outcomes to questions. • Limit the number of questions. • Eliminate overly easy or “all of the above” answers. • Avoid nebulous or complicated questions. • Work in terminology to ensure it’s understood. http://www.flickr.com/photos/djmccrady/6118626233/
- 23. Assessment Modes Formative orDiagnostic • Polls • Interactive exercises or games • Quizzes/questionnaires Summative • Polls • Interactive exercises or games • Quizzes/questionnaires • Minute papers & muddiest point http://www.flickr.com/photos/azarius/225340667/
- 24. By the endof this class, you will be able to construct a flexible and usable thesis statement for your paper. Ideas… 1. Have them identify/list the qualities of a useful thesis statement. 2. Have them look at some sample thesis statements and consider whether they are too broad or too narrow. 3. Have them draft a thesis statement. Ask them to think about how they might broaden or narrow it, and/or submit synonym ideas. http://www.flickr.com/photos/djmccrady/6118626233/
- 25.
- 26. Simple Questions • Howmany of you have had a library instruction class here before? • Is this source peer-reviewed? / Is this source scholarly or popular? • Who is the author of this book? • Are you confused about topic x? http://www.flickr.com/photos/rakka/3451170932/
- 27.
- 29.
- 30.
- 31.
- 32.
- 33.
- 34.
- 35. The Good, theBad, the Ugly
- 36. Minute Papers &Muddiest Point http://www.flickr.com/photos/holtsman/4331034955/
- 37. Minute Paper ExampleQuestions
- 38. Question Drafting FinalThoughts • Get a reviewer. • If at first you don’t succeed…. http://www.flickr.com/photos/djmccrady/6118626233/
- 39. Review Responses • Withthe class? Yes! • Look for… • Percentage of correct answers. • Trends in missed answers. • What to change next time. http://www.flickr.com/photos/xingty/3421423768/
- 40.
- 41.
- 42. Question Delivery Tools •Kahoot! • Socrative • Padlet • SurveyMonkey • Poll Daddy • Poll Everywhere • Google Forms http://www.flickr.com/photos/litlnemo/5045121567/
- 43.
- 44.
- 45.
- 46.
- 47. Review • Write learningoutcomes. • Base your assessment questions on the outcomes and on student needs. • Review and reflect on the answers you get in class with the students, whenever possible. http://www.flickr.com/photos/demonbaby/4228327167/
- 48.
- 49. Other Resources • Ambrose,Susan et al. How Learning Works: Seven Research-Based Principles for Smart Teaching. Jossey-Bass, 2010. • Anderson, Lorin W. and David R. Krathwohl. A Taxonomy for Learning, Teaching, and Assessin: A Revision of Bloom’s Taxonomy of Educational Objectives. New York: Longman, 2001. • Bloom et al.’s Taxonomy of the Cognitive Domain. http://www.edpsycinteractive.org/topics/cognition/bloom.html • Cahoy, Ellyssa Stern, and Robert Schroeder. “Embedding Affective Learning Outcomes in Library Instruction.” Communications in Information Literacy, 2012. 6(1): 73-90. • Clay, Ben. Is This a Trick Question? A Short Guide to Writing Effective Test Questions. http://www.k-state.edu/ksde/alp/resources/Handout-Module6.pdf • Cornell University Center for Teaching Innovation, “Setting Learning Outcomes.” https://www.cte.cornell.edu/teaching-ideas/designing-your-course/settting-learning- outcomes.html • Oakleaf, Megan. “A Roadmap for Assessing Student Learning Using the New Framework for Information Literacy for Higher Education.” In The Journal of Academic Librarianship, 40.5, 510-514. http://www.sciencedirect.com/science/article/pii/S0099133314001451 • The Teaching Center, “Asking Questions to Improve Learning.” https://teachingcenter.wustl.edu/resources/teaching-methods/participation/asking- questions-to-improve-learning/ • Utah State University Library, “Assessment.” http://library.usu.edu/instruct/assessment/index.php
Editor's Notes
- #3 Choose your questions and model based on the outcomes you need to meet. Everyone has been watching the new Information Literacy framework develop. I agree with Megan Oakleaf that the framework doesn’t overthrow the importance of learning outcomes, though it might shift the way you were writing them if you were already doing them. We will incorporate the framework’s tenets today and talk about how you can assess the threshold concepts that they’re comprised of.
- #4 Example: terminology—article, journal, database This type of assessment is not an end in itself—it supports change and best practice in teaching.
- #5 Did they take away what you thought they were going to take away? If not, why? Threshold concepts: “core tenets in a particular discipline that are transformative, irreversible, integrative, bounded, and potentially troublesome” MEYER and LAND They tend to be highly constructive
- #7 In many cases, these words are used interchangably. Some people are moving in favor of Outcomes over objectives, because the term objective has become muddled over the years, so that now, objectives are sometimes written as teacher-centric instead of user-centric. That means that the language is geared toward what I will do (as the teacher), rather than what you will be able to do (at the end of the class session).
- #8 Our first step in good, quick assessment is to write learning outcomes. The assessment questions which you choose will, in general relate specifically to those learning outcomes. They will also provide a roadmap for you, as the instructor, in terms of what you must cover. Whatever assessment method you choose later will tie back to these. A good learning outcome will state what the student will be able to do, and WHY. http://valenciacc.edu/facultydevelopment/tla/Candidate/documents/ExamplesandDefinitionsofLOs.pdf
- #9 The basis for this assessment model
- #10 http://1.bp.blogspot.com/-UQSCrTCqVfw/UXNnUfiX0JI/AAAAAAAAFUE/rIYY8bSN0Hc/s1600/Blooms_Old_New.jpg
- #13 First step: consider the top three or four class priorities—two to five on the outside. Write them down. If you are going to do some basic things then you can have more, like four or five. If you want to start mixing in larger concepts—things that are likely to trip them up and require some deep thought to understand--then you may find that you can’t fit in as many—but also, you may find that the way you teach is transformed, and that those more specific pieces aren’t necessary. Think more broadly than “they will know which ten clicks will lead them to a perfectly refined search. Rather, help them understand information types and why they care about them in the first place. Don’t be afraid to focus on “Big ideas and enduring understandings” Do they understand why they would use a scholarly article search versus a Google search? Do they understand why the scholarly conversation is critical? Can they identify a scholarly article and its parts? Learning outcomes should stem from a needs assessment. The needs assessment will determine the gap between the students’ existing condition and a desired condition—that condition is usually an increase in knowledge, skills, or attitudes. Remember that these are just a pithy starting place for you. One outcome may take almost the full hour to unpack.
- #15 Future tense—timeframe Identifies learner
- #16 Benjamin Bloom, an American educational psychologist, helped to write these in 1956. Domains of learning. This chart represents the updated version. Cognitive domain.
- #17 http://www.celt.iastate.edu/teaching/effective-teaching-practices/revised-blooms-taxonomy Factual knowledge: bits of information Conceptual knowledge: more complex, organized knowledge forms Procedural knowledge: knowledge of how to do something Metacognitive knowledge: “knowledge about cognition in general as well as awareness of and knowledge about one’s cognition.” page 27, Anderson and Krathwohl-- can take place in all, but they chose to separate.
- #18 This should include context as often as possible without being too wordy or jargon-filled. So, for example, in this last I’ve said “advanced techniques” rather than saying boolean logic, subject header searches. You may want to cover those things, but probably not in your outcomes. You don’t have to have the context, but I like it. If you don’t include it in your written outcomes, still think about it and be able to articulate the context to the students.
- #19 Stem and outcome Demonstrable/assessable Bloom’s cognitive domain—high level No jargon Often INCREMENTAL—step-based. They fit together to make a whole. They also, if you show them, provide a sort of roadmap for the students. Giving them a meaningful outline of the topics you’re going to cover is shown to help students engage with the class more thoroughly. Let me again address the threshold concept piece of this. I hope it’s clear from each of these how they can be jumping off points for in-depth and transformative conversations, rather than point and click sessions. You can use them as a basis for teaching them the WHYs and WHENs of applying these skills, rather than just the HOWs.
- #20 Learner not identified—teacher focus Not assessable Poor verb No verb Jargon, too specific HUGE in scope
- #24 We’re going to talk about each and which might be best for you, given your setting.
- #26 Polls are good for feedback throughout the session--this kind of assessment is called formative assessment OR just at the end, for summative assessment. Polls are great if you have information that builds and is cumulative. So, you can get them to answer a question or two before you move on to the next topic. They’re also a fun active learning exercise and they give you immediate feedback. The questions have to be simpler, with no short answer or essay, but you can have more of them. You can set them up to be embedded on a site/LibGuide (if your students have computers), or use free tools to set them up as text messages. can take those polls and embed them into PowerPoints, and then the answers will show up there (PollEverywhere). They can also just raise their hands, of course!
- #27 Regardless of what method you choose to deliver the content (phone, embedded poll, clicker) you can have different question types. Multiple choice, yes/no. These probably seem basic, and that’s because they are, but they are often the basis for everything you’re trying to share with your class. If they have a shaky foundation, it means you can’t build a house on it and your efforts will be a failure, so these aren’t always exciting, but they’re critical. Substantive or friendly—can also help you to set the stage for what you’ll cover in the beginning. I know these aren’t hifalutin, but it’s hard to build on a shaky foundation. These can be your foundation questions. These can be great, because they are anonymous in the moment. So, if someone feels shy about raising a hand to say s/he is confused, this can help them to do it without drawing attention to themselves.
- #28 These can get into those higher level thinking skills and get students to really demonstrate understanding, application, evaluation, etc.
- #32 You can position these however you like in your class time. I used to deliver them about ten minutes from the end of class, give the students five minutes, and then use the last five minutes to go over the answers quickly and take one last stab at fixing any confusion issues that arise. You may want to have them answer the questions right when they come in, then discuss what they wrote as a group. It’s a great way both for you to get a sense of where the students are from the start, and to help them apply your teaching to something they have written in front of them. You can also do a pre and post questionnaire. With both quizzes and polls, you can easily convert your learning outcomes to question, if they’re carefully drafted. So, “You will know how to construct a search statement becomes “construct a search statement.” Not more than five “meat” questions, maybe a couple of close out questions that are a bit more free-form.
- #33 The intent of this first question is to learn what they’re anxious about and what I should focus on. The second one is to hear their thoughts on why the library is important. Often they come up with abstractions like “my teacher says I have to” and that can actually be a fabulous jumping off point for a discussion about source credibility. The third one tends to go much the same—they say things like “I click the peer-reviewed button.” When covering these, I’m always careful not to shut them down when their answers are off target—rather, I talk about how we are now operating in a different framework—the research framework—and that that is different from the open web. The fourth one has a specific function, and that is to ask them to apply what we are about to talk about to something already in their minds. This kind of anchoring technique can help them to relate what you’re saying to their reality,a nd therefore, find more meaning in it.
- #35 Pre and post, or just post. Pre and post can gauge learning, post can gauge what they’re walking out with. If you can get teacher cooperation, you can also do them again as a follow up, to gauge retention on various time scales. Some of these are open ended, some multiple choice. It’s a good idea to have a variety. Not too many easy ones, or you may not learn anything useful. You can also format these as quick assignments. Great for classes with one set task—evaluate and cite websites, for example. You can also treat these like scavenger hunts, if you have enough time, and ask the students to get up and locate things.
- #36 You don’t need to be perfect your first time. If your questions aren’t perfect the first time or your instruction isnt perfect, that’s okay You will realize even as you get more accomplished, that you sometimes don’t say things perfectly! You can correct that on the spot. You can use answers as an opportunity to congratulate them for their awesomeness.
- #37 What is a minute paper? You have them write or type for just one minute—only two real questions (and maybe some basic information). On paper or online is your choice. It’s easier to tabulate things online though. Drawbacks: It’s hard to evaluate these in the moment, but you can. It’s also very hard to tie these in with your objectives—however, they are useful. I like to do a combination of hand-raising polling, with a post quiz that has a minute paper attached.
- #38 Gives them space to clarify and explain, if they want to. You can also invite them to leave contact info.
- #40 With polls and quizzes, if they’re brief, it’s easy to tabulate and review responses immediately. I like to do this. Skim over the answers and address problems right there. “It seems like I was unclear on Point X. Can someone describe the answer? HAVE YOUR ELEVATOR SPEECHES READY. This is the most important step. If you don’t integrate the information, it’s not useful to anyone. We’re not perfect—this kind of thing can be jarring at first. Change can relate to the way the questions are worded, ordered, or what is included, and also in the ways you teach.
- #41 Create a warm and welcoming environment by walking around in the beginning of class and during. Insure the students that the process is difficult and messy, and the assessments are intended to insure that they have what they need and have a chance to ask questions while you’re there to help. Let them know the things you learn from their struggles help you to improve the classes for future students. Appeal to altruism=awesome.
- #43 Most of these can be used for both.
- #44 Absorbed Zoomerang Basic: free 10 questions per survey 100 responses per survey No fancy things like skip logic Select: $26 per month ($312 per year) Unlimited questions Unlimited responses Custom urls Skip logic etc.
- #45 Can handle short answer, you can embed images, and it does quizzes and surveys in addition to polls. If you’re asking what the difference is between a quiz and a survey in terms of this software, if you structure your setup as a quiz, you can set it to know the correct answers in advance. Free account: Used to be limited to 100 responses per month, but they took that away. So, the free version should be everything you want unless you want custom urls.
- #46 This is usable with any computer, tablet, or phone—even old ones, and it can handle short comment answers. You can embed in PowerPoint. Free version 25 responses per poll or 40 responses per poll if you are in K through 12 or higher education. They don’t love public libraries, apparently. The pricing beyond that point gets kind of complicated based on your situation, but it’s $14 per month and up.
- #47 Free, but somewhat clunky to use.
- #50 List of verbs: http://www.acu.edu/academics/adamscenter/course_design/syllabus/verbs.html