E-Coaching Tip 64 (#3 Spring 2009)

Three Best Practices in Assessment

Our tips for spring of 2009 are focusing on important basics of online learning — spiced with a few new ideas. This tip reviews three (of many possible) best practices in assessing student learning. These practices help to serve as a review of the range — from simple recall to knowledge creation — of assignments that instructors can use to assess student’s performance (i.e., assign a grade). These best practices build on some of the “Designing by 3s” strategies from the earlier design tip this spring.

Of special interest to experienced online faculty might be the ideas on expanding student choices for course projects. These include creating course podcasts, webinars, talk show interviews, wikis and blogs; in other words, branching out into more media and formats.

Best Practice in Assessment 1: Assess across the six levels of cognitive skills of Bloom’s Taxonomy

This best practice extends the “Design by 3” practice of assessing (1) facts and concepts; (2) simple “doing” applications; and (3) more complex “creation” projects. An easy example is to think in terms of a course in math or biology. The first level of facts and concepts requires learning core vocabulary, concepts and discovery stories; the second level is hands-on exercises with relatively simple problems; and the third level is grappling with more complex problems, even including those problems with no known answers.

Bloom’s taxonomy (1956) and the updated version by Krathwohl (2002) each have six levels of cognitive processing. One difference between the two taxonomies is that the updated taxonomy uses verbs rather than nouns to indicate the type of cognitive processing going on. A second difference is that the top level in the updated taxonomy is “creating.” As we all enjoy the processes and results of creation, this is a fitting top-level experience.

The updated taxonomy is illustrated in the pyramid figure below. (See Figure 1.). The two foundation processes, (1) remembering and (2) understanding might be compared to the assessing of facts and concepts. The middle two processes of (3) applying and (4) analyzing might be compared to the simple “doing” applications, which includes manipulating and working with content. The two top processes, (5) evaluating and (6) creating are processes involved when learners are planning and creating complex course projects. These processes are also in play when learners review the work of peers and respond with critiques or commentary.

The pyramid that names the six core cognitive processes also serves as a good reminder that learning new skills requires a series of steps; and each step involves developing links or making changes to our existing knowledge base. You might want to print a copy of this pyramid and place in your office somewhere, or on you desktop as an icon.

blooms_triangle

Bloom’s Taxonomy updated by Krathwohl, 2002. Figure 1

How can you effectively assess the range of Bloom’s taxonomy? Here are a few strategies.

Assess facts and concepts:

Use the quiz function for basic vocabulary, discipline-specific seminal facts, concepts, ideas, quotes, story lore or biographies. Story lore can be part of learning the when, where, how and why a story became part of a discipline’s history. Remember that quizzes are best used for practice, recall and building a common cultural base. Consider soliciting questions from each group of students or having a Jeopardy-like game context. Grading of these quizzes is simply by completion; extra points might be given for submitting great, innovative, or thoughtful questions or being the first students to complete a quiz successfully.
Develop a discussion forum assignment that encourages integration of basic content knowledge with existing knowledge. The strategy of asking students why and how they know content or concepts and how they have used or will use the information in the future is one way of doing this.

Assess with simple “doing” tasks

Students do enjoy “doing” rather than just listening, reading or watching content resources. So short assignments can be very effective and can be assessed with simple rubrics and guidelines. Examples of short assignments include evaluating web sites, doing research to find similar, alternative or comparative content ideas, preparing short reports/news reports, podcasts, or short blogs. These short assignments can include elements such as sharing the tracking of how ideas evolve and identifying links with others’ ideas. These types of assignments require students to analyze, categorize ideas and detect relationships and patterns. These are all examples of good middle layer assessment activities.
Individuals or small teams can also create graphical representations of important processes and then “explain” the processes with current examples or objects.

Assess with complex “creating” projects

This level of assessment continues the theme that students enjoy “hands-on” activities that often include interaction with others. (But not necessarily so.) A major part of an online course grade is often a course project. Other ecoaching tips have encouraged designing course project requirements that can be personalized, and customized. These projects have three phases or points of review and assessment — proposal, enriched plan or draft, and final project. Course projects can be individual, team or group projects. Course projects are examples of the higher level of “evaluating and creating activities.
Most course projects follow a traditional pattern of either a paper or a report of some type. Today’s students might enjoy branching out from these traditional projects to create radio, television or Internet news shows, interviews, short presentations such as webinars, or lasting contributions to Wikipedia or course resource databases. If you have an extra few minutes, explore the kinds of film projects that students create in the Campus MovieFest 2008-09 World Tour. This project started over eight years ago and more than 200,000 students have ‘told their stories” via the big screen, learning movie-making skills such as writing, editing, and filming with current software and technology tools.
Higher-level assessment projects generally require more sophisticated assessment rubrics than are provided in the project assignment. Providing the rubrics in advance enable learners to self-assess and peer-review along the way of creation. The rubrics also serve to remove some of the subjectivity out of the final grading. This is particularly useful as projects tend to be more personalized and customized.

Best Practice in Assessment 2: Assess the core concepts in your course

This best practice requires thoughtful analysis of the course content by you as you are planning your course. Learners will only learn and take away a limited amount of knowledge and skill from your course. Determining what those core concepts are and then relentlessly focusing on those core concepts from multiple perspectives is what drives student’s acquisition of core concepts. Sometimes this work has been done for you by textbook publishers and are captured in the course performance goals. Reviewing just what those core concepts are and how the course experiences and requirements assist the learners in achieving those is part of the assessment design process.

See if you can answer these three questions?

What are the four, five, or ten core concepts to which you can link everything else in your course? (Can you build a concept map linking these core concepts?)
How do the course experiences assist the learners in “making those core concepts” their own and integrating them into their knowledge base?
What assessment tools will you use to gather evidence of your students’ grasp and understanding of those concepts?

Best Practice in Assessment 3: Help students succeed on assessment tasks

This is best practice #8 from the set of best practices, Best Practices in Assessment: Top 10 Task Force Recommendations from the American Psychological Association (APA). This best practice reminds us of the value to students of (1) explicit expectations; (2) detailed instructions; and (3) samples and models of successful performance for the assessment activities. This best practice also encourages providing opportunities for practice and detailed feedback. In other words, the best assessment is ongoing, and embedded into the learning experiences with no surprises. This means rubrics, feedback with multiple reviews (self, peer and expert). We want our learners to succeed.

Classic Resources on Assessment

Here is a short set of annotated resources on assessing student learning that you may want to explore.

This website — Internet Resources for Higher Education Outcomes Assessment — is probably best described as the granddaddy of all assessment resources on the net. It is hosted at North Carolina State University and currently contains about 1,000 links, including about 375 college and university assessment sites. Ephraim Schechter first posted the list in 1995 at the University of Colorado at Boulder. Later, it was hosted at the University of North Carolina system office and at North Carolina State University. Dr. Schechter still maintains this list. Retrieved January 2, 2010 from http://www2.acs.ncsu.edu/UPA/assmt/resource.htm.
The most cited list of assessment principles is 9 Principles of Good Practice for Assessing Student Learning. These principles were developed in 1991 under the auspices of the AAHE Assessment Forum with support from the Fund for the Improvement of Postsecondary Education. Authors of the principles include Alexander W. Astin, Trudy W. Banta, K. Patricia Cross, Elaine El-Khawas, Peter T. Ewell, Pat Hutchings, Theodore J. Marchese, Kay M. McClenney, Marcia Mentkowski, Margaret A. Miller, E. Thomas Moran, and Barbara D. Wright. Retrieved January 2, 2010 from ttp://www.iuk.edu/%7Ekoctla/assessment/9principles.shtml
Note: These assessment principles often combine principles assessing individual student learning within a course with principles assessing the learning of student cohorts. The set of best practices listed next is a good complement to this set.
This site - The Assessment CyberGuide for Learning Goals and Outcomes in the Undergraduate Psychology Major —is part of the APA professional site. One of the top resources is another set of best practices — Best Practices in Assessment: Top 10 Task Force Recommendations.
National Teaching and Learning Forum’s Frequently Asked Questions (FAQ) On College And University Teaching & Learning. This is a brief FAQ (1996 – 2001) useful for quick definitions of assessment and its role in learning and higher education. This FAQ also provides links to other sties on assessment. Retrieved January 2, 2010 http://www.ntlf.com/html/lib/faq/al-aahe.htm

References

AAHE Assessment Forum, 9 Principles of Good Practice for Assessing Student Learning. Retrieved March 4, 2009, http://www.uky.edu/Assessment/principl.htm

APA. The Assessment CyberGuide for Learning Goals and Outcomes in the Undergraduate Psychology Major Retrieved March 4, 2009 http://www.apa.org/ed/guide_outline.html

Bloom, B. S. and Krathwohl, D. R. (1956). Taxonomy of Educational Objectives, Handbook 1: Cognitive Domain. Addison Wesley Publishing Company.

Forehand, M. Bloom’s Taxonomy. In Orey, M.(Ed.). (2001). Emerging perspectives on learning, teaching, and technology. Retrieved January 2, 2010, from http://projects.coe.uga.edu/epltt/index.php?title=Bloom%27s_Taxonomy

Krathwohl, D. R. (2002). A revision of bloom's taxonomy: An overview. Theory into Practice, 41 (4), 212-218. Retrieved on March 4 2009 from http://findarticles.com/p/articles/mi_m0NQM/is_4_41/ai_94872707.

Overbaugh, R. C. &  Schultz, L. (n.d.) Bloom’s Taxonomy. Retrieved March 4, 2009, from http://www.odu.edu/educ/roverbau/Bloom/blooms_taxonomy.htm

Smythe, K. & Halonen, J.. Using the New Bloom's Taxonomy to Design Meaningful Learning Assessments. Retrieved March 4, 2009 http://www.apa.org/ed/new_blooms.html

Ecoaching Table of Contents