Assessment: Thinking on puzzle pieces

"puzzle pieces" flickr photo by cadavis9797 https://flickr.com/photos/christine_davis/5499010245 shared under a Creative Commons (BY) license

Continued thinking here on assessment
As a critical piece(s) in a learning picture
Drawing from personal experiences
And readings
And my beliefs on learning
Wanting to more deeply understand the potential of leveraging the power of technology to transform assessment

PERSONAL EXPERIENCE (excerpted from an earlier post)

Out of sync
Always wanting better
Not in agreement
Time and time again

More than 45 years ago, GRE exams
My answers and scores significantly impacted by the recent death of my mom
Not a reflection of my learning at all 

More than 30 years ago, a Master's thesis and comprehensive exams
Real opportunities to demonstrate what I had learned 

More than a decade ago, applying for a new position and answering a question on assessment
Replying that the process and the projects themselves illustrated what students had learned 

In that new position, creating rubrics for projects for high schoolers aspiring to become teachers
Knowing that the state used the same type of rubrics to assess new teachers
Crying in the car after being told to change the rubrics to written tests by an old school CTE administrator 

Almost 14 years ago, with a terrible cold, an appointment to take the assessment component as a NBPTS candidate
Timed testing, no kleenex allowed in the testing room, cranking out as fast as I could responses to 6 questions
No time to think 

High stakes testing even for our beloved Harley as his obedience training classes came to completion
In the dark in a strange park, he would not stay as I walked away passing 9 of 10 tasks and failing
He curled up in the corner of the back seat with his head down on the way home 

Some 6 years ago, designing and creating online professional development courses for Ohio teachers
Always pushing back against suggestions of quizzes, of tests
Pulling for learner created content and powerful questions that enabled deeper learninG 

Assessment--
Grades, points, projects, quizzes, standardized tests, rubrics-- 

And then MOOC 2008 with vast learning landscapes, autonomy, openness and distributed learning
I assessed my learning in the open here on this blog (in the blog search box, enter CCK08 for 6 pages of posts)
That learning experience was empowering and scary and full of wonder

HEUTAGOGY (as a foundation)

Heutagogy (based on the Greek for “self”) was defined by Hase and Kenyon in 2000 as the study of self-determined learning. Heutagogy applies a holistic approach to developing learner capabilities, with learning as an active and proactive process, and learners serving as “the major agent in their own learning, which occurs as a result of personal experiences” (Hase & Kenyon, 2007, p. 112). As in an andragogical approach, in heutagogy the instructor also facilitates the learning process by providing guidance and resources, but fully relinquishes ownership of the learning path and process to the learner, who negotiates learning and determines what will be learned and how it will be learned (Hase & Kenyon, 2000; Eberle, 2009). 
When designing a self-determined learner experience, certain considerations should be made. A heutagogical approach to learning and teaching is characterized first and foremost by learner-centeredness in terms of both learner-generated contexts and content. Course design elements that support learner-centeredness in a heutagogical approach are presented below.
 Learner-defined learning contracts: Learning contracts support students in defining and determining their individual learning paths. These individualized contracts, such as those used at distance education institution Empire State College (see www.esc.edu), define what will be learned (e.g., scope), how it will be learned (e.g., teaching and learning approaches, learning activities), and what will be assessed and how it will be assessed (Kenyon & Hase, 2010; Gilbert, 1975; Cristiano, 1993).     

Flexible curriculum: In a self-determined learning environment, the learner is the driver in creating flexible curriculum, which is defined by the student: learners create the learning map, and instructors serve as the compass (Hase & Kenyon, 2007; Hase, 2009). Flexible curriculum in this sense is negotiated action learning, which adapts and evolves according to learner needs (Hase, 2009; Hase & Kenyon, 2007). Learners negotiate “how, when, where and to what upper (rather than minimal) level they want to take their learning” (Hase, 2009, p. 47).     

Learner-directed questions: Learner-directed questions and the discussion that results from these questions are what guide learners and serve as mechanisms for helping learners make sense of course content, bring clarity to ideas, and promote individual and group reflection (Kenyon & Hase, 2001; Eberle, 2009). Guiding learners to define self-directed questions is one of the biggest challenges facing developers of heutagogical courses, as designers must be “creative enough to have learners ask questions about the universe they inhabit” (Kenyon & Hase, 2001, para. 29).     

Flexible and negotiated assessment: In heutagogy, the learner is involved in designing his or her assessment. Negotiated and learner-defined assessment has been shown to improve the motivation of learners and their involvement in the learning process, as well as make learners feel less threatened by instructor control of their learning process (Hase & Kenyon, 2007, p. 115; Hase, 2009; Ashton & Elliott, 2007; Canning, 2010). One way of incorporating negotiation into the assessment process is through the use of learning contracts (Hase, 2009). The assessment should include measurable forms of assessing understanding of content, including whether the learner has achieved the competencies desired. Rubrics can also be used effectively in guiding learners in their self-assessment process, for example by assessing “discussion skills, quality of work, outcomes, collaboration, academic soundness and knowledge of material” (Eberle, 2008, p. 186).

Another dually important characteristic of heutagogy is that of reflective practice, “a critical learning skill associated with knowing how to learn” (Hase, 2009, p. 49).http://www.irrodl.org/index.php/irrodl/article/view/1076/2087

ASSESSMENT AS LEARNING 



Dean Shareski talking assessment as learning
Of documenting learning along the way and reflecting
And the big advantage that technology affords  
Documenting with blogs and vido
That enables feedback from others 
And opportunities to self assess quality of work

LEARNING CONTRACTS
from Cathy N. Davidson

Why Student-Centered Learning Needs An Alternative Credentialing Mechanism Cathy N Davidson

This is the sixth in a series of blog posts I’m writing based on my own experiences with student-centered, open, peer, or connected learning (you can choose whichever term suits you: I’m agnostic about the terminology). My purpose is to offer step-by-step advice about the thinking, methods, assumptions, and practical choices that go into redesigning a classroom inspired by equality, not oppression (to use Paolo Freire’s famous terminology). A pedagogy of equality aims to support and inspire the greatest possible student success, creativity, individuality, and achievement, rather than more traditional hierarchies organized around a priori standards of selectivity, credentialing, standardization, ranking, and the status quo. 

That, of course, is the most binaristic way of framing the redesigned student-centered classroom. However, in the real world in which most students live, if they are paying tuition, they also want something more concrete than a sense of their own learning: they want some formal, institutional recognition of the effort they have invested in their learning. (Otherwise, why not just learn from a friend or from a book or online?)

That is where contract grading and peer evaluation come in. To my mind, they are the most expansive alternatives to conventional grading while still offering the student a meaningful, documentable, responsible credentialed form of credit for learning attainments. 

And from Dave Cormier who has an excellent reference list at the end of his post:

You need to ACTUALLY be open to student control
The one thing that everyone seems to agree upon is that shaping the course for themselves is the critical element to contract grading. If you create a situation where the contract exists, but students get little or not input into how its carried out (say you set things up where choice is very robotic, or checkbox like) it will not work.

And my personal experience with learning contracts (with learner examples), which became learning pledges at a learner's suggestion.

AUTHENTIC ASSESSMENT
From the BlendKit2017 Week 3 reading:

Authentic assessment—assessing student abilities to apply knowledge, skills, and attitudes to real world problems

Observations, presentations, videos that document learning
With feedback from global experts-- authors, historians, scientists
Collaboration with experts in the field-- gathering data
Presenting to professional meetings
Moving to collective action projects to make a difference
And leveraging the technology to collaborate, share
Examples from Michael Wesch

And

 

ASSESSMENT 2.0
Lots to think on here; an excerpt below:

Characteristics of Assessment 2.0
The type of assessment activity best suited to the digital native would exhibit some or all of the following characteristics.

• Authentic: involving real-world knowledge and skills.
• Personalised: tailored to the knowledge, skills and interests of each student.
• Negotiated: agreed between the learner and the teacher.
• Problem oriented: original tasks requiring genuine problem solving skills. Socially constructed: using the student’s social networks.
• Collaboratively produced: produced in partnership with fellow students.
• Recognise existing skills: willing to accredit the student’s existing work.

And the type of evidence that best fits this type of assessment would be:

• naturally occurring: already in existence or generated out of personal interest
• digital: such as e-mail, instant message logs, blog posts, wiki contributions, audio and video recordings
• multimedia: existing in text, audio and video format
• distributed: may be scattered across various sources (such as web sites, blogs, inbox, iPod).

FORMATIVE ASSESSMENT (last but so important)

"Powerpoint Slide: "The power of formative assessment"" flickr photo by Ken Whytock https://flickr.com/photos/kenwhytock/8516967532 shared under a Creative Commons (BY-NC) license

The 5 Formative Assessment strategies to improve learning
•1. Clarifying, sharing, and understanding learning intentions and criteria for success – getting the students to really understand what their classroom experience will be and how their success will be measured.  

•2. Engineering effective classroom discussions, activities, and learning tasks that elicit evidence of learning – developing effective classroom instructional strategies that allow for the measurement of success.  

•3. Providing feedback that moves learning forward – working with students to provide them the information they need to better understand problems and solutions.  

•4. Activating learners as instructional resources for one another – getting students involved with each other in discussions and working groups can help improve student learning.  

•5. Activating learners as owners of their own learning

Technology again, transforms the possibilities with Formative, Padlet, Tricider, and blog posts that ask learners to make their thinking visible. 

So many pieces to think on
To adopt the best lens for assessing learning
Always a puzzle picture in progress
Always room to make it better
Always seeking approaches that support learners in enhancing, deepening their own learning

To be continued-- this puzzle always needing additional pieces

She could not say it--

Full disclosure: I have permission to blog about this conversation.

It was going to be an easy conversation
Or so I thought

A Connected Coaching eCourse participant
Self assessing her learning and sharing with me the grade she felt she had earned and why

It was the first day of school for her and she had just returned from a first session of music with second graders. She shared with great delight how long it had taken for them to finish one simple task. We giggled and laughed together over the story--

It was such a good day-- the joy enveloped our Skype call.

The conversation turned toward her learning- "Had she found the content of value?" I asked. "How are you feeling about your learning?"

Without hesitation, she described what had been of greatest value to her and how it would be instrumental in the work she would be doing during the course of the year. She added a concern she had had, unwarranted, about "taking over conversations". She asked about a learning opportunity the group let pass them by (creating a collaborative rubric).

There was a silence--

So I asked, what did she feel should be the grade I reported to the institution.

There was silence--

Then she asked me what I thought it should be.

And we both laughed.

And I asked, why is self assessing so difficult?

And we laughed again.

And she said her second graders were very good at it as were her daughters--

And I asked her again.

And again there was silence.

She never could bring herself to say it.

I finally asked, both of us laughing some more, "Does the letter have points or curves?"

"Points" she said.

To which I replied, from what you have told me, of course, it is an A.

Why is self assessing for very smart learners so hard? We didn't use video in the Skype call; did that make a difference when we couldn't read each other's body language? Had we not developed enough trust between us?

Lots to think ---

Image: 'it can't be true! you're so posh,+more+than+me!'
http://www.flickr.com/photos/47207654@N00/47651477
Found on flickrcc.net

Thinking on assessing learning--

Out of sync
Always wanting better
Not in agreement
Time and time again

More than 45 years ago, GRE exams
My answers and scores significantly impacted by the recent death of my mom
Not a reflection of my learning at all

More than 30 years ago, a Master's thesis and comprehensive exams
Real opportunities to demonstrate what I had learned

More than a decade ago, applying for a new position and answering a question on assessment
Replying that the process and the projects themselves illustrated what students had learned

In that new position, creating rubrics for projects for high schoolers aspiring to become teachers
Knowing that the state used the same type of rubrics to assess new teachers
Crying in the car after being told to change the rubrics to written tests by an old school CTE administrator

Almost 10 years ago, with a terrible cold, an appointment to take the assessment component as a NBPTS candidate
Timed testing, no kleenex allowed in the testing room, cranking out as fast as I could responses to 6 questions
No time to think

High stakes testing even for our beloved Harley as his obedience training classes came to completion
In the dark in a strange park, he would not stay as I walked away passing 9 of 10 tasks and failing
He curled up in the corner of the back seat with his head down on the way home

Some 6 years ago, designing and creating online professional development courses for Ohio teachers
Always pushing back against suggestions of quizzes, of tests
Pulling for learner created content and powerful questions that enabled deeper learning

Assessment--
Grades, points, projects, quizzes, standardized tests, rubrics--

And then MOOC 2008 with vast learning landscapes, autonomy, openness and distributed learning
I assessed my learning in the open here on this blog (in the blog search box, enter CCK08 for 6 pages of posts)
That learning experience was empowering and scary and full of wonder
And my thoughts on assessment are shifting --again

WHO/WHAT CONTRIBUTED TO THE SHIFT
Very honestly, I've never been a real quiz, test person. In the classroom, I had conversations, I observed, I watched process and explored projects. With proficiencies and standardized testing, I was pushed to change. Conflicted, some testing entered the learning experiences of my students-- not without tension.

Now I'm old, less disposed to "push"-- more likely to follow my instincts---  especially when I've had an experience such as MOOC 2008 and I'm finding other educators seeking a shift/making a shift too.

Sheryl Nussbaum-Beach undoubtably has provided the most fodder for my thinking. She has articulated in so many places a vision of what could that truly resonates with me. That wayfinding, co constructing meaning and self directed processes play a huge role in shifting is an understatement.

Let the co-learners in the room decide context- the how of what we will share, the teaching and learning processes and the why of how we learn it this way.

Let us do our own wayfinding. Let us construct our own meaning and design, our own sense making activities. Let us show mastery of what we learned in ways that align not only with the standards but also with the self-directed processes and creative ways in which each learner chooses.

maybe we need to standardize (make it business as usual) that learning should be self-directed and knowledge co-constructed.
http://plpnetwork.com/2011/03/12/disconnect-content-context-common-core/

Anne Fox joined the group of PLP Connected Coaches last fall and in a conversation shared this resource on Heutagogy. Totally new to me, very exciting as learning moves from self directed to self determined. Maybe it's because so much of my professional learning was self determined (I rarely felt that the organized PD that was a staple in my career was of much value) that this notion speaks to me so clearly as does the accompanying concept of developing capabilities. This resource has had a critical impact on my thinking.

Heutagogy (based on the Greek for “self”) was defined by Hase and Kenyon in 2000 as the study of self-determined learning. Heutagogy applies a holistic approach to developing learner capabilities, with learning as an active and proactive process, and learners serving as “the major agent in their own learning, which occurs as a result of personal experiences” (Hase & Kenyon, 2007, p. 112). As in an andragogical approach, in heutagogy the instructor also facilitates the learning process by providing guidance and resources, but fully relinquishes ownership of the learning path and process to the learner, who negotiates learning and determines what will be learned and how it will be learned (Hase & Kenyon, 2000; Eberle, 2009).

When designing a self-determined learner experience, certain considerations should be made. A heutagogical approach to learning and teaching is characterized first and foremost by learner-centeredness in terms of both learner-generated contexts and content. Course design elements that support learner-centeredness in a heutagogical approach are presented below.

    Learner-defined learning contracts: Learning contracts support students in defining and determining their individual learning paths. These individualized contracts, such as those used at distance education institution Empire State College (see www.esc.edu), define what will be learned (e.g., scope), how it will be learned (e.g., teaching and learning approaches, learning activities), and what will be assessed and how it will be assessed (Kenyon & Hase, 2010; Gilbert, 1975; Cristiano, 1993).
    Flexible curriculum: In a self-determined learning environment, the learner is the driver in creating flexible curriculum, which is defined by the student: learners create the learning map, and instructors serve as the compass (Hase & Kenyon, 2007; Hase, 2009). Flexible curriculum in this sense is negotiated action learning, which adapts and evolves according to learner needs (Hase, 2009; Hase & Kenyon, 2007). Learners negotiate “how, when, where and to what upper (rather than minimal) level they want to take their learning” (Hase, 2009, p. 47).
    Learner-directed questions: Learner-directed questions and the discussion that results from these questions are what guide learners and serve as mechanisms for helping learners make sense of course content, bring clarity to ideas, and promote individual and group reflection (Kenyon & Hase, 2001; Eberle, 2009). Guiding learners to define self-directed questions is one of the biggest challenges facing developers of heutagogical courses, as designers must be “creative enough to have learners ask questions about the universe they inhabit” (Kenyon & Hase, 2001, para. 29).
    Flexible and negotiated assessment: In heutagogy, the learner is involved in designing his or her assessment. Negotiated and learner-defined assessment has been shown to improve the motivation of learners and their involvement in the learning process, as well as make learners feel less threatened by instructor control of their learning process (Hase & Kenyon, 2007, p. 115; Hase, 2009; Ashton & Elliott, 2007; Canning, 2010). One way of incorporating negotiation into the assessment process is through the use of learning contracts (Hase, 2009). The assessment should include measurable forms of assessing understanding of content, including whether the learner has achieved the competencies desired. Rubrics can also be used effectively in guiding learners in their self-assessment process, for example by assessing “discussion skills, quality of work, outcomes, collaboration, academic soundness and knowledge of material” (Eberle, 2008, p. 186).

Another dually important characteristic of heutagogy is that of reflective practice, “a critical learning skill associated with knowing how to learn” (Hase, 2009, p. 49).
http://www.irrodl.org/index.php/irrodl/article/view/1076/2087

Dean Shareski  has recently reflected on assessment as he teaches again his pre service online course at the U. of Regina. When I read his post, I reread the the sentence "The biggest change this term was to have my student's assessment themselves for the entire course." and I read it again. I like that and his "I really don't care much about the grade at all." as I don't either. It took me back to my learning in the 2008 MOOC-- of how much I learned, of how excited I was.

The biggest change this term was to have my student's assessment themselves for the entire course. In the past, it was required for a few assignments but not all. This term I was clear that they were going to tell me their grade and justify it. As long as their documentation was clear and from my perspective truthful, that's the grade they would receive. I suppose in some respects, I'm still assessing, assessing their assessments but my goal was to do two things. First to empower them to think deeply about their learning. While I've always advocated for reflection, I tried to emphasize more documentation. I still need to structure this better but that was my intent. Secondly I wanted the pressure of grading to be removed from learning.

I really don't care much about the grade at all. I'm interested in their documentation of learning and how I might support them and indeed provide new and better opportunities for them in the future and for my future students as well.
http://ideasandthoughts.org/2012/04/18/adventures-in-assessment/

Dave Cormier  has been talking assessment around the course he began teaching this spring. Over the years I've been learning with Powerful Learning Practice, I've become convinced as is he that "making meaning, creating knowledge, is something that happens in public". And so his students are negotiating grading contracts with him.

I’ve been lacking a way to bring a method of assessment to the course that reflects the philosophy of education I’ve been working with. The idea of saying that you understood 92% of the ‘right’ way of seeing something is the exact opposite of the way that I see this course. From a traditional perspective… I want you to cheat. I want you to ‘get the answer’ from your neighbour. I want you to tell me that you did that… but more importantly, I’m hoping that you’ll tell each other that. So the contract measures how much work you’re doing… How much you are contributing. And, if you take anything from this course, is that making meaning, creating knowledge, is something that happens in public.
http://davecormier.com/edblog/2012/05/16/why-we-work-together-cheating-as-learning/

Student work in this course is evaluated by ‘contract’ – meaning that each of you decide how much work you would like to do for what grade.
http://davecormier.com/edblog/2012/05/09/ed366-learning-contract-prior-to-student-input/

The contract grading approach, loosely speaking, is one where the student and the instructor negotiate a ‘contract’ for how that course is supposed to go.

Here is an excellent example from Cathy N. Davidson that I’ve been borrowing from. (or, as she suggests, pilfering from)
http://davecormier.com/edblog/2012/05/07/grading_contract/

I'd rather contract grading become contract learning or rather a learning contract. I'm starting to put pieces together. Learning in public, self assessment, learning contract, no talk of failure--

Cathy N. Davidson  was one of the resources to which Dave Cormier provided a link. With her contract grading the learner decides on what she/he wants or needs from the course (yes I remixed that too from "what grade"); the learner empowered to follow personal passions and needs truly resonated with me. If you follow the link below the quote, Cathy did not create her grading contracts with that in mind; yet her words "what you need or want" are important in my thinking.

The advantage of contract grading is that you, the student, decide how much work you wish to do this semester; if you complete that work on time and satisfactorily, you will receive the grade for which you contracted.  This means planning ahead, thinking about all of your obligations and responsibilities this semester and also determining what grade you want or need in this course
http://hastac.org/blogs/cathy-davidson/contract-grading-peer-review-heres-how-it-works

WHY THIS MATTERS NOW
As the facilitator for the PLP Connected Coaching eCourse  I had to submit a syllabus to NDSU; we wanted to provide an opportunity for learners to earn graduate credit if they so wished.  Required by the credentialing institution, I laid out assignments and their influence on a total grade to be earned (no points, percentages).  Not entirely comfortable with that, I've been thinking hard on ways to incorporate all that resonates with me best for learning and assure that learners in the course who seek to earn graduate credit have that option.

In addition, I wanted the course to be of value to the diverse range of educators who were interested for varying reasons. Some wanted to become PLP Connected Coaches, some were face to face coaches and wanted to learn about strategies for coaching in online spaces, some were classroom teachers who wanted to move to a more inquiry based classroom and sought insight on facilitating communication. Some were in it to go deep; others more on the surface. My quandary, a way to facilitate deep learning that met the needs of everyone, remained true to the inquiry based model of the course, and promoted self governed learning.

HERE'S MY REMIX
Adapted, adopted from those who have gone before me, with a lot of Lani sprinkled throughout, I requested that all learners create a learning contract. Really anxious, very much believing in it's potential, yet still nervous I posted this and asked each learner to post theirs in our asynchronous virtual space.
Here it is: https://docs.google.com/document/d/1qWQgDmtsJflNfLbS3OaJHHg2h-YtjRCBWHtauQSVSZY/edit

It's not perfect. I haven't figured out how to give those taking the course for credit more options although I have personally encouraged them to add their own personal learning goals, to negotiate percentages for the graduate assignments, to create a timeline for demonstrating their learning that best meets their own personal schedule within the time limits of the course. I've asked them to determine how they will show evidence of their learning and to assess themselves using strategies of their own choosing. In the end, the reality is, I have to submit a grade; we'll Skype or phone; and if there is a need, we'll negotiate--  I am feeling at this point, this is a step in the right direction. If it proves to be of value in the learners' eyes, I will attempt to modify the syllabus for the university to more closely reflect this philosophy of learning.

AND THEIR INITIAL REACTION
2 weeks of learning in, I asked for initial reactions (yes I remain anxious and nervous, very much wanting this to be of value) in a recent webinar. Participants placed red stars on the value line on the whiteboard to indicate their feelings. This coupled with comments -- "scary but valued, learning about me,respecting the learner" --encourage me.

Stay tuned-- In late August when the formal course time for learning has come, I'll be back here sharing what I learned, what we learned --

 Image: 'Thinking & Playing'
http://www.flickr.com/photos/43433634@N05/6919665797