On Rubrics, Critical Thinking and Five-year Olds

Earlier this afternoon I met with the CTLT Learning Designers to look at some of my daughter Karina’s Montessori work and think about rubrics that I have been writing for Palouse Prairie Charter School that I am helping to create in Moscow, Idaho.

The work that attracted the most attention of the Designers was a bound portfolio from Karina’s first semester and an unbound collection of the same type of work from her second semester. The portfolio is a chronological collection of all the work of this type. I believe the unbound collection to be complete. It also has a few items from her 4th semester added.

From the discussion, I got a homework assignment, which was to ask Karina to select from the unbound collection “some” pieces that showed “her growth.” She and I talked before dinner about this task, and about how she had grown in ability in many ways. I regularly note this idea to her (couldn’t use a spoon, couldn’t get shoes on right feet, etc. and now can and does.)

The work in question is called “Metal Insets.” It is done with colored pencils on 5” square paper. A metal stencil is laid over the paper and its single simple geometric shape is traced. Then the student is to draw lines back and forth from edge to edge in the shape. “Quality Work,” the teacher’s term, looks like teeth in the mouth of a monster and this is called “jaws.” The lines that zig-zag back and forth are to stay within, but reach the perimeter outline.

The teacher showed us her notebook where this exercise is described and its primary purpose is to develop the hand and the skill of holding the pencil with a light touch, preparatory to penmanship. A secondary goal was to create a pleasing design or form, but the teacher stressed that metal insets are not art.

For this study 38 metal insets were examined (the unbound collection). A 39th was discarded during the process because it had a metal inset on front and back and we became confused as to which side we were examining. Karina does many metal insets that do not meet the “Quality Work” criteria as I understand it.

Karina was eager to help with my homework and sorted the pile into two groups while I was absent (it took about 2 minutes). One pile she called “Yes” the other “No.” I presumed that “Yes” was her denotation for “shows growth.” There were 10 Yes and 28 No.

I asked Karina for the rules for each category and wrote them down as she dictated. Words in brackets [] have been added to provide clarity
•    I stayed in the lines (mostly)
•    I like the shapes [of the stencil used to create the outline(s)]
•    [I used my (currently)] Favorite colors
•    I did “jaws”

•    Not my favorite colors anymore
•    I scribbled
•    Not my favorite shapes – I don’t like circles

To check how reproducible this classification was, I shuffled the two piles and then with her to check me, I classified each piece “Yes” or “No” giving my reasons from her list (above).  I was not able to do this work and keep records of my scoring. The biggest reason I seemed to be wrong in my classification was “favorite color.” “Not favorite color” was pretty easy (brown and black), but what makes favorite was harder. There must be a hue of purple and pink that are not in favor now which I could not recognize. Karina reclassified three (of 10) of  her “Yes’s” to “No.”  She also reclassified four (of 28) of her “No’s” to “Yes.” The reasoning for these changes was not clear.

I then found two more items that we had overlooked during our classification activities above. I evaluated each by her rules and found each a “No.” She agreed. These were set aside and not considered further. (One might be “Quality Work” the other is not.)

I then asked if she could adopt her teacher’s perspective and classify the 38 items as “Quality Work” or not.  The “Yes” pile now contained 11 items (10-3+4) and the “No” pile 27 items. I asked for the rules for “Quality Work” and wrote them down as she dictated:
* stayed in the lines [of the perimeter figure]
*  jaws
* color does not matter
* no scribbles

The 27 “No” pile was sorted into 5 “Quality work” and 22 not. The “Yes” pile turned out to have all quality work, despite my questioning what seemed to be scribbles and even lack of jaws on several. One of the pieces, dated May 10, 2006, has this annotation on the back from the teacher, “This is one of the most beautiful Metal Inset this whole year [smiley face] Cindy” In the procedure above, this item was first classified a “No,” and then later re-classified a “Yes.”  It was also classified “Quality Work.” It does not look like “Quality Work” to me, so the praise raises a concern to me that the artistic quality of the piece might impact its assessment.

Using the language of the Palouse Prairie School’s Critical Thinking Rubric (adapted from WSU’s Critical Thinking Rubric) I think Karina was demonstrating she could identify and summarize the problem, but did not articulate nuance or embedded issues. She also showed that she could identify her own a perspective, and it has some richness and is not her teacher’s cultural norm. She could also recognize the perspective of others (the teacher). She did not identify if this perspective was right or wrong, or if there might be yet more perspectives. (Another activity would be for me to classify the items with a rubric I create and see if she could understand and implement mine to sort the work.)

I then sorted the 38 items by date of creation (recorded on the back in a woman’s hand). Seven had no dates (six scored “No” and two scored “Quality Work”). The date range is 1/18/06 to 1/19/07. All but the last 4 items are from Spring 06 (ending May, 25) Of the dated items, the “Quality Work” pieces are scattered among the  dates with no obvious pattern, while the “Yes” pieces are mostly recent.

5 Responses to “On Rubrics, Critical Thinking and Five-year Olds”

  1. Ashley Ater Kranov Says:

    One, I’m very impressed, Nils, by how you facilitated this exercise in such a low key and fun manner. Not easy. It was interesting to me how aware she was of both the teachers standards for quality, as well as what she considered her own, and how they differed. She also very apparently valued her own standards – preferred shapes and colors – as much or more than her teachers’.

    I’m also very interested in the potential mixed message you described regarding the teacher’s applaud for a “pretty” drawing that did not fulfill her stated standards for quality metal insert work as a pre-penmanship exercise.

    I can’t wait to see Karina’s next round of drawings.

    As of now, I’m thinking that leading the parents through a similar exercise of guessing teacher intent (in relation to the rubric) and student fulfillment of outcomes based on the products (Karina’s), then revealing what the standards are, as well as this exercise and facilitating a discussion around that could be very interesting. We need, though, to be aware of the deep-seated need that many have for “take aways” at the end of any given activity and provide those. Looking forward to the next design circle.

  2. Tara Petrie Says:

    I think this is an interesting process, Nils.  I have a couple of perhaps random observations, inquiries, and curiosities.

    1)  First, Differences in Standards – Where Do They Come From? 

    I thought it was interesting that Karina seemed to be somewhat aware of the teacher’s standards for “quality work,” yet she still articulated her own criteria for “growth” – which really centered upon her own understanding of her changing or evolving stylistic preferences (for certain colors and shapes). Thus, Karina’s standards are/were internally generated based upon her understanding and reflection upon her own changing preferences.  On the other hand, the teacher’s standards apparently did not (explicitly) value preferences of that sort given that the nature of the learning outcome was explicitly NOT related to art.  Rather, the learning outcome of these activities is supposed to be the development of good pen-woman-ship skills.

    Perhaps there is a discrepancy here between what the student hopes to get out of such exercises and what the teacher hopes students will get out of such exercises.  Or, similarly, maybe there is a discrepancy between what the kid perceives the assignment to be about as contrasted to what the teacher perceives the assignment to be about.  My own feeling is that most kids will be perceiving the point of such metal inset exercises as art related.

    Speaking of the assessment of rationality, is there any research to suggest there are not other, more straight-forward ways to teach good penmenship without confusing kids with fun art projects that strike them (probably) as just that – art projects? I bet there are.  Karina’s standards seemed to be based on perhaps both what she thinks the assignment is about (art) and what she hopes to get out of the assignment measured by “growth” (her changing stylistic preferences). 

    I have a slight fear that if the kid’s perception of the project is that it is, or should be, an art project – then critical feedback left upon a piece that the kid may otherwise think is a very creative piece that represents creative (right-brained) growth — may only go to squash that kid’s creativity in favor of fundamentally different ways of thinking. And even if at some point that same kid can regain that earlier, more creative way of thinking – it may take some work and feel pretty uncomfortable to him/her given their experiences in school that discouraged it (even if unintentional).

    I wonder if you had framed the teacher’s standards to Karina – as a mutual and shared goal – that is – Karina’s own growth – whether or not, then, Karina’s own standards would have been more aligned to that of the teacher’s (not art-based)? That would speak volumes relative to whether or not Karina was actually understanding that there were in fact different standards in play and the fact she was consciously choosing her own.

    In other words, I think – as an exercise in critical thinking – it would be very interesting to explicitly explain to Karina what her teacher’s stated objectives were and what the teacher’s definition of growth entailed and ask Karina if she agreed with that definition of growth – why or why not?

    2)  What Do 5-Year Olds Really Need?

    Another interesting reflection to highlight is that I think the teacher’s assessment of Karina’s work, an assessment that seemed to ignore her own stated learning outcome goals/standard, might perhaps speak to what the teacher knows is *really* important for children at that age (positive feedback, motivation) more so than the teacher unconsciously or intentionally defying her own standards (you would need to candidly interview the teacher to figure this one out).

     3)  What about Gender?

    I also would suspect there may be an element of gender involved in the assessment practices. I wonder if the boys in the class got comments to the effect of – “This is one of the most beautiful metal insets all year.” I would guess (and it is a guess), that boys would receive feedback that really spoke more towards their abilities in fulfilling the “real” assignment goal – their ability to master penmenship. Perhaps the girls are being assessed on how pretty they can be and boys are being assessed on how well they are completing the tasks and learning the traditional skills supposedly asked for. You should totally check into this because if this is going on – what your daughter is being taught there is gender – not critical thinking.

    4)  Parent Involvement 

    I think getting parents involved in this process is an interesting idea. I think prior to this sort of engagement, though, a discussion with the teacher concerning his/her vision of total course goals vis-a-vis her/his expertise in developmental theory may be good background knowledge to have in understanding why they assess as they do.

  3. Laura Says:

    I find the assignment itself a little scary…requiring students to draw within the lines, but since this is actually for a penmanship exercise rather than an art exercise, hopefully they explain this to students and allow artistic interpretation during art class!

    I think that our usual discussion and development of rubrics may need to be adjusted for the age level. We need to research cognitive science sources to find out about children’s developmental stages so that we don’t expect critical thinking or judgment that they aren’t ready for.

    Also, I think there’s a certain genius to the early, non-judgmental attitude of an artist. Many famous painters try their whole lives to recreate that innocence of childhood in their work, that quality of enjoying art as a process rather than a product without judging. Why squelch it in children when it’s naturally there? When developing any rubric or assignment for children of age 5, we need to think about honoring the special ways they see the world and gaining something from that, rather than putting them into the adult dualistic boxes of “yes” “no”, “good” “bad”, “favorite”, “not favorite”, etc.

  4. YoonJung Cho Says:

    What I found most interesting is that Karina was able to articulate her own standard of evaluation as well as her teacher’s. It is also interesting that she still considers color and shape as important criteria although she is obviously aware that those criteria do not matter to the teacher. This indicates that there should be a rich conversation between students and a teacher in relation to what matters in fulfilling a particular activity as well as ‘why’ a certain criteria matters. Understanding how the ‘why’ makes a difference in quality work may be as important as just being aware that students and the teacher use different criteria.

  5. Gary Says:

    Is it time to bring in the teacher? (or has that invitation been sent already?)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: