being an 'Excellence Judge'
Updated: Jan 24, 2020
In November 2019, I became an 'Excellence Judge' for the first time. Here, I describe how I found the process.
An 'Excellence Judge' is a someone who uses the process detailed in Beverly Serrell's 2006 book Judging Exhibitions: A Framework for Assessing Excellence with a cohort of peers. The Scottish Learning Group has formed an Excellence Judges cohort. A learning community, we have come together to actively train ourselves in finding excellence. Using a visitor-centered perspective, we seek out aspects of four Criteria: 'engaging', 'meaningful', 'reinforcing' and 'comfortable' at exhibitions and attractions. The process the Framework guides us to do this is: form a cohort (First Meeting), attend exhibition (Visit), discuss with your cohort (Second Meeting) and repeat steps 2 and 3. The purpose is: for the joy of learning! It's for your own CPD - not to produce a report for the exhibitors or anyone else.
We use a private Facebook group where members can find out e.g. the dates/times of the next Meetings. A detailed description as well as instructions on how to join our cohort, is in our earlier blog post, Introduction to Excellence Judges Framework.
1. cover of my copy of The Judging Exhibitions book and 2. the first of the Framework worksheets, which details how to have the First Meeting
My journey began when fellow Learning Wayfinder, Beccy Angus, lent me her copy of the Book. I read it and was energised - here was a way to codify experience as a visitor and gain a deeper understanding of learning, engagement and interpretation, both on my own and during group discussion. I knew that I wanted to set up a cohort of Scottish Learning Group members to do this work. In October 2019, I held our First Meeting.
first surprise: I now use the Framework to look at everything I do, from visiting sites to participating in training. I ask the four criteria: how is this 1 meaningful? 2 engaging? 3 reinforcing and 4 comfortable?
Serrell recommended amount of time: 2 hours
Amount of time it took us: 2 hours
cost: no fee to attend, I bought 1 chai tea latte
During the First Meeting, the cohort is formed. We learn the Framework's language, make sure everyone knows what they're doing, and, at the end, to choose the exhibition/attraction for the first Visit. In order to get this rolling, I set up the First Meeting via Eventbrite and called it 'Visitor Experience Training'. I thought that was wider than Excellence Judges. I advertised via social media to existing Scottish Learning Group members and others. Six people came along, and others had expressed an interest but were unable to make it.
Before the First Meeting, I skimmed the book and made overview notes. I was feeling a little overwhelmed by this point, even though I'd read the book months before.
There are new terms to learn (criteria, aspects, call-outs, etc) and even if you're familiar with the words per say, the point is you're using them as they're laid out in the Framework. The purpose of the cohort is not a detailed analysis of e.g. if you agree with the Framework's definitions of meaningful. The purpose is to use it as is.
Beccy walked us through the Framework: each stage of the the process and then a detailed look at the Aspects which make up each of the four Criteria. She gave us examples of each aspect that she's seen. This was invaluable. It was only once Beccy had gone through the process that I felt ready to do so, too.
I can see why this is such a vital part of the process, and I ask all 'new' members to the cohort to email email@example.com and arrange to meet with me so that I can do the same (i.e. provide a First Meeting).
Recommended amount of time: 1/2 hour to 2 hours
Amount of time it took me: 2 hours
cost: none - free entry
We had all chosen to see the refurbished Gallery 1 of the Museum of Childhood on Edinburgh's Royal Mile.
the exterior of the Museum of Childhood features a new sign and colours, which are continued into the new gallery (image: Becca Boyde)
Despite there being a month between the First Meeting and the scheduled Second Meeting, I was unable to get to our chosen exhibition until two days before the Second Meeting. I wasn't rushed or worried, though. I'd scheduled in the afternoon to do the process.
First, I reminded myself briefly of the four Criteria (Engaging, Comfortable, Reinforcing and Meaningful) but knew from Beccy's presentation that I didn't need to worry about having all aspects of those memorised. After all, the point of visiting is to experience the exhibition as a visitor.
During my visit, I used the Framework (PDF available here) as I went through the exhibition. I had printed them off and put them on a clipboard. As you can see in the link to the PDF, there are two sheets of A4 to do the call-outs during the walkthrough.
Immediately, I found that I had a lot to say, and I didn't like having the blank sheets to write on. [Tip: use a notebook to write notes during your visit.]
The point of the visit is to find mostly positive things to notice. It is far too easy to focus on the negative of anything, so part of the Excellence Judges Framework's job is to make you see what is appropriate to the exhibition itself. For example, at first I found the noise of a video playing too noisy, as I could hear it in the front of the exhibition. Later, however, when it was my turn to watch the video, I found that I needed the sound that loud in order to hear it over the other visitors. And so I concluded that the sound was not only appropriate to the exhibition, but also a good thing - it led me, through sound, to continue through the gallery.
I have rarely paid such complete attention to an exhibition before. I was so excited to be there and be doing this Framework that I'd read -and heard- so much about.
The Framework really opened my eyes and ears to what I was experiencing. As Beccy had said, once I got going I felt I 'knew' roughly what I was looking for, and found it. For example, an aspect of comfortable is orientation: I noticed that the angle of the exhibition wallboards formed a wavy path for me to go down - between that and the mention of the three themes of the exhibition (play, life and learn), I felt like I knew where to go. I also noticed that each theme's wall/exhibition space was colour-coded, and I don't think I would've consciously noticed that before.
pictures: Museum of Childhood exterior and Gallery 1 by Becca Boyde . I found that I wanted to take lots of pictures, both to remind me of my experience and to flag up interesting things that I'd noticed. I've been showing them to people every since and telling people to go to the exhibition!
I completed one full walk through of the entire Gallery, then I went back and took more pictures and had a think about things like how the exhibition showed the Criteria, especially the criteron of reinforcement as that is by its nature something that is built up through an (excellent) exhibition. Finally, I felt ready to leave.
Recommended amount of time: ?
Amount of time it took me: 2 hours, plus extra thinking time while I hung up the laundry
I went home and immediately began the next two parts of the process: Assessing the Aspects of the criteria, and Rating the Criteria. There is space for both of these on the Framework PDF and I appreciated having them there, so I'll use the PDF for those again.
I really enjoyed Assessing the Aspects. I copied them out into my notebook as headings and wrote bullet-point notes (referring to my Call Outs) for each one. Reading the aspects which make up the criteria and assessing what you've found against these was rewarding. I liked having a process that codified my thoughts and I felt like I'd have a lot to say at the Second Meeting.
[Tip: colour-code the Criteria and each of its corresponding Aspects. I highlighted Comfortable (green), Engaging (pink), Reinforcing (blue) and Meaningful (yellow). This made both writing up my notes and discussion on the night (of the Second Meeting) easier.]
This brought up things I'd not noticed during my visit. For example, without thinking, I had kept my winter coat on, but open, the entire time. Later, when the Framework asked me about how I found the temperature of the exhibition (an aspect of the 'comfortable' criteron), I concluded that it must've been fine because I wasn't messing about with my coat. It wasn't something that I had actively noticed. Another way of thinking of this is that you'd maybe expect to be cold at some point during an exhibition on the Titanic and hot during one about deserts - both of these would be appropriate to their exhibitions.
Of all the parts of the process, at the time I did not enjoy Rating the Criteria. This is figuring out a numerical score to give each Criteria, based on your Assessments of the Aspects. By this point I'd been doing the Framework for the afternoon. So I did this part quickly, and distilled my notes into a paragraph under each Criteria to illustrate why I'd scored it as I did.
Although you're 'scoring' something you aren't doing so for the sake of it. It is to guide discussion during the Second Meeting.
recommended time: 2 hours
amount of time it took us: 2 hours
cost: no fee to attend, bought myself 1 chai tea latte (staff at venues require that we buy drinks)
At last, we were coming together to complete our first Framework! Despite lots of interest, in the end only three of us made this meeting. This made me realise that although 6-10 members of a cohort is a good idea, not all of us can actually make the meetings - so I am opening up the cohort to more than 10 so that we can have full Second Meetings.
Serrell recommends you write down each judge's initials and their scores, we did just that. To start off the discussion, we picked scores where we found a 3-point different in scoring and talked about what we'd found. Next, we went through each criteria and aspect, and used that to guide our discussion. During the discussion, I saw the exhibition, which I know so well now, through new eyes. That was invaluable. Thanks to my peers, I also learned a lot about how exhibitions are put together and their constraints, as well as looking at it more through the lens of access than I had when I went through.
Be prepared that you'll have a lot to say at the Second Meeting. I was surprised by this. The two hours flew by! At first I used my initial, messy Call-out notes to guide my part in the discussion, and I was all over the place. Once I started referring exclusively to the Rating the Criteria notes (the ones I'd disliked writing earlier) when there was a point I wanted to make, I realised how clear they were by their very succinctness. Also, as they were in order of Criteria, my discussion was in order.
By the end of the evening, we had reached consensus on a couple of things that we all agreed on, which was interesting in itself. A couple of us experienced 'social moderation' where we changed our scores after the discussion on a criterion - in other words, we saw things in a new light, and wanted that change to be recorded as a group!
Following the Framework to the letter during the Second Meeting like we did (well, I facilitated it so we did) meant that there was no time to chat informally whether or not we 'liked' the exhibition overall, until the end. And actually, that is one of the great points of the Framework - it gets you to go beyond 'I liked it' to 'I liked this and this, and here is why'.
Conclusion: now that I have completed one whole cycle of the Framework process, I am more in love than ever. As you can see in the quote above, I now use it for every aspect of my work (and some leisure). It really did deepen my thinking and get me seeing interpretation and learning in a new way. And I really know all the terms, and how to use them. We've already chosen our next exhibition to see.
Why not join the cohort and do the same? Our next Second Meeting is in the first week of February 2020, so you have plenty of time to get in touch and arrange your First Meeting and see the exhibition.
I'd love to hear from you!
--Becca Boyde, Learning Wayfinder, Scottish Learning Group firstname.lastname@example.org