Episode 3: What Is a Reading List For?

The Teach Learn Collaborate Repeat podcast is a teaching and learning podcast hosted by Talis. We’ll be talking teaching and tech with experts from across higher education.

In this episode, Hazel Rothera, Academic Liaison Development Team Leader and Dan Croft, Scholarly Communications and Research Team Leader, Oxford Brookes University, join Matt East, Education Lead at Talis to share their ‘What is a reading list for?’ project. This project explores how reading lists can best support academics and students. We learn about the framework and guidance they’ve put in place, and hear successful case studies that have come out of the project.

Want to learn more, share your thoughts or be a guest on the podcast? Email tlcrpodcast@talis.com or tweet us at @talis using #TLCRpodcast.

Listen to this episode now:

Watch this episode now:

Here’s the episode transcript:

(generated by Otter)

Matt: Hi, and welcome to the Teach Learn Collaborate Repeat Podcast. I’m Matt East, Education Lead at Talis and I’m really delighted to be welcomed by Dan and Hazel from Oxford Brookes today to talk about ‘What is a reading list for?’. Dan, Hazel, can you just introduce yourself to kick off, please?

Dan: Sure. So my name is Dan Croft. I’m based in the library at Oxford Brookes University, and I work in the scholarly communications team.

Hazel: Hi, I’m Hazel Rothera, and like Dan, I work in the library at Oxford Brookes, and I lead one of the academic liaison teams looking after Humanities and Social Sciences. I’m also the operational lead for Talis Aspire in the library.

Matt: Awesome. really delighted to have you both with us today. So a couple of years back, you started a project looking into actually redefining what is a reading list for, and I’m really keen to hear from you about how these projects materialised. You know, what, what initial problems were you initially trying to solve through for your work that you undertook? Dan this is a question for you.

Dan: Sure. Well, I guess it starts with kind of my personal experience as a subject librarian at the time, and I’m sure Hazel will share this experience too, that the reading lists that we were working quite a lot on didn’t seem massively relevant to what was actually happening in terms of the teaching and learning in the modules. So from that starting point, we were always talking about how can we help improve engagement with the reading list those types of things. And quite a few of us were doing a course, a postgraduate certificate course with the staff development team, Oxford Brookes, who at the time was OCS LD, the Oxford Centre for staff and learning development, I think it was. And through that, we were meeting these ideas about pedagogy. And one of the main ones was constructive alignment. So at least that caught my attention a great deal, because that really addressed this question of what is a module for how well how does learning activities contribute to this. And it really helped me see the reading list challenges we had through this kind of pedagogic lens enabled, and then it kind of enabled me to see it and as a potential as a way of readdressing reading lists from a very different angle than we had been doing before, which was a lot about just how can we improve communication, about workflows, about technology. And this was a different kind of lens look at it, and it seemed very promising. So that was the start of the project.

Matt: So What process did you go through to undertake the research to create output, which we’re talking about today?

Dan: Well, actually, we were very fortunate Oxford Brookes has some teaching fellowships, and we got some funding through that, and basically just undertook it in a normal procedure would do for research projects. So I started with a literature review, looking at what else had been written. And that was quite time-consuming. I wanted to be quite thorough about it and make sure it wasn’t reinventing the wheel if you like. And that was actually really helpful. There’s a number of very good articles on reading lists in the literature. And it certainly kind of reinforced the point that we were, our experience wasn’t unusual that a lot of people have been trying to engage with this for for decades, really. But it didn’t seem like there’d been much of a breakthrough in that time, because people were still reporting the same difficulties year after year. And each of the articles as they were coming out. I also saw that there were some articles not just reporting this problem, but also trying to improve it some kind of ideas how to kind of address the situation. But there always seems to be like what we were attempting kind of procedural changes or workflow changes. And we must think different, so. So after literature review, or sorry, during the literature review, I was also talking to a lot of academics around the institution, and in other universities, seeing what kind of reading list practice they were doing. Because there was some examples of excellent reading list practice, the majority was was kind of where we’re having this, this difficult relationship, I’m trying to find out what was going on and how successful it was. But some people were reporting exemplary practice of embedding their reading list into their teaching practice. So we were talking to them getting their ideas. One particular person was great was lecturer, David Aldridge, who worked in the School of Education at the time, I think he’s now Edgehill University, but he was doing some fantastic stuff of this reading list. And he was obviously an expert in pedagogy as well. So in the conversations we had based on his experience and also looking at the literature, from, like, the library literature, if you like about reading list, bringing us together, it really was the foundation of the project. We moved on to the methodology of our approach, which was to find some modules around the institution where the module leaders were happy to kind of experiment with these ideas, and to kind of work with us to redesign their reading lists through this pedagogic lens. And then what we were doing was kind of trying to improve those reading lists and then seeing later if they were happy with that redesign, if you like, it would have been better if we could have actually looked at the student outcomes, but it wasn’t really in the scope of the project. The amount of funding the timescale we’re working in. So instead, we were looking at more was a module leader, did they think it was a good change? Did they think the reading list would be brought closer into the module through the changes we’ve made?

Matt: So you, you touched on the approach for the kind of investigation and fundamentally the output, but I’m interested to hear a bit more about the findings. So from this lit review, you’ve found some themes from the work of academics were doing, you found some things, what were those kind of standout findings from this project, that led to what you eventually added to the output?

Dan: Sure, well, certainly one of the main things of literature review, and this very much tallied with our experiences that the academics had a very different idea about what reading lists were for compared to what the students thought it was for. And this was quite strongly reported in literature, when people were surveying the academics and is, you know, and the academics are often saying, it’s a starting point, it’s a guide, it’s, you know, it’s they should go on and be reading in-depth, and then exploring their own. And the students saw it very much more as a, like a prescriptive list of this is everything I need to read, and then working on that basis. So that was definitely one of the things that came through. One of the things which wasn’t in the literature so much, but certainly came out of our projects was about student engagement with the reading list and how significant that was, because with the modules, we worked with, a lot of the module leaders were very happy with the redesign. But were still reporting the students actually weren’t doing a lot of the reading as far as they could tell. And so some of the kind of that student engagement was quite disappointing in a sense. And what we found where it was successful is because the assessment of the module was either assessing directly engagement with the reading list, or the module lead had made it very explicit how the reading list was linked to the assessment of the module.

Matt: which comes back to the core principles around constructive alignment. So you know, that’s really, I’m really interested in that, in that point, you just made that about the interpretation of what the list is for being completely sort of different between staff and students. And I’m sure we’ll come on to talk about that in a bit more detail later on. But I’m sure you’re not alone in those findings. I imagine most institutions are battling with, with similar, you know, similar issues there. So you created this resource, and the thing that really jumped out to me, within the resource ‘What is a reading list for?’ was almost the kind of staging that you built in. So like, step one, step two, step three, and I love the different examples that you’re given that were backed up by case studies, I thought that was just really accessible and really, really, like progressive, I guess, really, in many ways for colleagues who you know, want to just hit a baseline or actually want to try and do some new and exciting things and get some real value. So you know, kudos to you on the production that you created there. I guess what I’m what I’d like to find out a bit more about is what you kind of want it to achieve from that resource, like, what was the goal? And what kind of change did you want to bring into the course the institution?

Dan: Sure, well, the project was producing those two different outputs, the article and the guide, because we didn’t expect the module leaders to engage with the article, we didn’t expect them to go and read the reading list literature. So we want to kind of a hands-on guide. As I said, before, we didn’t want it to be a procedural thing or a workflow thing, but we wanted it to be kind of informed by pedagogy, but to wear that lightly if you like. And so we wanted a kind of how to improve your reading list in these four easy stages, which you said, and they do go up in these kinds of levels of difficulty. And we do call them levels if you like. And it’s just so that people if a module leader wanted to try it out, they don’t have to kind of redesign everything, they can start quite simply. And if they like that, they can move on to the next level and the next level. And it was, yeah, it’s trying to keep it simple, but at the same time underpinned by this research that we had, and that we’ve been reading about in the literature review. So the first level we had was with this one called organize. And this is just a very, very basic level where the typical reading list is just one list arranged in alphabetical order of author’s surname. And that has no relation to how those resources on that list should be used in the module, how they relate to the other parts of the module, how they relate to the assessment with a single list. And that leaves a huge gap for the students have to interpret for themselves, how these resources are arranged in alphabetical order relate to the different parts of the module, which weeks relate to which topics which assessments. So this is the very basic level of level one of organize, which is basically put your reading list into sections, which clearly relate to the rest of the module. So it’s really entry level point. And then it builds up from there. Hazel I don’t know what your experience has been of kind of implementing this with academics at Oxford Brookes.

Hazel: Yeah, sure. I can certainly talk about that a bit. One of the things that I really like about the guide is the way that itself, it mirrors the sort of process that we’re encouraging both academics and then students to do in that it moves from a fairly as Dan says fairly simple, relatively shallow level of thinking you could do and moves you from surface learning into deeper and deeper engagement. The guide itself does that. And so one of the things that I love about it and any new member of academic staff or anybody who wants to refresh their thinking about their reading lists, before I give them any of the procedural guides, we also have, we already have those how to do this that and the other, the first thing I always do is give them Dan’s guidance. And before you do anything, have a read of this, have a think about what is your reading list for because one of the things the other things I really like about guide, this is not prescriptive, it doesn’t say your reading list is for this or this or your reading list this should be this or this. It’s asking the academic to think about what’s the purpose of your reading list, in line with the teaching of your module? And crucially, how do your students know that? One of the things that the guide walks the academic through is how are you communicating to your students how the reading list works? And I mean, this sort of links in quite nicely to the sort of question that you were, were asking earlier about the key finding and the gap between how academics were thinking about their reading lists and how students thought about them. And I had not long after the guide was launched, in fact while it was still in draft, I had to say, can we use this, even though it’s only a draft with someone because I had a member of the School of Education got in touch with me this is several years ago now, she was tearing her hair out, because she just had a tutorial with a second-year student who had failed one of her modules. And she said to the student that one of the reasons you failed was that the key theorists were missing from your assignment, you weren’t addressing the work of the people I expected you to address, they were all on the reading list. And the student literally said to her, “was I supposed to read the stuff on the reading list?” in a kind of baffled way. And so at that point, she went ‘ah’ communication failure. And so she came to me said, What do I do? And I said, Well, as it happens, my brilliant colleague Dan has just produced this really handy guide, why don’t you go away and think about that, she completely redesigned that particular reading list along those lines. And then that sort of practice spread a bit for the School of Education. And we began to use some of the more in-depth sort of things that we could do one of the great advantages of doing the first level thing of restructuring your list so that everything matches your teaching, is that you can then embed the relevant sections of your reading list week by week into the VLE, which is a really strong signal to students “This relates to how you learn on the module”. Most of our students, particularly since the pandemic, where they start everything has been on the VLE, I was able to do a really nice case study last year with another colleague in the School of Education. And we presented at our internal staff sort of conference last year, where she completely redesigned her reading list from a kind of ‘bleugh here’s the reading list’ to this section section section, annotate, annotate, annotate, explain, explain, explain, this is why you’re doing this, embed it all into the VLE. And the views, the clicks from Google Analytics on her list went up from the previous year, the whole list was viewed, I think, just shy of 600 times; with it redesigned with individual sections, she got over 14,000 section views. And more importantly, because obviously, you know, clicks just numbers, she could then really see she gave me some really nice qualitative evidence from the students taking that module that year, their engagement with reading had improved massively, she could see that in their work, she was giving a big increase in 60 to 80% grades. And the feedback that she and the other markers were were giving the positive feedback was around their wider reading and engagement with the reading. So there was both quantitative and qualitative evidence that that whole approach really was working, we got quite a nice case study.

Matt: That’s a fabulous case study actually is a really, really powerful example of the difference that some subtle changes can actually make on independent learning or even collaborative learning, I guess. So that example you gave is an absolutely brilliant case study. I’m interested to hear what impact you feel this project has had on the kind of wider quality of reading lists across Oxford Brookes. Have you observed, you know, a step-change in what’s the norm? Really across your academic departments?

Hazel: I think we’ve certainly seen we haven’t done a sort of universal survey of it. Certainly as someone who does the kind of week by week, Google Analytics kind of stats to feedback to academics. There’s been a huge increase in the tendency for people to embed list sections into the VLE, the stats are definitely showing that there are all sorts of highest performing or the majority of the highest performing kind of things that are clicked on from week to week are embedded sections of lists into the VLE. And that tells me that people are structuring their lists because you can’t embed into the VLE unless you’ve structured the list appropriately. So we’ve certainly got that quantitative evidence that a lot of people are, are doing it. I haven’t done a sort of qualitative survey across the university as a whole. But it certainly has enabled us when we had to do the sort of mid pandemic thing to say, okay, all essential stuff has to be online that people have got to be able to access them with their reading online. It gave us a really helpful sort of pedagogic backup to underpin that to say not only is this necessary, because pandemic, this is also good pedagogic practice for you to think about, your reading list as part of your teaching. Obviously, you can never sort of unpick one factor from another. But we certainly saw between the 2019/20 and 2020/21 academic years, we saw an increase in reading list use about 50%, and an increase in academic engagement with reading lists, in terms of how often they updated lists, of about 75%. Now, clearly, the lockdown was driving some of that, but then the having the tools to say to people, there are good pedagogical reasons for doing this as well as we have to do it because pandemic really helped. And I think one of the sorts of suggestions that underpins that is that having sort of had that really steep climb, it hasn’t then fallen off this year, as students have come back to campus, the academic engagement has plateaued, it’s still up there, it’s still up at that high point, I was really concerned that it was going to be a pandemic spike and that once students could get back onto campus and could access physical material again, some of the academics focus on reading lists would go out the window, but actually that didn’t happen.

Matt: The fact you’ve got sustained change, there is absolutely brilliant. I’m interested to… have you observed a change in academic behavior around when reading lists are updated as well? So I think it’s safe to say across higher education, there is obviously quite a significant spike at the kind of start term in September and January, in updating lists, but have you observed anything, any change in behavior, updating more regularly, weekly, for example, or anything like that?

Hazel: I think it really does obviously vary across, you know, sort of both individuals and the discipline. And one of the things that we always stress to the academic staff is it’s not for us as the library to dictate how do you do it how when you want to do it, lots of people do just do a once a year or before the start of the semester update. But certainly in the faculty, I’m most closely involved with humanities and social sciences, because I see all the assigning of reading lists to my individual team members to review, we’ve got a lot of academics now who do a lot of small updates to a reading list, as the semester goes on. It’s a reading the reviews are kind of flying in. And it’s you know, sort of first half of the semester, my team were complaining, it felt like whack a mole, you know, they dealt with one and then another three popped up. But very often it was just, you know, one or two changes here’s next week stuff, it was lots and lots of small updates. And I think that does sort of speak to this, this idea of the reading list as an ongoing kind of changing piece of the pedagogy that every time as you’re changing the teaching of the module week by week, it is you are keeping the list kind of going along with that. And again, you know, embedding those relevant bits into the VLE so that students think “right, that’s this week’s reading”.

Matt: Well it should be an evolving resource. Should definitely never stagnate. So, you know, the fact that we’re hearing people are doing more weekly updates. That’s great.

Dan: I agree. Because what we were finding a lot of time is that the reading list before were very much almost administrative chore that they were they had to do as part of the validation of a module, whereas to hear that it’s becoming more of a, a live list, which is responding to the flow of each kind of semester that that module is run is fantastic. It makes it and it’s evidence that it is much more embedded in that in that module, to hear that it’s even in that run of that module.

Matt: Absolutely.

Hazel: And one of the nice things about having also, in the last couple of years started to make much, much more regular use of our Google Analytics stats is that we can then feedback the evidence to the academic staff to say, here’s when your students are accessing your lists. And for example, one of our most used lists last academic year was a particular anthropology one and we were able to show the academics week by week across the semester, this is what happened. And one of the things that they were really pleased to see was the engagement with the reading list didn’t substantially fall off as the semester went on. I mean, it is a fairly typical pattern, there’s a big spike early in the semester, and then you know that there’ll be a sort of gradual tailing off within another spike, usually just before the assessment. But actually, it was much steadier than I think academic staff had expected. And they were really pleased to see again, because week by week by week, here’s this week’s readings, and then here’s a load of extra stuff, you might want to back you up for the assessments and of course, big spike in that shortly before the assessment was due but there was that sustained, sort of interest and sustained engagement. And they were really pleased to see that we were really pleased to have the data to show them that yes, we had to say to them, okay, they’ve clicked, we can’t promise you they’ve understood what they’ve read, but we can at least promise you they’ve clicked on it.

Matt: On the topic of data. We’ve recently conducted a survey to students using Talis Aspire from across all of our customers. We asked students, you know, what, what do they really value from reading lists? And what improvements could we make or could be made on reading lists, and actually, it chimes very much with exactly what you’ve just described, their lack of kind of section, organization is problematic. Lack of context is the most problematic thing. So you know, by providing those two pieces of information and that scaffold, and embedding it, you know, is a really it’s obviously, you know, hugely, hugely valuable. And I think all of the data you’ve talked about, you know, really backs it up. So, this project occurred a couple of years back from memory. Has there been anything additional, you’ve done sort of as a continuum of this project? To kind of extend the work that you did initially?

Dan: I’ve moved areas, I’m not working on reading lists so much anymore, but certainly hearing Hazel talk about before, I wondered whether there is scope for a second edition but changing the title from what’s a reading list for to ‘what is your reading list for?’ Because I think that’s actually would be an even better title, really hammer home the point that Hazel was making about it. It depends on the module, really. But yes, it’s something I think I’ve been watching Hazel from within the library, seeing how she’s been building incrementally, you know, trying to improve engagement, from academics with the reading lists. And that’s kind of, obviously, our end goal is student engagement with the reading lists but we have to go through academic engagement with reading lists first to kind of enable that to happen.

Hazel: Yeah, I, because I’ve got the sort of the dual outputs of Dan’s project that for anybody, you know, academics being what they are, who wants to see the evidence and wants to see the literature, I can say, well, here are the findings. It’s not just what we found at Oxford Brookes there are these broader findings across the sector. But then I’ve also got that really nice, practical, unthreatening positive, step by step, here are things you can do guide that it’s been really helpful to be able to embed that that whole approach into how we talk about reading lists. And as Dan said, getting people excited about them as being part of their teaching, rather than looking at here’s another piece of admin that I have to do, because yeah, academic admin is not exciting, and people are incredibly overburdened by it. Whereas what they actually want to do is teach their modules to teach a thing they’re interested in and getting them to literally reimagine their reading lists and the case study that my colleague Jane and I presented to the conference last year where we called it reimagining reading lists because the whole theme of the conference was around reimagining different aspects of teaching, and learning, sort of in the wake of the pandemic, getting people to reimagine their reading lists as an active part of their teaching rather than a passive piece of annoying admin has been made so much easier by having that guide so that when people buy into the idea, but then say, well what do I do? you say well start with this, there’s some really simple stuff you can do at the kind of top surface level, and then it’s up to you how much deeper you want to go. But if you only do the first couple of levels, if you just restructure your list and annotate it clearly, that in itself will make a massive difference to students and will enable you to do things like then embedding your list sections into the VLE, which in itself will automatically engage your students if you then want to redesign, you know, the whole way that you teach your module so that it hangs off the reading list, great and in fact, we’ve got one lecturer within the School of Education his focus is very much kind of children’s literature and children’s fiction and he teaches a master’s module on children’s literature for which his reading list actually starts with a link to a 10-minute video of him explaining how the reading list underpins the whole of the teaching of the module and underpins the assessment and how you succeed. Last year, he was voted in the student teaching awards, our most inspirational lecturer of the year. And the week he got that award, that list was viewed over 500 times. So yeah, that’s someone who’s so bought into the whole idea, that it, you know, it completely underpins how and how he teaches.

Matt: Everything, everything you’ve just said, the thing that really jumps out to me is, this is helping academics really want to use reading lists rather than have to use reading lists, which is, which is just absolutely brilliant. And I think again, just to call out the fact that this isn’t a policy, it’s supportive guidance on how you can make incremental changes to do to help students learn, which is just brilliant. And, you know, many across the sector, I’m sure will either be thinking about doing projects like this, or will be wanting to make a change to the guidance instruction, they provide for their academic colleagues. So for anyone who is seeking to do this at their own institution, if you got any advice on, you know, based on what you did, and what you learn, and how you approached the research and the methodologies, etc. What, what advice would you give to those who want to do this themselves?

Dan: Sure, the guide is available to everyone, it’s on our institutional repository. And I think we’re gonna put together a reading list from this podcast so that people can access it there. And of course, the article is published in a journal so people can find it and read that too. But in terms of practical advice, I think it’s what we’ve been talking about, it’s asking the module leaders to see it as not an admin task, but to think about it as an exciting possibility of a way of delivering their teaching. And I think we’d agree most academics are excited by their teaching and excited by engaging with students and are excited when students are engaged with their teaching. So it’s just trying to get them to perceive that the reading list is part of that. It’s not part of the central administrative administration of the module, it’s part of the delivery of teaching, and it can be students, you know, will be engaged and will be excited by a well-made reading list.

Hazel: And I think that links back to what we were saying a little while ago about what really kind of hooks academics is ‘what is your reading list for’? So I think the other thing I would say is implement it in a way that is going to work for your context. Think about your academics, the disciplines, they’re teaching what is exciting for them, we find that that Dan’s guide is used vastly differently across different faculties and in different courses. And there’s no single answer to what should a reading list be? You know, I’ve got lists in humanities and social sciences that are hundreds of items long, we’ve got a fantastic list over in biology, which consists of one core textbook, and then a half a dozen different sections on different aspects of genetics, and biology, all of which are pointing to videos. It doesn’t, we decided at Brookes to stick with calling it reading lists because everybody knew what it was. We tried experimenting with things like calling it an online resource list, and it didn’t really work. So we went back to calling it reading lists. But one of the other things that we really try and convey to academics is that that’s reading in its broadest possible sense, where anything could be a text, because you can put anything on your reading list, you can put video, you can put audio, it can be websites, it can be stuff for students to actively engage with to watch to listen with, it doesn’t have to be reading sort of bog standard text. So again, whatever is going to work in different people’s contexts with different people’s disciplines of reading, this can be anything, and it just has to fit with what you’re teaching and how that’s gonna work.

Matt: That’s a really important point, I think, to end on really in the context is king with all of this kind of stuff. As you say, it’s not just about the different subjects, but actually the different individual pedagogies, as well, they’ll define how this stuff is, how these areas are being structured, and how students are expected to engage in how students can maybe even activities or change the list. So just to finish up, as we’ve talked about this a lot, I guess there’s a final question, which you’ve got to ask, which is, what is a reading list for?

Dan: Well, I think there are probably lots of different answers to this. And I guess the question I’d throwback is, what’s the module for? What are you trying to achieve through your module? Because the reading list is as much a part of that as anything else. And the constructive alignment answer to that would be to deliver the learning objectives of the module. So for teachers when to do this, it would be if they want to redesign their reading list, and they’re asking, Well, what do I do? How do I do it? Well, we would ask them to think about their learning objectives and how the reading list can deliver those.

Hazel: Yeah, I guess all I can add to that is that certainly in our context, a reading list is a three-way piece of communication between the academic staff and students and the library. And we haven’t really talked about the library aspect of it because we’ve been focused on teaching and learning. But obviously, at the library, we can only support students in accessing resources in a timely fashion if we know what they’re expected to get access to, when and to what degree? So I think it’s, you know, it’s all of those things. It’s a part of the modular teaching. And it’s a crucial piece of communication between students, the library and the academic staff about how reading in its broadest sense helps to underpin all of that.

END