When carrying out a review, reviewers will:

 

  • interview leaders and service teams
  • interview consumers and families
  • observe clinical interactions
  • observe the environment and processes
  • collect organisational and patient-facing documents
  • analyse the data collected
  • prepare a draft review report
  • seek feedback from stakeholders
  • finalise the review report.

 

Section 3 of the Guide will take you through these activities.

Gathering data

 

Clinical and environmental observations

 

During the review, you’ll need to make sure you’re observing a service from a health literacy perspective.

 

These videos discuss how this differs from clinical observations and interviews.

 

The Clinical observation guide outlines what reviewers should look for when observing how clinical staff interact with consumers and families.

Video transcript

Angela Wilson (Quality and Project Facilitator, Māori Health Development Group, CCDHB):

When we were observing in the outpatient clinics I found it particularly useful having someone who had no clinical expertise, and in particular someone who was looking at what we were observing through the lens that we were meant to be observing with and that’s the health literacy lens. My journey was learning to have that lens because I look at things as an experienced nurse with a clinical lens and the project is around health literacy, not around clinical expertise.

So the conversations that we had afterwards when we debriefed about what we saw, I learnt about a whole lot of interactions that I knew were there. I recognised them when I heard them but I didn’t see at the time because I was picking up more process detail and hearing the conversations that were being had and then having the story reiterated back when from a clinical perspective it was very expert and it was really well done, but also not noticing that there were things that the mother hadn’t picked up on, that were not properly clarified when it was explained or when there was a question asked, and it was a yes/no type response and that would often be the default.

Dr John Falkiner (Managing Director, Mighty Mouth Dental):

Well, first of all I was a little anxious about having an observer in the clinical environment. I felt that there were great advantages in videoing the interaction, all the participants hopefully would act a lot more naturally, so we set up a video camera in here and that avoided that foreign person in the room at the time. The other advantages – that a video could be used as a training tool later and would have a great deal of value in this context.

The whole staff embraced the involvement and it generated a lot of discussion and some changes in protocols as a result.

Also part of this process – we decided to implement a follow-up communication with 18‑year‑olds as they come out of state-funded dentistry because there’s far too great a drop-off and too many of them are never seen again until they’re driven there by pain.

Andrea Taylor (Nurse Consultant Manager, Health of Older People & Clinical Support, NDHB):

The clinical observation for me was a challenge being a clinician was to actually be an observer and not get engaged into the clinical content of the consultation but to watch, listen and record what was actually happening in the environment around, and also watch the body language of the person in the consultation and the body language of the health care professional too.

[Logos: Ministry of Health and Workbase]

Video transcript

Christine McKay (Portfolio Manager, Oral Health & Child Health, CMDHB):

I think for environmental observations in interviews you really have to understand the patient journey, right from that moment at home where the patient might realise they’re in pain or some discomfort, or they suddenly think that maybe they should be participating in oral health.

We also reviewed what happens when someone phones in for assistance so there’d be telephone calls with a receptionist or a patient care assistant. There were also walk-ins where people actually had concerns, either pain or just discomfort.

Angela Wilson (Quality and Project Facilitator, Māori Health Development Group, CCDHB):

Before we undertook the observations we looked at the patient journey. We walked through the hospital so there’s 2 main entry points into the hospitals.

The letters we found out, or I found out, encouraged people to park in 2 places. The primary one was in our underground carpark. The other place was outside the actual children’s hospital clinic, the Grace Neill Block, but it identified that there were very limited carparks.

So what we did was we then made the journey from the carpark up into the main hospital and then I followed a person who did not know their way around the hospital to the clinic. And that was actually quite an eye-opener for me, having worked in the place for over 30 years. It’s made me realise that you get used to the quick routes, the route to the place and although there may be signage, whether it’s useful or not, you don’t take any notice of it because it’s always there.

We made our way into the actual outpatient clinic and we sat there and observed people waiting, the interactions with the parent and the child, the parents and the clinic staff, the admin staff, and then the meet and greet process to get them into the actual clinic.

Andrea Taylor (Nurse Consultant/Manager, Health of Older People & Clinical Support, NDHB):

The receptionists are really vital people in how people are invited into the environment, how they’re received, how they’re welcomed, how they’re guided to where they need to be. The noise, the clutter.

If it’s a calm environment it sets people up for a really good time with the health care professional they’ve gone to see. If it’s chaotic and busy and a lot going on, the environment as a whole with what’s on the walls is part of that – we visited several areas and there were huge differences.

Some areas were very clean and tidy as in not lots of clutter on the wall, and others have lots of old information on the walls, and lots and lots and lots, and you didn’t know where to look, and it actually was just a bit overwhelming.

[Logos: Ministry of Health and Workbase]

Interviewing people

 

After the first few interviews it can be useful for the reviewers to get together and discuss what they are seeing, to make sure the questions they’re using are generating answers relevant to health literacy.

Video transcript

m

Cheryl Goodyer (Manager, Capability, Māori Health Development Group, CCDHB):

I guess after having an understanding about health literacy and a health literate organisation, we had 2 slightly different approaches, 1 for the staff and 1 for patients and whānau.

While we could talk to staff about the health literacy directly, with patients and whānau we were mindful that it wasn’t to be a patient deficit model so ‘do you understand?’ didn’t quite – wasn’t what we wanted to ask. So putting it into context, getting an understanding of what they knew and then I guess the teach back, getting them to give us the information back was far more appropriate.

Angela Wilson (Quality and Project Facilitator, Māori Health Development Group, CCDHB):

The questions we found useful were not asking them what they understood about what had happened but more indirect questions such as what would you tell your partner as to what happened and the outcome of this consultation, or what would you tell your employer. And that provided a conversation where the family member was able to put into their own words their understanding of the conversations that had been said and what was going to be the next steps as a consequence of that consultation.

Dr Siniva Sinclair (Public Health Physician, CMDHB):

So when interviewing service providers it’s helpful to know what role each of them has in terms of helping the patient along their journey, how they help the patient understand what has happened to them so far, and what’s going to be happening in that encounter, and then what the next steps are after that. And there are some questions in the guide that can be useful in helping explore that with people.

Andrea Taylor (Nurse Consultant/Manager, Health of Older People & Clinical Support, NDHB):

Listen to the story. Ask them to explain their journey, explain how it started and what happened, rather than direct questions, because you get a richness from a story that you will not get from direct answers.

You’ll often get a yes or a no from a question. Although it could be an open-ended question, they will say a yes/no answer. Whereas if you allow them to tell you a story it’s so rich, it’s so valuable and it has an awful lot of personal meaning behind it which is really helpful.

You can hear where the good learning has happened, where the understanding from a health literacy perspective has been good. Has it come from a health care professional, or has it come from a friend, or has it come from a newspaper or media? It really tells you a multitude of different things, listening to the story.

[Logos: Ministry of Health and Workbase]

Using existing data and documents

 

Where possible, reviewers can make use of existing data to uncover health literacy issues or successful initiatives. For example, patient experience surveys can be a great source of information for the review.

Video transcript

Susan Reid (Consulting Manager, Workbase):

One of the things that reviewers have to do as a part of health literacy review is to look at strategic documents an organisation has and to see how health literacy is positioned within those documents.

One of them was the Northland Health Services Plan 2012–2017 and we looked through to see if health literacy was mentioned, and it was. And we looked then to see how it’s positioned, and when we looked at the Northland plan we could see that it was based on a patient view. It was about impressing the patients’ health literacy or the consumers’ health literacy.

So you could say it was implied that health professionals would be doing some of this but the explicit statement was about patients so we knew at the beginning of the health literacy review that health literacy was seen as being a patient concern rather than necessarily being seen as something shared between the patient, the health professionals working in the system and the DHB and its systems and processes as well.

The other set of documents that reviewers will be looking at will be patient documents, the sort of documents that health professionals would use with patients and families.

So the first thing we’d say is start and see if they’re written in plain English or plain language, and the Ministry of Health has produced a guide that has a really good checklist at the back. And then you’d work through a series of other questions – at the back of the guide there’s an appendix that will help work you through with a templated series of questions that will help you work through that, and then at the end you have a summary of what’s good and at the end what things could be improved.

Angela Wilson (Quality and Project Facilitator, Māori Health Development Group, CCDHB):

We got pulled out of our system the letters that were sent to the parents advising them that their GP had sent a letter or that they had an appointment, a first appointment or a follow-up appointment.

One of the things that I picked up when we were interviewing a GP out in primary care was the fact that the conversations they had with their patients were that they were going to see a specialist in the hospital.

Susan Reid:

Some people might use readability tools when they assess patient education material so they might use the SMOG index or they might use Fleisch Kincaid or something like that. Now readability is one aspect of documents. It’s not – it shouldn’t be seen as the be all and end all – it’s just one aspect you look at.

[Logos: Ministry of Health and Workbase]

Video transcript

Christine McKay (Portfolio Manager, Oral Health & Child Health, CMDHB):

In oral health services we do have a lot of data and information. We have the New Zealand Oral Health Survey for 2009 that gives us a lot of information about what people do and how often, whether it’s brushing their teeth or the level of dental disease in the community But we also have a lot of consumer insights into the why.

Angela Wilson (Quality and Project Facilitator, Māori Health Development Group, CCDHB):

When we went back and looked at the attendance by Māori, it was identified that the child health clinic had services within their service that would have Māori DNAs up to 33% compared to the Capital Coast Health yearly average which was around 5% or 6%. And across the organisation did-not-attend by Māori patients tend to average somewhere between 12% and 15%. So we were very interested in seeing how we could make a difference to our most vulnerable service users.

Child Health were very keen to be involved and were also really interested because it was a mechanism by which to support clinician best practice.

Cheryl Goodyer (Manager, Capability, Māori Health Development Group, CCDHB):

Part of the advantage of having a project that was aligned with another project that was in the organisation like the DNAs is that a lot of the information was already available and while anecdotally we knew that it was all about communication, having the measures about patients that didn’t turn up, and some of the issues that had arisen from the preliminary work, enabled us to look at that data and assist with moving this project forward.

Andrea Taylor (Nurse Consultant/Manager, Health of Older People & Clinical Support, NDHB):

Existing data and information from the practice was really relevant and helps the planning of ‘well what are we going to do with health literacy?’ so the use of previously done surveys, consumer feedback, even the suggestion box that people give – it’s kind of a heads up on ‘this isn’t quite what it could be and we would like to see an improvement in this’.

If I ring up I don’t want to have to go through 4 or 5 pressing buttons. If I walk in the door it would be nice to be said hello to and not have somebody look down at the desk or be on the phone and not acknowledge that I’m there. Little cues from the users in the environment are key.

In addition to that, plans that the practice has made, so strategic plans, development plans that they’re looking at, where they want to go in the future.

[Logos: Ministry of Health and Workbase]

Video transcript

Dr Siniva Sinclair (Public Health Physician, CMDHB):

It was really helpful when we analysed the data to think of it under the 6 dimensions for a health literate organisation.

It’s suggested in the guide that you do some data analysis as you go through and we found that helpful as well because then it meant that we were able to test our findings with the service providers and check whether there was anything else that we needed to explore in more detail for instance.

Cheryl Goodyer (Manager, Capability, Māori Health Development Group, CCDHB):

The advantage of having the guide was that we were able to look at the 6 dimensions of the health literacy and that enabled us to see what was relevant in the information that we’d found. Whilst most of it was about communication and engagement, there were findings on the periphery that actually weren’t relevant to the health literacy so it enabled us to stay focused on the project.

[Logos: Ministry of Health and Workbase]