Many grant funders will expect you to have completed some sort of consultation. If you need to consult people there are various ways of going about it. You choose the most appropriate according to what you need from it (robust data or understanding of what people think) and your deadline and budget.
1. What kind of information do I want?
− The main decision to make is over the kind of information you want to end up with. If you’re going to make financial decisions as a result of the consultation, it’s best to aim for robust data quoted in percentages – quantitative data (because it counts quantities of things.) That means doing a survey.
− If you want to know in depth what motivates your customers, how they feel about you or what influences them to think the way they do, it’s best to go for qualitative research, which usually means focus groups, mini groups or semi-structured interviews, but can be done in other ways. This information is very useful, but you can’t generalise from it, so it’s not as robust as percentages.
2. Surveys
− Surveys give you results in the form of statistics: xx% say this. If done well they are therefore robust – it’s safe to base financial decisions on them. However, they can take more time to organise and carry out than qualitative research. They may also be more expensive.
− To give you the findings in the percentages you need a survey must be are based on a questionnaire. This means that they only deal with a limited number of topics, and the boxes to tick for responses are likely to be pre-set. Therefore you may be limiting the answers that your respondents can give. To avoid this disadvantage, it’s best practice to carry out some qualitative research before you design your questionnaire. Then you can make sure you cover the topics that your customers think are important, as well as the ones you do. That adds to the expense though.
− Surveys can be done in different ways but the most common methods are by post (or self-completion), by an interviewer face-to-face, by interviewer on the phone, or online. Each method has its pros and cons.
2.1 BY POST (SELF-COMPLETION)
− Postal surveys are generally cheaper than other methods. That is their main advantage. The other one is that you don’t have an interviewer there, so respondents may be more likely to give you an ‘honest’ response.
− Some postal surveys can get very good response rates, but if people aren’t expecting your questionnaire, you’re likely to get less than one-third of questionnaires returned (i.e. a response rate of less than one-third or 33%). Sometimes response rates can be as low as one in ten (10%). This means that your respondents are a minority group, and can’t really be seen to represent the whole population (i.e. your respondents are not representative of the population).
− There are things you can do to improve response rates, such as targeted publicity, sending out personalised letters, using white envelopes, making the questionnaire look attractive, and sending out reminder letters or second questionnaires, but all these things cost more.
− The second disadvantage, which is a considerable one, is that the people who answer your survey choose to do so. People who respond to questionnaires are likely to have strong views (positive or negative) on the subject, or they just like filling in forms. That makes them likely to be different from the majority of the population, so your respondents are even less representative of everyone.
− The third disadvantage of a postal survey is that there is no-one to help the respondent work out what you mean. They have to use their own judgement if they’re in doubt, and different people may decide that you mean different things. Therefore your questions may not all be answered in the same way, or respondents may not understand them and may not answer them all. Postal or self-completion surveys have lots of mistakes. Therefore your questionnaire needs to be first-rate.
2.2 FACE-TO-FACE
− Interviewing is a skill, and should be done by fully trained interviewers. The common standard for a well-trained interviewer is the IQCS (Interviewer Quality Control Scheme – see www.iqcs.org).
− If your interviewers know what they are doing, they will be able to make sure that all respondents understand what you mean by your questions, and answer in the same way. There will therefore be fewer mistakes – you get better quality data for your money.
− If there’s an interviewer present you’re also likely to get a higher response rate – a good interviewer knows how to encourage people to take part, and how to keep them interested. That makes the data more representative of the whole population (your respondents are less of a minority group), and so it’s more robust.
− The downside is that a poorly trained interviewer may be influencing the way that people respond to questions. That means that your results may be biased.
− The main disadvantage of face-to-face surveys, however, is cost. You need to pay people to travel to and spend time in the place where you want the interviews to happen, and they also need to eat. Face-to-face is higher quality, but more expensive than self-completion.
− Another thing worth noting is that you may need people’s address details to identify those you wish to approach for the survey. Alternatively you can choose a random starting point within an area or ward, but there’s more about that under sampling…
2.3 TELEPHONE
− Telephone interviews are often seen as a compromise. You get better quality data, and higher response rates than a postal survey, but the interviewers are fixed in one spot. Therefore it’s less expensive than face-to-face. Telephone interviewers may still influence responses, but quality controls mean that this is easier to identify and correct, as (at a call centre) supervisors can listen in to calls.
− The problem with telephone surveys is getting hold of telephone numbers. Some people are ex-directory, some only have mobiles (which are more expensive to phone and may change quite often), and a lot are now registering with the Telephone Preference Service (www.tpsonline.org.uk), which means they don’t want to be contacted and you may get fined if you phone them! For all of these reasons if you choose a telephone survey some people won’t have the chance to take part, which makes the data less representative of the whole population and therefore less robust. However, at the moment this isn’t felt to be a bar to carrying out market research by phone.
2.4 ONLINE
− In research terms, the disadvantage of online research is that you’re limiting it to people who have internet access. Numbers with access are steadily increasing, but you’re still excluding a considerable minority of the population. That makes the results less representative of local residents, and therefore less robust. However, if everyone in your population has internet access (for example if you’re surveying online customers), that problem doesn’t arise.
− Security software means increasingly that pop-up surveys won’t ever make it to the screen, so invitations sent out by e-mail (if you have up-to-date e-mail addresses), or letters posted out (if you don’t) are perhaps a better method of recruiting respondents. If people sign up to the survey you can them direct them to a web page, or e-mail a link. Be aware that this best practice route of giving them the chance to ‘opt in’ means that fewer will respond, which makes your respondents less representative, and so introduces bias. Still, it remains an acceptable method for market research.
− Online questionnaires can be made more appealing than paper ones, which increases the response rate. However, designs need to consider the spec of the ‘average’ pc (people won’t respond if the questionnaire won’t open because the file is too large). They’ll wait for about eight seconds before clicking away.
− You don’t have an interviewer with an online survey, which removes bias introduced that way, but also means that your questionnaire needs to be clear. You can set it up to minimise mistakes, so the data you collect online is likely to be very high quality.
− Unless you have a friendly IT department who won’t charge you extortionate rates for their expertise online surveys are not a cheap option. Costs are probably comparable to postal surveys. However, the advantage is in the speed at which huge volumes of questionnaires can be processed. Online surveys are ideal for surveys that are regularly repeated, and deal with large numbers of respondents. The set-up costs are quickly outweighed by the advantage of using the same questionnaire programming each time.
− Online surveys are also useful for respondents like young people who see paper questionnaires as boring, but may be entertained by the pictures and special effects in an online questionnaire. They are also useful for academics or senior professionals who have limited time, but can log on whenever they have the time.
3. Qualitative research – feelings and perceptions
− Qualitative research doesn’t restrict the topics that come up for discussion. The whole process is less structured than a survey, and respondents are able to introduce additional ideas. It’s a balancing act though, to get the right weight between time spent on your interests and on the respondent’s. The job of the facilitator is difficult but important – again it’s a real skill to keep the discussion on track without leading it, and whilst remaining flexible.
− If we want to engage local people (as the Government is trying to encourage authorities to do) we need increasingly to treat residents as partners in decision-making. Therefore they must be able to understand the background and implications of their decisions and recommendations. To achieve this they need to be given relevant background information and helped to understand it. This takes more time (and so more money), but the results may have greater weight. The ultimate goal for engaging the community is sometimes devolving or sharing decision-making power:
Service user representation on decision-making bodies
Neighbourhood appraisal
Large-scale community events
Policy conferences
Community associations
Round tables / User fora
Citizens juries
Deliberative focus groups
Residents’ fora
Tenants’ associations
Focus groups
Referenda
Simulations
Residents’ panels
Neighbourhood fora
Public meetings
Service-specific surveys
Regular (tracking) surveys
Residents’ surveys
Video box
Quality check phone calls
Complaints and suggestions
Mystery shopping
Advice leaflets
Newspapers
Reports
3.1 FOCUS GROUPS
− The basis of qualitative research is usually a group discussion facilitated by a moderator, based on topics set out on a broad topic guide. Focus groups tend to take the form of a gathering of 6-10 individuals for 1-1.5 hours to discuss a given topic, which they may not know much about. That is perfectly acceptable, since you want to collect the views of a variety of people, both ‘expert’ and not.
− The advantage of a focus group is that the moderator can really probe and find out what is behind people’s views and experiences. You can gain a closer understanding of what your customers think. The disadvantage is that you are only talking to about 6-10 people each time.
− When recruiting focus groups you need to allow for dropout, but you also need to aim not to have more than 10 people there. If you have less than six that can also be difficult, as people feel more visible and may be reluctant to speak out.
− As for surveys, the people you include in the consultation should be broadly representative of the population you’re targeting. Therefore for a Borough-wide consultation you need to try to include both men and women, young and old, white and minority ethnic groups, disabled and not. It works better to have people with similar outlooks at a group, so it doesn’t work well to combine old and young people together, and sometimes that applies to combinations of different ethnic groups. If you can afford it, it’s best to do more than one group.
− The role of the moderator is very important, and is hard work. A moderator can either make or break a focus group. They need to be able to make people feel comfortable enough to talk, but they also need to make sure that no one person dominates the group. If there is someone keeping quiet, it is the job of the moderator to give them the confidence to speak out.
− The moderator doesn’t take part in the discussion or voice opinions – they are there to be a sponge, drawing out and soaking up a range of views. They shouldn’t lead the discussion, and should not put words into people’s mouths, however they need to make sure that the topics of interest to the client (the Council) are introduced. If these topics aren’t interesting to the attendees, they need to be able to cope with that situation in a productive way.
− In a discussion that is flowing well the talking happens amongst the group members, and the moderator’s job is to watch body language, and only interject comments or questions here and there for clarification. However, if a group is uncomfortable the moderator needs to take a more active role to get the conversation going. Care needs to be taken to make sure that the discussion doesn’t turn into a question and answer session between moderator and attendees. The moderator should not be seen as the ‘expert’ on the topic in hand, or attendees may hesitate to speak out.
− If the moderator doesn’t have an assistant to take notes, it’s helpful to make a tape recording of the discussion. However, this can only be done with the permission of all those present. Quite often the moderator also has to take notes.
3.2 DELIBERATIVE CONSULTATION
− Deliberative consultation is qualitative research that goes further than getting people to give their ‘top of the head’ views. Instead it makes them go through a stage of ‘deliberation’ or weighing up the pros and cons to come to a decision or recommendation. To do this they need to be provided with, and understand, all relevant information before the discussion and decision-making can take place. This information needs to be presented in an accessible way, to make it easily assimilated by non-experts.
− Deliberative consultation on a large scale takes the form of citizens juries, where a group of people are invited to attend a session lasting several days. During this time they hear evidence from expert witnesses and at the end of the consultation they are required to provide an answer to the question they have been set. For example, citizen’s juries have been set up to deal with the problem of which local hospital should be closed, given resource constraints. They tend to be expensive!
− On a smaller (and cheaper) scale, it is possible to carry out deliberative focus groups to provide solutions to a problem. These are slightly longer than ordinary focus groups, and are attended by an ‘expert’ (for example a Council officer) who makes a presentation giving relevant background information, and who is there throughout to act as an information resource. The Council officer does not take part in the discussion unless invited to contribute by the group, and should not sit with the group. Like the moderator they should not contribute their own views to the discussion, and therefore need to watch their body language even when they are not speaking. The discussion tends to divide into three stages: presentation, questions and answers, and then discussion.
3.3 DEPTH (SEMI-STRUCTURED) INTERVIEWS
− A middle stage between a survey and a focus group is a depth interview, where the interviewer asks questions from a short topic guide, and probes for the reasoning behind the answers. There are no suggested responses, and so it can be useful to tape record the interview, if permission is given. If additional topics are of interest to the interviewee, the interview should be flexible enough to allow these to be investigated without a script.
− Depth interviews can be done face-to-face or (to reduce costs) by phone. They need experienced interviewers to be successful, who know how to probe responses fully.
3.4 E-MAIL SURVEY
− If you have up-to-date e-mail addresses for all your population, it can be cost-effective to carry out qualitative research using e-mail. This involves e-mailing a list of questions to respondents who have agreed to take part. The respondents then type in their responses within the e-mail or attach them as a Word document (making them as detailed or short as they wish) and e-mail them back again. This is a useful and flexible way of consulting senior managers, professionals or academics, who can complete the survey in their own time, and without the constraints of a questionnaire. It can extract large amounts of detailed information.
