By Ryan Buchanan, Charlotte Cook, Julie Parkes, & Salim I Khakoo
The World Health Organization has recently set a target for the global elimination of Hepatitis C. However, to monitor progress it is necessary to have accurate methods to track the changing prevalence of Hepatitis C in populations that are most affected by the virus. People who inject drugs are a marginalized and often hidden population with a high prevalence of Hepatitis C. As such, tracking Hepatitis C infections in these populations can be difficult. One method to do just this Respondent Driven Sampling or RDS. However, prevalence estimates made using RDS make several assumptions and it is difficult to test whether these assumptions have been met.
However, our recently published article in the International Journal of Social Research Methodology describes a novel way to do just this. This blog shares some of the challenges faced in doing this work and how, by using novel social network data collection techniques, we were able to test some of the assumptions of RDS in an isolated population who inject drugs on the Isle of Wight in the United Kingdom. However, before delving into how we did this, a brief introduction to the RDS method is necessary.
RDS requires that researchers start with a carefully selected sample within a target population. These individuals (called seeds) are asked to refer two or three friends or acquaintances to researchers who are also eligible to take part. These new participants are asked to do the same and recruitment continues in this way through ‘waves’ until the desired sample size is achieved. Then, using appropriate software, the data collected during the survey about each persons’ social network allows for the estimation population prevalence (e.g., how common Hepatitis C is in the population of people who inject drugs).
Using RDS to estimate the prevalence of Hepatitis C among the population of the Isle of Wight, we hypothesized that the treatment program was closer to achieving the elimination of the virus than the available data suggested.
However, concerns remained about the potential flaws of RDS and we were interested in how one could develop methods to assess these flaws. Here our study on the Isle of Wight presented a unique opportunity. The small island population made it possible to map the social networks connecting people who inject drugs through which the sampling process passes. With this network ‘map’ it would then be possible to test whether some of the assumptions underlying the method had been met.
To achieve a mixed methods social network study was run alongside the main survey. Interviews were conducted with people who inject drugs on the Island as well as the service providers who worked with them. These interviews explored how they were all interconnected. Survey participants were also asked about their social networks which then aided in the construct of a representation network through which the ‘waves’ of the sampling process passed.
Unsurprisingly, many survey participants were unenthusiastic about identifying friends and acquaintances who also inject drugs. Instead, unique codes for each individual described were utilized. These comprised of their initials, age, hair colour, gender and village or town where they lived. Participants were asked about each individual they described e.g., how frequently do they inject or if they use needle exchange services? In this way a picture of the social network of people who inject drugs on the Isle of Wight was gradually built up which provided insights into this population even though some of the target population hadn’t come forward to directly participate in the survey.
With this ‘map’ in-hand and the personal information collected we collected it was possible to test some of the assumptions of RDS like (1) if the target population for the survey are all socially interconnected or (2) if members of the population are equally likely to participate in the survey.
The researchers would like to thank the individuals who came forward and took part in this study, the community pharmacists who provided research venues in towns and villages across the Island, and the study funders (NIHR CLAHRC Wessex).
By Christina Silver, Sarah L. Bulloch, & Michelle Salmona
Our contribution discusses synchronous online teaching of digital tools for qualitative and mixed-methods analysis, known as Computer Assisted Qualitative Data AnalysiS (CAQDAS) packages, during the COVID-19 pandemic. Teachers must take responsibility for, and be sensitive to, the current additional challenges and pressures upon learners and attend to them effectively. Learners are never homogenous but, in these contexts, their heterogeneity and personal situations bring our responsibilities as teachers into sharper focus.
Challenges of teaching CAQDAS-packages
Teaching CAQDAS-packages is challenging as research methods and technology are taught together, and researchers often need support overcoming hurdles associated with integrating technology into research practice. Although it can support critical reflection on methods-driven research, novice researchers have trouble connecting method and software (Salmona & Kaczynski, 2016; Schmieder, 2020).
Traditionally CAQDAS is taught in-person but even before COVID-19, there was a gradual move to online courses, which can be cost-effective and reach wider groups. However, teaching CAQDAS online has its own challenges, including possible technical problems, catering to different learning styles, and interactional issues (Kalpokaite & Radivojevic, 2020). Learning CAQDAS-packages online also heightens challenges in overcoming barriers to successful technological adoption due to the lack of support normally present in-person (Salmona & Kaczynski, 2016). Teaching CAQDAS-packages online during COVID-19 poses additional challenges related to learner availability, real-life bleeding into the classroom, and resultant interactional issues.
Learner availability in the COVID-context
Pre-COVID-19, both in-person and online, certain assumptions were often made concerning the ‘availability’ of learners:
They would be present for the duration, unless specific exceptions were brokered; e.g. warning they may have to take a call or leave early.
Only registered learners would be present – not family-members, carers, or dependents as well.
Learners would be in a state of mental and physical health suited to learning.
Teachers could generally assume to be engaging not with whole individuals, but with focused
“learners”: the mentally present and mentally well, physically present and physically well, the not-distracted, the captive from start to finish, solo individuals.
Real-life bleeding into the classroom
During COVID-19 these assumptions no longer hold true. We cannot expect learners to focus for the whole allotted time because they cannot necessarily physically or emotionally remove themselves from their home-life contexts. New distractions and stresses include: interruptions from household members, capacity to concentrate for lengthy periods of screen-time, and mental-health issues associated with being more isolated. However, because in-person interactions have largely vanished, learners are keen to participate in online sessions, despite the distractions and stresses. Online sessions also provide learning opportunities for those previously unable to access in-person events.
As we teach and learn from our homes, real-lives bleed into the classroom. Sharing our images via video-stream allows others into our lives, which is potentially risky. We’ve found more learners choose not to share their video-stream than do, especially in larger groups and when they don’t know each other.
What we miss by not ‘seeing’
Those used to teaching in-person can find this tricky, as the non-verbal cues used to judge learners’ progress are absent. CAQDAS teachers can no longer ‘walk-the-room’ observing learners’ computer-screens to identify those needing additional support. Screen-sharing can be a solution; but is more time-consuming and ethically difficult when working with confidential data, and impossible if using two devices (one to access the meeting, the other to operate the CAQDAS-package). We miss a lot by not seeing in these ways.
One risk is that those who can actively participate inadvertently soak-up attention at the cost of those who cannot. It’s our responsibility as teachers to be aware of this and design creative solutions to enable every learner to participate as much as they are willing and able, whilst still benefiting from the session.
Adjusting tactics for the ‘new normal’
We’re therefore continually adjusting how we teach CAQDAS-packages online during COVID-19. Current uncertainties land responsibilities on us as teachers, not on our course participants: we must find out what they need, reflect on our practice, and refine our pedagogies.
Moving from in-person to online always requires a redesign (Silver & Bulloch, 2020), but during COVID-19 we are also:
Educating ourselves about accessibility to ensure we sensitively and effectively open our events to every type of learner
Engaging learners more before sessions to understand personal/research needs and provide pre-course familiarisation materials
Reducing class-sizes. It’s often assumed class-sizes can be larger online, but we find the opposite, especially during COVID-19. Although we’ve recently experienced pressure to increase group size, we’re resistant because of the increased need to balance the requirements of every learner, and provide individual support
By co-teaching we provide additional support in synchronous online events during COVID-19. Learners can be split according to their needs and two groups supported simultaneously
Providing more post-course resources to support learners’ continued use of CAQDAS-packages and hosting platforms for them to communicate with one another afterwards (e.g. VLE platforms)
Diversifying teaching tactics to provide as many opportunities as possible for learners to engage and participate. Awareness of different ways people learn has always been central to our pedagogies (Silver & Woolf 2015), but our sensitivities and reflections have increased. We’ve found mixing up tactics (see image) in shorter sessions more effective.
Where do we go from here?
Teachers continually critique and reflect on practice, but COVID-19 requires a re-evaluation of learners’ differences and reflection about their more challenging situations. We are all learning and must continue to do so.
COVID-19 brings ethical issues even more to the forefront, including the appropriateness of requiring or encouraging learners to share their image via video. We must think about disabilities, access to technology, and socio-economic issues in a context where learning is only available online. Positives have also emerged, as sessions can be followed from a range of devices and locations.
COVID-19 forces us to explicitly consider the well-being of learners. Despite coming at this difficult time, we welcome this focus. All our situations have changed, so we need to think about the issues differently. What are the additional ethical issues we must now address? How do we keep this conversation going?
About the authors
Together we have 50+ years experience teaching CAQDAS-packages and 30+ years experience teaching online. Dr Michelle Salmona is President of the Institute for Mixed Methods Research and an international consultant in: program evaluation; research design; and mixed-methods and qualitative data analysis using data applications. Michelle is also an Adjunct Professor at the University of Canberra, Australia specializing in qualitative and mixed methods research. Dr Sarah L Bulloch is a social researcher passionate about methods, with expertise in qualitative and quantitative analysis, as well as mixing the two. She has worked in academic, government, voluntary and private sectors. Sarah teaches introductory and advanced workshops in several CAQDAS packages as a Teaching Fellow for the CAQDAS Networking Project at the University of Surrey, as well as teaching quantitative analysis using SPSS. Dr Christina Silver is Director of Qualitative Data Analysis Services, providing training and consultancy for qualitative and mixed-methods analysis. She also manages the CAQDAS Networking Project (CNP), leading its capacity-building activities. She has trained thousands of researchers in the powerful use of CAQDAS-packages, including NVivo, and developed the Five-Level QDA® method with Nick Woolf.
Kalpokaite, N. & Radivojevic, I. (2020). Teaching qualitative data analysis software online: a comparison of face-to-face and e-learning ATLAS.ti courses, International Journal of Research & Method in Education, 43(3), pp. 296-310, DOI:10.1080/1743727X.2019.1687666.
Salmona, M. & Kaczynski, D. (2016). Don’t Blame the Software: Using Qualitative Data Analysis Software Successfully in Doctoral Research. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 17(3), Art 11, http://nbn-resolving.de/urn:nbn:de:0114-fqs1603117.
Schmieder, C. (2020). Qualitative data analysis software as a tool for teaching analytic practice: Towards a theoretical framework for integrating QDAS into methods pedagogy. Qualitative Research, 20(5), pp. 684-702.
Silver, C. & Woolf, N (2015) “From guided instruction to facilitation of learning: The development of Five-level QDA as a CAQDAS pedagogy that explicates the practices of expert users” International Journal of Social Research Methodology, Vol. 18, Issue 5. Pp527-543
Silver, C. & Bulloch, S.L (2020) Teaching NVivo using the Five-Level QDA(R) Method: Adaptations for Synchronous Online Learning. Paper presented at the QSR International Virtual Conference, Qualitative Research in a Changing World. September 24th 2020
A qualitative survey? What about face-to-face interaction? All the non-verbal cues? Probing and following up? Depth of data? These are the types of sceptical questions we hear a lot when we talk about our research using qualitative surveys. Our doctoral students have even been told at that they must supplement their qualitative survey data with another data source such as interviews, otherwise they will not have the depth of data they need. Sceptical questions like these are partly what motivated us to write about our experience of using online qualitative surveys for the International Journal of Social Research Methodology (LINK TO PAPER). We were also motivated by our enthusiasm for this method and wanted to share with other social researchers why we think it is a valuable addition to their methodological toolkit. We’ve used qualitative survey data over the last decade or so to explore everything from students’ responses to a gay pride T-shirt (Clarke, 2016, 2019) to male body hair removal discourse (Terry & Braun, 2016). We have also supervised numerous students using surveys – including Elicia Boulton, Louise Davey and Charlotte McEvoy, our three co-authors on this paper.
What is a qualitative survey then? Usually a series of questions focused around the topic of interest that participants answer in their own words. But qualitative surveys are not limited to questions and written responses, other possibilities include drawing tasks (see Braun, Tricklebank & Clarke, 2013) and responding to stimulus materials such as audio and video clips. Qualitative surveys are necessarily self-administered – if they were administered by researcher, they would essentially be a rather structured qualitative interview that would fail to reap the benefits of ‘messy’, participant centred qualitative interviewing. Qualitative surveys can be delivered in a variety of formats (hardcopy by post or in person, email attachment) but delivery via online survey software is pretty much the norm now, and that delivery mode is the focus of our discussion in our IJSRM paper.
When we think of (quantitative) surveys – as the sceptical questions we opened with illustrate – we typically think of breadth and more prosaically, larger samples. Whereas qualitative research is typically associated with depth and small, situated samples. How then can a method typically associated with breadth, and quantitative research, have anything to offer qualitative researchers? To appreciate the possibilities of qualitative surveys, we first need to recalibrate how we think of depth – shifting from associating it with individual data items, as is typically the case, to assessing depth and richness in terms of the dataset as a whole. This is not to say that individual survey responses can’t be rich, they can, and we include a powerful example in our paper from Elicia Bolton’s survey of experiences of sex and sexuality for women with obsessive compulsive disorder. Not all responses will be like this though – well, certainly not in our experience of using qualitative surveys so far. But an entire dataset of 60, 80 or a hundred responses will provide a rich resource for qualitative analysis. Survey data also have their own unique character, they are not simply like reduced interview data. They are very focused, dense with information – to the extent that a dataset that runs to the same number of pages as a small number of interview transcripts can feel like a lot of data! Our students typically cycle through an initial panic at the start of data collection or piloting – the responses aren’t very detailed! – to feeling delighted, or even overwhelmed, by the amount of information in the final dataset.
Okay, so survey data can be rich, but why would I use a qualitative survey though, rather than do some interviews over Zoom or Skype, with all the advantages of virtual interviewing? Let’s start with some of the practical and pragmatic benefits of qualitative surveys – for us as researchers. There are no bleary-eyed video calls at 6am or 11pm. Data collection can be relatively quick – and there’s no transcription! – leaving plenty of time for data analysis, which is particularly useful if working to a tight or fixed deadline. We note that we are not advocating for quick (and dirty) as inherently good, however; good quality qualitative research takes time, and using a qualitative survey can allow time for the slow wheel of interpretation to turn when we do not have all the time we would ideally want and need to complete our research. In research with no funding, there are few or no costs associated with data collection (especially if you have access via your institution to online survey software). When it comes to student research, we think qualitative surveys can open up research possibilities – because there is no direct interaction with participants, there are likely fewer ethical concerns around inexperienced researchers addressing sensitive topics. For example, one of our undergraduate students researched young adults’ experiences of orgasm using a qualitative survey – it’s highly unlikely they would have received ethical approval to research this using interviews (see Opperman, Braun, Clarke & Rogers, 2013).
For participants, there are even more practical benefits – not least that they can participate when it is most convenient for them. Louise Davey noted that her participants often completed her survey on experiences of living with alopecia early in the morning or late in the evening; unlikely times for an interview. Online survey software will also usually allow completion over multiple sessions, so participants can complete the survey in several short bursts, fitting participation around their schedule, commitments, and indeed energy. This is one of the ways in which online qualitative surveys can give participants a greater sense of control over their participation. Surveys also typically ask less of participants – they do not have to spend an hour or two talking to a researcher at a particular time, they do not have to travel to meet a researcher in person. They also have the advantage of a strong sense of felt-anonymity (in practice, online qualitative surveys are not completely anonymous) – this can be vital for some topics. In Charlotte McEvoy’s research, for instance, on therapists’ views on class and therapy, some participants commented that they were glad of the anonymity of the survey, they would not have shared what they did – and we can speculate, perhaps even not participated at all – if they were invited to take part in an interview. This connects to another advantage of qualitative surveys – that they have the potential to open up participation for groups for whom face to face participation is challenging in various ways. This includes some disabled people, people with caring responsibilities, people with visible differences – such as alopecia – who may feel anxious about being visible to and open to scrutiny by the researcher, and people for whom social interaction with strangers can be profoundly anxiety inducing (such as people with OCD).
This is just a taster of some of the benefits and possibilities of qualitative surveys. We hope we have enticed you to read further about qualitative survey literature and discover the joys, and challenges, of this method for yourself!
The COVID-19 pandemic poses new challenges for researchers in the social sciences. Several research methods require communication and interaction between the researcher and respondents. Some of my colleagues were planning to conduct focus groups with consumers in multiple countries earlier this year. Due to travel restrictions and social distancing requirements, they had to put their plans on hold. Waiting for the restrictions to be lifted, they re-organized their research plans to focus on other tasks they could complete in the meantime. But as the development of the pandemic remains uncertain, they now must think about alternatives to conduct face-to-face focus groups in order to proceed with their project.
Online focus groups provide a way out of this situation. Many researchers might be hesitant to use qualitative online methods that are not well-established yet. As part of my current research, I conducted both face-to-face and synchronous text-based online focus groups, implemented in the form of web chats. This approach helped me to explore relevant methodological advantages and disadvantages. Based on this experience, I identified key aspects to consider during the preparation and implementation of synchronous text-based online focus groups. These include the size and composition of the groups, the discussion process, the moderator’s lack of control, and technical issues.
With respect to group composition, an obvious benefit of online focus groups is that people from different locations can be brought together. In my study, the focus groups were composed of participants from all over Germany, which would have been doable in a face-to-face setting. The online setting instead allows for the participation of target groups that are otherwise difficult to reach, such as physically disabled persons or people living in rural areas. Further advantages result from this mostly anonymous environment including that participants felt more comfortable discussing sensitive topics, and potential power imbalances became less influential.
On the downside, it is difficult for the moderator to manage the discussion due to the synchronous nature of the discussion process. The lack of visual and vocal cues aggravates this problem and makes it hard to capture feelings or moods. I experienced that participants who type rather slowly found it difficult to keep up with the pace of the discussion. Some were still formulating answers to a question, while the discussion had already moved on. Another major issue is information threading, meaning that the group discussion splits into parallel conversations on different topics.
The moderator’s lack of control also extends to the discussion environment. After one session, I noticed that one respondent had copied and pasted a statement from a website as a response to one of my questions. This was something that previous research accounts of focus groups had not prepare me for and could lead to serious problems for the validity of this method. In my article published in the International Journal of Social Research Methodology, I elaborate on these aspects in more detail and provide recommendations on the selection of suitable chat software to reduce the risk of technical issues and disturbances. It is important to keep differences between online and face-to-face focus groups in mind; but with good preparation, some disadvantages of remote discussions can be overcome. Synchronous text-based online focus groups are a method worth exploring – not only in times like these.
What does it mean to co-produce research? The term is used so extensively and diversely that it might be considered merely a convenient buzzword used at the intersection of academics, funders, and communities. There is an extensive series of justifications for co-producing research with communities. Co-producing research, for example, empowers, emancipates, enacts equal and democratic relationships, and aligns with rights-based discourses. Despite these noble aspirations there is a long-standing critique that co-production reproduces the academic or elite power relations it claims to unsettle, whether as tyranny, governmentality strategy, or regulation. A consensus position is that co-production is the space, discourse, and practices that allow us to unlearn and re-imagine research beyond the inheritances of academia as a society of letters. What grounds and commitments might help us navigate the critique and work towards worthwhile aims in collaboratively researching across and beyond the boundaries of the campus?
The co-productive imagination is inspired by speculative and process approaches to explore ways of theorising and practising collaborative research that are adequate to understanding the complexity of the world. Following A.N. Whitehead, this approach seeks to continually question academic abstractions that may result in our minds and inquiries following familiar grooves. Instead, co-production becomes an imaginative act where we iteratively develop propositions – encounters with new ways of feeling and thinking – and then we cultivate our imaginative capacities to understand what potentials are made possible in this situation. Although including concerns of empowerment and equality the focus becomes the creative realisation of events, which are understood as new ways of thinking and feeling that changes our orientations to the world.
The focus of the article emerges from the Loneliness Connects Us youth co-research project which explored loneliness with young people through arts-based and creative methods. I describe my slow and eventful realisation that one of the youth co-researchers was playing a crucial role in co-producing the project aim of creating a space in which we could explore loneliness ethically, productively, and with care for one another across difference. The way this finding came about questioned any claims I might make for co-production as equality, empowerment, or democracy. Instead, it justified thinking about the imaginative processes and practices of doing the research and realising that something matters! Drawing on speculative and process approaches, the article identifies a series of practices and commitments – appetition, techniques of relation, patterned contrasts – that work towards eventful realisations. It is, of course, important that in the shift from co-production as equality to co-production as eventful realisation that young people are not marginalised in research in pursuit of what thoughts and feelings are interesting. Yet, it is equally important to not let abstractions of co-production fix the ways in which we develop research. The aim therefore is not to lose young people but rather to fold commitments for young people with an ethics of the event, finding and tracing multiple and eventful contributions, collaboratively creating new propositions and possibilities to re-imagine a world in process.