Strategies in Qualitative Research

• Home
• Conference series
  • 1999-2004
  • September 2005 Durham
    • Programme
      • Abstracts
    • Papers
  • September 2006 Durham
• Author index
• Site search
• Site map

Author: Ozlem Bak, MBA, MA.
School of Management, University of East Anglia
TITLE: Enhancing the manual data analysis: A critical validation tool.
SESSION 1B: GROUNDED THEORY AND QDAS

Abstract:
This paper represents a qualitative study where the data analysis divided into two stages; Phase A and Phase B. The data analysis of Phase A, was analysed manually using the grounded theory. The set of data incorporated in-depth interviews; participant observation and documentation. There are three reasons related to the adoption of grounded theory in this study. First, grounded theory was useful here because it allows a focus on contextual and processual elements as well as the action/interaction of players associated with the research subject. Second, the iterative nature of the methodology requires a steady move between the data and the concept allowing to themes emerges requiring a constant comparison. Allowing Phase A data analysis to propose an initial formulation of the key categories and interactions. The categories and interactions presented in Phase A was also reflected further in Phase B data analysis that incorporated the same set of data and analysis of data. The two distinctive methods has been used a source and a tool for validation. Despite same emerging categorisation some enhancements on findings have been noticed.

Therefore, the structure of this paper will be elaborated on these grounds into four main sections. The first section identifies the linkage between grounded theory method and its application in this study, followed by the ways of data analysis whether manually versus software (N6), and if yes, what are the patterns.


Author: Pat Bazeley
Research Support P/L, Australia
Keynote address
TITLE: Quantifying qualitative data: the construction and interpretation of codes

Abstract:
Coding, as a way of identifying content or meaning or other features in data, is an essential component in almost any system for analysis of data. Coding serves as a tool for data management and/or data reduction, a way of locating and grouping or retrieving all responses or material on a topic (whether that be descriptive or interpretive). It also marks the beginning of an analysis process as categories are determined and decisions are made about what to code (particularly with unstructured non-numeric data).Yet codes—the way they are generated, what they stand for and the way they are used—lie at the heart of differences between text and numeric data and tools for analysis of that data .

Currently available QDA computer software makes transfer of data from one form to another and hence integration of text and numeric data for combined analysis relatively simple. Transfer of categorised or numeric information from a statistical dataset into a qualitative analysis (where it might be used to compare subgroups within the sample or to make sense of scores on an instrument, for example) generally raises few concerns. But when data is converted from textual form to numeric form, differences in the process of coding and in how codes are used to represent the material being coded become critical issues for interpretation in subsequent (statistical) analyses. Questions to consider include evaluation of the numeric properties of the converted codes, implications of the directionality and specificity of codes and of absent or ‘deviant’ codes, and potential loss of data where qualitative interpretation relied on the intersection of multiple codes. Choices about the segmentation of data, and ways of dealing with overlaps and repetitions will all have bearing on the handling and interpretation of quantified qualitative data.

These considerations, along with sampling issues, will impact on choice of statistical techniques and on the ability to interpret results from the statistics. The potential, however, is for deriving a whole new dimension of understanding from the statistical analysis, further enriched by the ready availability of the source (text) data.



Author: Dr. Kristina Bennert
Thomas Coram Research Unit, University of London
TITLE: Using NVivo for data processing and preliminary analysis in discourse analytic research
SESSION 3C: USING NVIVO IN DIVERSE QUALITATIVE TRADITIONS

ABSTRACT:
Discourse analysis - used here as an umbrella term for approaches working with detailed transcripts of spoken language data - is concerned with the fine-grained turn by turn analysis of verbal interaction. As such, it tends to focus attention on relatively short data extracts, taken from longer stretches of talk such as interviews, focus groups or naturally occurring conversations. Researchers often build rather large corpora of recordings and/or transcripts of the type of interaction they are interested in, and then chose a handful of small data chunks for detailed analysis. Extracts might be chosen on the grounds of their typicality or atypicality within the larger corpus. However, the exact process by which extracts are selected often remains unaccounted for, and large parts of the gathered material might be discarded or put to the side at an early stage.
With the possibility to code at the level of single words or syllables and to edit text which has already been coded, NVivo has a flexibility and capacity for fine-grained textual analysis that was absent from previous software packages and that make it of interest to discourse-analytic research, though to date, not many discourse analysts seem to have tapped into its potential as yet.
This presentation will use data from a project examining communicative frames in genetic counselling to illustrate how NVivo can fruitfully support data processing and preliminary analysis in discourse analytic research. More specifically, I will demonstrate how NVivo can be used:
• To organise and manage multi-page transcripts
• As a tool for preliminary analysis through coding, mapping and indeitification of recurrent patterns
• As a means to contextualise specific data extracts within their interactional and thematic landscape
• To make the whole of a data corpus accountable and guide the systematic selection of illustrative extracts
The presentation will conclude by drawing attention to a few shortcomings of NVivo for discourse-analytic purposes and formulating a wish list for future versions of the software.




Author: Pat Chung
Southampton University
TITLE: Using N6 to analyse inteview data from carers of people with dementia-preliminary analysis
SESSION 1B: GROUNDED THEORY AND QDAS

ABSTRACT:
Introduction: This paper makes an attempt how N6 has been used to assist the process of data collection and analysis in a research study. Occupational therapists increasingly ask carers to become involved in activity programmes which engage their family member with dementia within the homes. The few studies carried out in this area have focused on institutional settings. It is crucial for healthcare professionals to listen to the views of carers of people with dementia. The present study aims to explore this issue by asking the carers how they view activity within the cared-for individuals and their home lives. This is key if professionals are to work in partnership with carers for people with dementia and really address what matters to them.
Method: This was the final stage of an in-depth interview study of up to 30 co-resident carers of people who are formally diagnosed with dementias, and supported by a local Community Mental Health Teams for Older People. Participants were invited to take part in audiotaped interviews and discuss a) how they involved the cared-for person in activity which was considered beneficial to the person, b) concerns they have, c) additional support they would value. Interviews were tape-recorded, transcribed and analyzed using grounded theory method and N6.
Results: Findings showed how a combination of the use of N6 and grounded theory has enabled the researcher to identify the process in which co-resident carers engaged individuals with dementia in activities which were of benefit to them, and highlighted the barriers both human and non-human which they experienced.
Conclusions: In order to deliver the most appropriate care it is important to understand the meaning of activity for the carer. The strategies used for the analysis was appropriate. Despite this, some constraints were identified.


Authors: Keith Coupland and Dr. Lynne Johnston
University of Gloucestershire
Title: Using QSR NVivo in Phenomenological Research: The Experience of Recovering From Psychosis Through Groupwork
SESSION 1C: MULTI-LEVEL DATA AND RIGOUR IN THE RESEARCH PROCESS

ABSTRACT:
This presentation will look at aspects of qualitative research in the phenomenological tradition using QSR NVivo. QSR NVivo is used to hold and analyse multi-media data in order to understand the aspects of recovery of members of a group who meet to deal with their problems with psychosis, especially hearing voices (malevolent audio hallucinations). The nature of psychosis is such that straightforward transcribed audio interviews alone may not be sufficient for the participants to be able to articulate their experience. Therefore, the data is collected in many forms such as audio and video recorded interviews and groupwork, minutes of meetings, biographies and creative illustrations such as poems, pictures and photographs. The use of multi-media data has implications for true informed consent, especially in research with persons experiencing psychosis. This multi-media approach has been as important for the participants in the research as it has to the researchers in helping to understand the experience of psychosis. This study has been conducted over seven years, building an unusual level of trust and confidence within the participants. The variety of data allows a greater level of understanding of the participants' experience of recovery in the group. However, with each increase in medium used there seems to be an exponential leap in the complexity of organising and analysing the data. There is a lack of clear direction within the research methods literature regarding the use of NVivo within different methodological approaches or the use of the package with multi-level data. This session will explore these complexities by illustration from the data and the researchers use of QSR-NVivo.





Author: Judith Davidson, Ph.D.
Graduate School of Education, University of Massachusetts-Lowell
TITLE: Grading NVivo: Making the Shift from Training to Teaching with Software for Qualitative Data Analysis
SESSION 4A: TEACHING WITH NUD*IST AND NVIVO

ABSTRACT:
As the use of software for qualitative data analysis becomes more widespread among researchers, it will be increasingly important to consider how these tools will be integrated into research preparation programs in higher education. This shift from training to teaching raises a host of new problems, not least of which is the need to grade students’ use of the software. In this paper I will explore the tensions that emerged as I sought to shift from trainer to teacher of NVivo in the context of a doctoral course on qualitative research methods, and the path I followed as I learned to evaluate student’s use of NVivo and the products they created from that use. Teaching with NVivo (as opposed to training) opened my eyes anew to the complexity of the software, bringing new understanding of the features and the ways they could be combined; it reminded me again of how different human beings are in the ways we each approach a new task; and, it raised productive dilemmas for me about the relationship of the methodology to the technology. In this paper, I will compare the ways seven advanced doctoral students progressed in their understanding and use of NVivo. I will also share a rubric that I have begun to develop to capture the range of issues that one must consider in the grading of qualitative research processes and products that are embedded in the use of software like NVivo. Ultimately, I came to feel that NVivo was an ally in developing a rich assessment of student learning, helping me to capture student’s progress over time in the successive iterations of the project that I received from them.




Facilitator: Judith Davidson, Ph.D.
Graduate School of Education, University of Massachusetts-Lowell
Panelists Lyn Richards, QSR, software developer; Lynn Johnston, University of Gloucestershire, UK; Silvana DiGregorio, SDG Associates; Pat Bazeley, Research Support, Australia
Panel Discussion
TITLE: Evaluating the performance of NVivo users:  What demonstrates skill with NVivo...and why is it important to do so? 
As the use of qualitative research software enters the mainstream of methodological practice, no longer is it enough to simply declare that you use this form of software as a warrant of trustworthiness.  Now, increasingly, it becomes important to understand the qualities of performance that users display if one is to understand whether or not the research is trustworthy.  Moreover, as these tools become a required part of advanced methodological training, instructors must be able to describe and discern levels of performance and to guide students from beginning to advanced performance.  Drawing upon the experience of the software developer and highly experienced software trainers and teachers.  The aim of this panel is to extract from these experts their embodied knowledge about NVivo and the qualities of performance in its use.  In this way, we will seek to uncover information critical to evaluating the performance of NVivo users.  Each panel member will begin with a brief overview of their thinking on this issue, and then the facilitator will moderate a discussion that will include her questions and the audiences.  Two recorders will capture critical notions on user performance as the discussion unfolds.



Authors: Jacques de Wet & Dr. Zimitri Erasmus
Department of Sociology, University of Cape Town, South Africa
Title: Towards Rigorous Practice in Qualitative Research
SESSION 1C: MULTI-LEVEL DATA AND RIGOUR IN THE RESEARCH PROCESS

ABSTRACT:
Qualitative research, in particular data analysis, is too often seen as ad hoc, intuitive, unsystematic and thus without academic rigour. We challenge this view. The central purpose of this paper is to illustrate that qualitative data analysis can be systematic, procedural and rigorous.

In this paper we provide an overview of analytical procedures we followed during a study about students' experiences and perceptions of 'race' and racism at a Medical School in South Africa. We outline our implementation of these procedures and reflect on their value in optimising our research outcome. We track steps in our analysis by working backwards from one cluster of key findings in the study concerned in order to demonstrate the ways in which we came to these particular findings. Where appropriate, we note the ways in which Computer Assisted Qualitative Data Analysis Software (CAQDAS), specifically the qualitative software package QSR Nvivo, contributed to systematic and rigorous practice at critical points in the analysis. When we outline and reflect on our analysis, we draw on Miles and Huberman's (1994) and Wengraf's (2001) approaches to qualitative data analysis and on Morse et al.'s (2002) understanding of rigour.

We conclude that our use of QSR Nvivo facilitated systematic organisation and procedural analysis of our data. QSR Nvivo provides a system of electronic tools for organising, retrieving and verifying data thus enabling one to work with data more efficiently. It does not do the analysis, nor does it think for one. Well-organised data enables researchers to implement procedures more effectively, which in turn contributes to rigorous analysis.

Finally, we conclude that while it may be true that some of the accusations that qualitative research is sloppy and unscientific are unjustified, our experience is that qualitative researchers do not instill confidence in their research by continuing to produce research reports where their methods of analysis are not well formulated. The challenge is to identify methodological and analytical benchmarks for qualitative research. This paper demonstrates our latest attempt at “working at sensible canons for qualitative data analysis” (Miles and Huberman, 1994:2).



Authors: Claire Fox & Nathan Hughes
Institute of Applied Social Studies, University of Birmingham
Title: The Challenges of a Complex National Evaluation
SESSION 1A: DOING EVALUATION RESEARCH IN TEAM CONTEXT

Abstract:
The Children's Fund was set up by the Children and Young People's Unit as a complex intervention aimed to tackle social exclusion. At the heart of this intervention was the development of innovative models of collaboration in the design, implementation and evaluation of preventive services for children and young people aged 5-13, and their families. The Children and Young People's Unit has commissioned  the National Evaluation of the Children's Fund (NECF) with the aim of exploring what works in prevention and why.

The programme of research is being undertaken by researchers at The University of Birmingham, in partnership with researchers at the Institute of Education. In 2004 and 2005 the Birmingham team is undertaking detailed case-study work in eighteen programmes through three six-month blocks of activity. The case studies aim to achieve enhanced understandings of how interagency partnerships work in participative ways for prevention. This paper details the work of the Birmingham team’s in-depth qualitative analyses of structures and processes, following the completion of the first wave of case study work, exploring how the functions of NVivo allowed us to tackle the numerous problems of a large scale evaluation

We are using Activity Theory (Engeström, 1987; 1999a; Leont’ev, 1978) as the framework for our analysis of the case studies. Activity Theory enables us to understand systems and the processes that lead to outcomes. It requires a very structured analysis, highlighting key elements of the activity system within a strict categorisation, with little opportunity for flexibility. NVivo was utilised to allow for a consistent analytical approach within a large multidisciplinary team and across multiple case study sites. A key element of the theory is the highlighting of contradictions and tensions within a system. NVivo provides the opportunity to do this both within and across interviews. It also enables us to examine whether there are multiple understandings of the ‘object’ of activity. Furthermore, through Merge for NVivo we have been able to trace the development of key issues across case studies to maximise our understanding of ‘what works’ and why.


Author: Amândio Graça
University of Porto;ISMAI;Technical University of Lisbon
TITLE: Using NVivo 2 to analyse physical education cooperating teachers’ educational perspectives
SESSION 4B: STRATEGIES FOR ANALYSING DATA ACROSS THE QUANTITATIVE/QUALITATIVE DIVIDE

ABSTRACT:
This study focused on physical education cooperating teachers’ conceptual orientations in relation to teacher education by looking into their knowledge, experience, and purposes for action; and also their perceptions about the student teachers' fears and needs, or about the kind of relationships they establish with them.

The theoretical framework of the study was based on the Grossman's (1990) model of professional knowledge for teaching, and on Feiman-Nemser's (1990) conceptual orientations in teacher preparation.
Fifteen physical education cooperating teachers from 7-12th grade schools in the District of Porto (Portugal) participated in the study. Cooperating teachers were purposefully selected according to teaching and cooperating teaching experience. Three even groups were constituted. Data collection was made by means of open-ended semi-structured interviews. The interviews were recorded, transcribed and introduced into the qualitative data analysis NVivo2 software, which assisted the coding and drawing conclusions processes. Data analysis began by coding the material of interviews according to a coding frame depicted from the theoretical framework, which included 5 major themes: teacher education goals; teacher role; teaching and learning; knowledge for teaching; and learning to learn: Themes are not sought to be mutually exclusive, as if they intend to reveal specific components of personal conceptual orientations, they also have to acknowledge their own interaction and interdependence. Similar and contrasting perspectives among cooperating teachers' conceptions were systematically searched within each particular node. Exhibits in tables were generated from the condensation of raw material collected by means of matrix (mainly Boolean searches: attributes x nodes intersection). NVivo2, while confined to basic operations, proved to be a very handy and helpful resource.



Author: Graham R Gibbs
University of Huddersfield
TITLE: Narrative analysis and NVivo
SESSION 3C: USING NVIVO IN DIVERSE QUALITATIVE TRADITIONS
Abstract
Many undertaking narrative analysis can see little use for CAQDAS. I will suggest ways in which coding and searching in NVivo are useful in supporting a narrative analysis.

I distinguish those aspects of qualitative data analysis that are concerned with the careful organisation and management of research and those that are concerned with interpretation. The organisation and management of research is an important aspect of analysis which contributes to the quality of analytic work. Moreover, whilst interpretation is the preserve of the human analyst, other aspects of analysis can be enhanced by the use of NVivo.

I examine the use of the text search facility in NVivo both to get to know the data and to find particular language use, such as metaphors, accounts and narrative moves. There are some strategies that can be used to improve the searching and to record what has been searched for. I will also consider the use of coding in narrative analysis, and explain how this coding can be used to promote the reading of transcripts when developing an analysis. Such coding is not done primarily to support comparisons or matrix searches, but to underpin the analysis of the narratives in the text. This will be illustrated from a project examining student motivations for study.



Authors: Graham R Gibbs & Celia Taylor (University of Huddersfield), Ann Lewins & Nigel Fielding (University of Surrey)
TITLE: User needs in learning to use NVivo and N6

SESSION 4A: TEACHING WITH NUD*IST AND NVIVO
ABSTRACT:
This paper will report the early results from an ongoing ESRC project aimed at proving Online Support for researchers learning QDA and CAQDAS such as NVivo and N6. The project is surveying current CAQDAS users to assess their learning needs. On the basis of this it will create and evaluate online training materials for qualitative researchers. The materials will be integrated with other aspects of the CAQDAS Networking Project.

The survey will identify the types of researchers using QDA and CAQDAS. There are several key groups here: Postgraduates and other researchers familiar with qualitative data analysis, but new to CAQDAS, researchers familiar with research methods but new to both QDA and CAQDAS and researchers from outside the social sciences who are also often new to both QDA and CAQDAS.
The paper will report on the kinds of training support these new users need and suggest some of the ways that online training can meet these needs.


Authors: Dr. Linda Gilbert & Dr. Silvana di Gregorio
Title: Team research with QDA software: Promises and pitfalls
SESSION 3A: TEAM-WORKING WITH QSR SOFTWARE

Abstract: 
The purpose of this paper is to explore actual and potential influences of QDA software on qualitative research performed in teams. We will draw on literature on teamwork and on observations of QSR software in use with teams, in order to offer useful strategies for members of qualitative research teams. 

In the last decade, qualitative research conducted in teams has attracted increased attention: there has been a spate of recent case studies, including a complete issue of Qualitative Health devoted to team research (Barry, Britten, Barber, Bradley, & Stevenson, 1999; Erickson & Stull, 1998; Martinez-Salgado, 1999; Richards, 1999). Most of these case studies offer lessons learned, advice, and warnings based on their authors' own experiences. Many bemoan the paucity of previous literature, particularly in methods texts. 

However, very little has been published reflecting on the impact of qualitative data analysis programs on teamwork. Reports from team members using QDA either tend to focus exclusively on the technology or to virtually ignore it. There are exceptions, most notably Lyn Richard's reflection on a 3-year project using an early variant of the software she co-developed with her husband (Richards, 1995). Similarly, Sprokkereef and her co-authors (1995) observed that the use of QDA software affected their team interactions, since not all the team members used or valued the software. A few more recent works suggest ways to use software with a team, though most of these are highly targeted to address specific needs (di Gregorio, 2000a; MacQueen, McLellan, Kay, & Milstein, 1998; Northey Jr., 1997) 

With the exception of developers and a few reflective users, the overall level of awareness about the intersection between technology and teamwork seems low among qualitative researchers. This paper seeks to fill that gap with practical advice. 




Author: Lynne Johnston
University of Gloucestershire.
Title: Technical and methodological learning curves: Reflections on the use of QSR NVivo in Doctoral Research.
SESSION 3B: SOFTWARE IN THE DISSERTATION

ABSTRACT:
This presentation draws on a range of experiences developed over the last 10 years starting with my own doctoral thesis using NUD*IST version 3, to my observations as a software trainer and consultant, through to my more recent experiences as a doctoral supervisor and examiner. Despite the introduction of the Economic and Social Research Council (ESRC) research training guidelines for postgraduate students in 2001, many of the original problems that I encountered as a student remain. This emanates from the existing separation of methods training from qualitative data analysis (QDA) software training and the dearth of existing methodological papers on the impact of integrating training (Jackson, 2003). The lack of clear training guidelines for doctoral supervisors and examiners exacerbates the situation. The well-documented problems associated with getting too close to data (di Gregorio, 2003; Gilbert, 1999; 2003; Richards, 2002) are commonly experienced within doctoral research in the UK. However, I would suggest that there are several reasons for this. First, QDA software programmes have arguably increased the popularity of qualitative research and researchers are starting to explore innovating ways of using the software (e.g. Bazeley, 2003). Second, the transparency that QDA software programmes permit may merely highlight a problem that has always existed. The problem for current doctoral students is that their examiners and supervisors can have unparalleled access to the analysis processes. This has resulted in a much higher level of transparency in terms of research processes (Bringer, Johnston & Brackenridge, 2004). Finally, the free tutorials, which are distributed with the software, have influenced the way in which doctoral students who are self taught have used the software. The misuse of these tutorials can lead directly to a code and retrieve cycle.


Authors: Dr. Dan Kaczynski (University West Florida) and Dr. Ed Miller (Research and Evaluation Associates)
TITLE: Evaluation Team Design Considerations Using NVivo
SESSION 3A: TEAM-WORKING WITH QSR SOFTWARE

ABSTRACT:
This session will consider the unique issues of designing a multi-site qualitative evaluation study by teams of evaluators. Two-member evaluation teams are using NVivo to manage and analyze qualitative data from nine different communities throughout the United States. Research and Evaluation Associates (REA), a Research Triangle applied research firm, is conducting a longitudinal process evaluation of the Youth Offender Demonstration Project (YODP), a national initiative funded by the U.S. Department of Labor. Communities are visited twice over a one-year period. Each team spends 7-10 days on site for each visit observing organizational aspects as well as how youth connect with the project and employers. Results of the first round of visits were used to further modify the evaluation design for the second round of extended visits.

Design methodology will be presented covering team member selection criteria, training, structured code tree protocols, free node guidelines, and code book modification guidelines. Particular attention will be given to flexible emergent design considerations that occurred from the initial conceptualization, implementation and mid point of the study. The initial design involved team members preparing narratives and reflective memorandums at the end of each day’s observations and interviews. When the researchers left the field, they began, in consultation with the REA office staff, final coding of text data, and organizing the data into more precise conceptual categories to support their analysis. To strengthen investigator triangulation, inter-coder reliability verification was enhanced through a two-stage review process. Team members submitted their coded NVivo project to the REA staff who then conducted a second coding of the project. The NVivo project was then reviewed by REA project administrators. To further enhance dependability and confirmability, an analysis oversight committee was included in the design. The committee holds quarterly meetings to review team feedback, data analysis procedures, coding discrepancies and approve modifications to the emergent design.


Authors: Dan Kaczynski (University West Florida), Kristi Jackson (QuERI), Lyn Richards (QSR)
TITLE: Examining the Relationship of QDAS with Theory and Practice
SESSION 4A: TEACHING WITH NUD*IST AND NVIVO

ABSTRACT:
Qualitative researchers are progressively expanding the adoption of qualitative data analysis software (QDAS), as a tool, in the interpretation and analysis stages. This growing application of QDAS has been cited as a major contribution to the rigor and credibility of qualitative research. But there has been little systematic discussion of the different relationships QDAS has with various theoretical orientations. Moreover, software use has also raised concerns that the tools increasingly drive methodological training and practices.

Effective instructional delivery requires well designed lessons with clearly specified learning outcomes. Teachers and trainers of QDAS are challenged by the demands of designing lessons that integrate the technical skills of the software with qualitative field research that is already underway. Although trainees who are actively engaged in studies can immediately apply new skills, the demands on research design methodology are unfortunately stressful for the researcher. Often, if the software technical commands become burdensome then mastery of the software is delayed or abandoned. For the researcher, meeting a report or dissertation deadline has a higher priority than mastery of the software tool. Teachers and trainers must design lessons, therefore, that are immediately relevant to a researchers work and where the QDAS is a transparent tool.

This session will explore three theoretical orientations; social program evaluation, grounded theory, and applied ethnography. Each theory will be examined from a learning outcomes perspective. Session attendees will have the opportunity to participate in and critique a series of simulation exercises designed to help the researcher integrate NVivo and qualitative research training.  Well designed lessons can not only build competence in the use of QDAS but can help the researcher develop theoretically sound techniques for data construction, illuminating richer meanings from documents, refining interview techniques, in addition to category construction, coding, interpretation, and analysis.



Author: Jennifer Mason
Leeds Social Sciences Institute, University of Leeds
Plenary
TITLE: Ways, means and motives: a personal journey through 20 years of computer assisted qualitative data analysis

ABSTRACT:
This paper will chart the personal history of the author’s involvement in computer assisted qualitative data analysis, from the early ‘cut and paste’ days, through to the use of CAQDAS in large multi-disciplinary projects. Throughout the different projects that she has been involved in, a consistent theme has been that CAQDAS has been seen as a tool that can potentially assist in the management of large amounts of qualitative data, rather than a source of fascination in itself. The author would like to think she has retained a healthy cynicism about the limitations of CAQDAS, as well as an appreciation of its value. A concern with ways and means and motives – what can CAQDAS do for us and why should we want it to? - characterises the author’s engagement with CAQDAS over the years.

However, much has changed over time. The personal journey will follow several timelines, each of which is directly relevant to how we view and use CAQDAS in qualitative and mixed method work in shifting contexts over time. These timelines include:

• the changing state and availability of hardware and software
• movements through different kinds of working (eg individual and team working), and the changing nature of people’s relationship to the research process over time
• the changing nature of research teams and collaborations, in particular the moves towards bigger interdisciplinary teams doing qualitative or mixed method work
• the increasing interest in a wider range of qualitative materials (especially non-text based)
• changing fashions and emphases in social theory and explanation

The paper will argue that the point of a personal history such as this is not so much to explain ‘how we got here’, nor for the story-teller to be self indulgent, as to illustrate how CAQDAS usage is both shaped by and shapes the changing nature of research endeavours over time (and over lifetimes). We need to retain an active engagement always with how things could be other than they are if we are to get the best out of CAQDAS, as well as to ensure that qualitative and mixed method research are intellectually driven.


Authors: Kim Nichols Dauner, MPH: Sara J. Corwin, MPH, PhD; Willie H. Oglesby, MSPH, PhD; Kara Montgomery, DrPH; Donna L. Richter, EdD, FAAHB
Arnold School of Public Health, University of South Carolina
Title: Using NVivo for a Qualitative Evaluation of an After-School Program
SESSION 1A: DOING EVALUATION RESEARCH IN TEAM CONTEXT

ASTRACT:
Measuring the outcomes resulting from after-school programs is a challenge. Test scores and grades only reflect academic changes and do not fully describe youth development outcomes. Furthermore, it is often difficult to attribute academic changes to an after-school program. Because of these concerns, and the fact that this program was only in its first year of development and implementation, qualitative data was used. It allowed the evaluators to describe more subtle changes in youth development resulting from the middle-school after-school program and engage all of the programs’ stakeholders in the evaluation. Interviews were conducted with key school and school district personnel responsible for the oversight and day-to-day implementation of the program, community partners who provided cultural enrichment activities, and the program’s Community Advisory Board made up of area leaders. In addition, direct observations were made of the program. Nvivo was used to analyze both the interviews and the observational field notes, with each document representing one interview or one observation. Document attributes were assigned by importing an MS Word table of interviewee/observation characteristics of interest. The search tool and document sets were used to validate interviewee responses, and to corroborate interviewee responses with the observations. Intersection matrixes also were used extensively to determine whether different stakeholder groups had different perspectives on program outcomes and to make stakeholder-specific recommendations for program improvement.


Authors: Pernilla Pergert and Solvig Ekblad
Karolinska Institutet, Sweden
TITLE: A grounded theory study using Nvivo in the analysis and in method learning
- Focus group interviews regarding staff experiences from the care of families with immigrant background in child cancer care
SESSION 1B: GROUNDED THEORY AND QDAS

Abstract:
The aim of this study is to explore the situation of immigrated families within child cancer care in Sweden. To get different perspectives on the subject, a variety of sources and methods for data collection (triangulation) are used, such as: review of the current, relevant literature, document analysis, semi structured individual interviews with parents, and focus group interviews with healthcare staff. Five focus group interviews have been conducted with staff within child cancer care. Purposive and convenience sampling has been utilized and resulted in a sample of people who were knowledgeable about the phenomena under study. The data from the focus group interviews have been analysed using the computer program NVivo and the methodology of grounded theory. This is a complicated methodology which the researcher is learning by supervision and guidance from an experienced supervisor and a research group, but also by creating a model in NVivo. This model helps the researcher to, in a structural way, understand the different steps of this method and the relationship between them. The research group has also been utilized in the analysis of the data and to learn from each other how to best use the data program. The basic social problem from the perspectives of the healthcare staff is the development of a caring relationship with patients and their families with immigrant background in childhood cancer care in Sweden. This phenomena is being further studied, however, an opportunity to share the methodological experience with participants on the conference would be very useful during the present analysis of the material.


Authors: Alina Reznitskaya (Montclair State University) and Richard C. Anderson (University of Illinois at Urbana-Champaign)
TITLE: Quantifying Qualitative Data: Using NVIVO to Analyze Argumentative Discourse
SESSION 4B: STRATEGIES FOR ANALYSING AND MANAGING DATA ACROSS THE QUANTITATIVE/QUALITATIVE DIVIDE

ABSTRACT:
The academic study of argument is currently experiencing a renewed interest from educational researchers concerned with student development of reasoning . In this presentation, I will describe the use of NVIVO to devise analytic frameworks that help to capture and represent important features of argumentative discourse. I will discuss the application of NVIVO to three separate analyses of student argumentation . The data generated in these studies came from 1) oral discussions of controversial issues, 2) written persuasive compositions, and 3) written recalls of an argumentative text.

My colleagues and I applied Toulmin’s model of argument to assess the quality of student reasoning. NVIVO made it possible to code student productions into relevant categories (e.g., reason, counterargument, rebuttal) and to apply NVIVO search tools to create summaries of student codified contributions. Subsequently, the generated quantitative data was analyzed using statistical software packages, such as Excell and SPSS. Multiple analyses (MANOVA, Poisson regression) were performed.

My colleagues and I see quantitative methodology as a complimentary, rather than a rival, approach to analyzing oral and written discourse. It allows us to achieve the level of precision unattainable with verbal descriptions alone and to apply powerful statistical methods. Using quantitative methods with our data helps us to produce a different kind of knowledge and to expand our understanding of student development of argumentation and reasoning. NVIVO is an invaluable tool in this endeavour




Author:Tom Richards,
Founder and Chief Scientist, QSR International,
Keynote address
TITLE: Unleash the power within!
What Node Hierarchies are really all about, why they are the heart of powerful research techniques, and where they can take us next.

ABSTRACT  :
Coding is a central activity in most forms of qualitative research (QR). Earlier technologies (pen, paper, copiers, filing cabinets) required a certain style of coding in order to work reasonably efficiently given their limitations and inflexibility. Researchers have often carried that style of coding over into their computer-based QR projects.

But it is inappropriate. Indeed it can easily be downright dysfunctional, at least in the current generation of QR software. This paper will show how appropriate methods of computer-based coding derive from an understanding of the other tools QR software (well, N6 and NVivo) provide. I will argue that the researcher’s very choice of coding categories, and their cataloguing in tree hierarchies, is deeply affected by many of the analysis-oriented tools and functions in the software. You can’t just code away then hope the software will do something useful with it.

Consequently, getting the coding methodology right is crucial for carrying out the types of powerful research, going way beyond manual technologies, that this software provides. I will be demonstrating that in this presentation.

Finally I will argue that a clear-eyed understanding of the coding and cataloguing methodology suggests ways of extending the very concept of coding to literally another dimension beyond the present logic of categories and nodes. The future beckons….


Author: Lyn Richards,
Director, Research Services, QSR International.
Keynote address
TITLE: Validity and Reliability? Yes! Doing it in Software.

ABSTRACT:
Qualitative research is in danger of throwing out the crucial standards of validity and reliability with the now very murky bathwater of the debate over truth and reality. Enthralled by the important debates over reflexivity and relativism, researchers too often feel unable to claim ‘good authority’ for anything. The long tradition of negativism about ‘positivism’ undermines our ability to teach competent ways of handling data or arriving at robust outcomes from doing so. And the failure to teach such competencies means that practically, even if they saw it as appropriate, qualitative researchers are prevented from attempting to produce valid and reliable outcomes.

In this keynote I argue for the common language meanings of ‘valid’ and ‘reliable’, and for these as standards to set for our research, without distortion by misplaced stereotyping of scientific method. I examine in turn the pragmatic requirements for claiming validity and reliability, the many standards and techniques used, practical problems with what I term “Hollywood validity” and “Inside-Dopester validity” and with the “reliability-checking” techniques currently in demand. I look at the ways software assists accurate scoping of data, interrogation of emerging themes, assessment of saturation and keeping of log trails and how to use software to make reliability measures (coding reliability, ‘Triangulation’ and ‘Member checking’) reliable and (!) valid, and how software supports such claims and assists in addressing the problems.

Concluding that, with software support, such claims are now accessible, I ask why researchers retreat from making them. Why are quantifiable reliability measures prioritized over the validity goals of sound arguments based in good handling of evidence? And what can be done about it?



Authors: Donna L. Richter and Louis Clary
Arnold School of Public Health, University of South Carolina
TITLE: Using NVivo in the Analysis of Data from a Site Visit Program
SESSION 1A: DOING EVALUATION RESEARCH IN TEAM CONTEXT

ABSTRACT:
Qualitative research has become a primary means of research in the evaluation process associated with the Centers for Disease Control and Prevention /Association of Schools of Public Health sponsored Institute for HIV Prevention Leadership. Qualitative data is collected through site visits to selected community based organizations (CBOs), where participants in the Institute (scholars), as well as members of their respective CBOs and collaborating organizations, are interviewed using a discussion guide. Individual and focus group discussions are used.

After transcription, the data is imported into NVivo. Coding teams of two to three researchers code the data. After coding, they meet to arrive at consensus about the data. A initial codebook is developed based on the discussion guide, and is refined as themes emerge during the actual coding of the data. From the NVivo-produced node reports, trends are reviewed and analyzed. Boolean, text, and proximity searches are conducted to test initial findings and reveal additional findings. Attributes are utilized to differentiate types of interview participants (scholar, scholar’s peer, scholar’s supervisor, scholar’s staff) so that differences in perspectives can be analyzed and quotes appropriately attributed in final reports.


Authors: Marya L. O. Shegog, MPH, CHES1: Jaquie Fraser PhD2: Donna L. Richter, EdD, FAAHB1
(1) Arnold School of Public Health, University of South Carolina, Columbia, SC (2) Armstrong Atlantic University, Savannah, Georgia Title: Using NVIVO for a mixed method analysis: Lessons learned
SESSION 4B: STRATEGIES FOR ANALYSING AND MANAGING DATA ACROSS THE QUANTITATIVE/QUALITATIVE DIVIDE

ABSTRACT:
Combining qualitative and quantitative data analysis techniques can provide a more comprehensive picture of data collected. This process is often done separately, utilizing two different analysis packages. By utilizing the attribute function in QSR NVIVO or formatting the data utilizing templates and headers, qualitative and quantitative analysis techniques can be combined to expedite the research process.

The Youth Risk Behavior Survey (YRBS) developed by the Centers for Disease Control and Prevention provides a quantitative measure of the health risk behaviors of high schools students in the United States. The administrators of this survey in Savannah, Georgia, added open-ended questions to seek greater qualitative information on the perceptions of youth in the Savannah area regarding smoking. The resulting data included quantitative data and qualitative information on the participant’s perceptions on smoking and its impact on social status in the high school setting.

Initially the data was coded for both the quantitative identifiers and the qualitative input from each of the respondents. The process of coding over 900 individual documents for dichotomous questions and demographic information was very time intensive. In attempt to better facilitate the project, the researchers modified the coding process by conducting searches to code all dichotomous questions and demographic information. Although the searches did reduce the time spent on each document, further research and training in QSR NVivo highlighted more efficient methods like creating document templates, assigning attributes and importing SPSS files that would streamline the analysis process and maximize the utilization of the software. 



Author: Dr Chih Hoong Sin
Matrix Research and Consultancy, London
TITLE: USING NUD*IST (VERSION 6) IN EVALUATIVE RESEARCH
SESSION 1C: MULTI-LEVEL DATA AND THEIR MANAGEMENT AND ANALYSIS WITH THE USE OF NVIVO
ABSTRACT:
Evaluation can be broadly characterised by formative and summative approaches. In terms of policy evaluation, the latter have greater appeal in terms of the promise of providing answers to the question of ‘does it work?’. In the UK context, this has been spurred by the ‘Modernising Government’ agenda and by the introduction of Comprehensive Spending Reviews by the Treasury since 1998 that requires government departments to produce evidence on both the effectiveness of existing programmes, and the likely effectiveness of proposed new programmes, in order to gain funding. Unlike formative evaluations, however, summative evaluations tend to utilise more quantitative methods. However, there is a growing realisation amongst policy-makers in the UK that the establishment of causality using quantitative approaches alone is inadequate. Coupled with a parallel policy development of recognising the importance of localities, this has encouraged the use of more qualitative approaches in not just answering the question of ‘does it works?’ but also ‘where and why does it work?’. The use of Computer Assisted Qualitative Data Analysis Software (CAQDAS) in evaluations thus has to be situated within such contexts. Its use is reliant on (1) the position of qualitative methods and data in relation to other evaluation components, (2) the types of qualitative data that tend to be collected for evaluative purposes, (3) the scale of data collection, (4) the size of the evaluation team, (5) the types of outputs required from the qualitative component of evaluations particularly in relation to a policy audience, and (6) the time allowed for qualitative analysis. In these various ways, the use of CAQDAS for evaluative research may differ significantly from its use on more conventional forms of qualitative research projects. This paper explores the use of NUD*ist version 6 (N6) to manage data from the qualitative component of a large-scale evaluation, raising issues for consideration.



Authors: Smith, Lillian U; Richter, Donna, L; Watkins, Kenneth; and Usdan, Stuart and Miner, Kathleen
Arnold School of Public Health, University of South Carolina and Rollins School of Public Health, Emory University
Title: Using NVivo to Trace Diffusion of Distance Education in Schools of Public Health
SESSION 4B: STRATEGIES FOR ANALYSING DATA ACROSS THE QUANTITATIVE/QUALITATIVE DIVIDE

ABSTRACT:
In order to explore “why” and “how” the innovation of distance education was successfully diffused in schools of public health, qualitative research was utilized through a mixed methods approach. The researcher traced the diffusion process by utilizing a qualitative, multiple-case study methodology using a semi-structured interview to collect the perceptions from five schools of public health and a monitor survey (an internet windshield survey) of each program’s website to assess the presence of program components and corroborate those identified in the interviews.

All data was collected, stored, and analyzed in an NVivo project file, with the monitor survey data stored as attributes and interviews as text files. By using NVivo, the researcher was able to explore the data within the context of each question as well as through pulling out content themes. The content themes were organized into tree nodes with parent nodes for the major theme and child nodes for the sub-themes. As new ideas emerged from the data or as connections or meaning became clearer, the nodes were reorganized and renamed using the node browser. The system’s robustness and flexibility allowed the researcher to freely code passages in new or collapsed nodes, as well as enabling a seamless switch from question to content analysis.


Author: Chris Thorn
Wisconsin Center for Education Research 
TITLE: Nvivo as a teaching environment:eating your own cooking
SESSION 4A: TEACHING WITH NUD*IST AND NVIVO

ABSTRACT:
I am preparing an advanced course on applied qualitative methods for the Educational Psychology Department in School of Education at the University of Wisconsin-Madison. I plan to teach the course from within NVivo as I present alternate forms of analysis and operationalization of methods from both a scholarly (literature-based) and applied (multiple forms of primary data) evidence base. I will be using the modeler to describe process in the course and to show the links between method, data generation/collection, and analysis. It is also my plan to use NVivo to incorporate student work into my course as part of an iterative model of improvement in which we learn from each other about how to work with data and build arguments. 


Author: Fiona Wiltshier,
QSR International and Monash University, Melbourne
TITLE: Working in tandem: NVivo and EndNote, paper and PC
SESSION 3B: SOFTWARE IN THE DISSERTATION


Abstract:
Working in software from the proposal stage provides a strong start to the whole research process. The focus of my own doctorate proposal changed twice, once by choice, and once more because I found a reference to research covering the same topics and issues as those I’d proposed. I used NVivo and EndNote together to support this process of exploration.

Some of the literature reviewed in the earlier stages was easily incorporated into later proposals. Concepts such as sense of self, identity, the body and the search for meaning through phenomenology remained constant and were streams that flowed through each. The context however, moved. Initially the focus was on women who choose to become mothers at a later stage in life; now however the research focuses on women who participate in more ‘masculine’ type sports such as bodybuilding, and how they see themselves as sexual beings.

This process of combining NVivo and EndNote allowed fast and easy access to both the data found and the thoughts thus provoked. Demonstration of the resultant proposal project shows how the tools of both were combined to work with the data to the best advantage.

The other aspect of process that this paper will cover is the ability to use both programs in conjunction with paper based methods. Using software does not preclude the use of paper but rather enhances it. Referencing both paper based and online articles was made easy through proxy documents, and databites were used to link directly to online articles and other data such as photographs. A strong base was thus created to support the proposal process as the contexts moved.

The data generated through the proposals now forms the start of the research project itself.