Choices, Controversies and Challenges in Computer Assisted Qualitative Data Analysis

By Christina Silver, CAQDAS Networking Project, Department of Sociology, University of Surrey, UK & Qualitative Data Analysis Services Ltd. [email protected]

This article provides a brief overview of my Keynote presentation at the 1st International Symposium on Qualitative Research in Porto, Portugal in July 2016 when I reflected on the current state of Computer Assisted Qualitative Data AnalysiS (CAQDAS) and the issues we face in an increasingly digital future. A full account is in Computer Supported Qualitative Research, 2017 [1].

 

CAQDAS choices

CAQDAS packages are not new. Since the 1980’s we’ve seen significant developments and now there are many choices between and within dedicated qualitative analysis programs (collectively known as CAQDAS packages). There are many packages to choose among – see the CAQDAS Networking Project website for software reviews.

CAQDAS packages are not methods of analysis, but simply provide a suite of tools that can be harnessed in various ways, according to the needs of the analysis. Initially these programs provided tools to support qualitative approaches to qualitative data, but the choices have increased because they now also contain tools that enable quantitative approaches to qualitative data, mixed approaches to qualitative data, and mixed approaches to mixed data. These developments in CAQDAS packages appeared at times of changing methodological currents in qualitative research, and Davidson & di Gregorio [2] identified six stages of software development that map to the eight critical moments in the chronology of qualitative research described by Denzin & Lincoln [3]. The Figure below adds the release of the wide choice of dedicated CAQDAS packages onto Davidson & di Gregorio’s mapping.

As well as choices between CAQDAS packages, we also have many choices within individual programs – because a particular analytic task can often be accomplished in several different ways. These choices in the way that the software can be harnessed provides tremendous flexibility. Taken together, choices between and within programs reflects the wide variety in qualitative and mixed-methods research and personal preferences. That there continues to be space for so many programs in the market and that no one program has emerged as the “CAQDAS standard” underlines that there is no one “best” software [4].

 

CAQDAS controversies

Not all researchers use dedicated CAQDAS packages, and debates about their appropriateness impact the future of our community of practice. Researchers fall into three groups:

1. Those who advocate the use of CAQDAS packages;
2. Those who are concerned about or criticise its role and impact and therefore resist its use; and
3. Those who are unaware of the existence of dedicated software designed to facilitate qualitative and mixed methods analysis.

Jackson, Paulus & Woolf [5] describe the polarising discourses that have characterised the field, and the importance of open debate between these groups rather than to allow the controversies to continue. We have a responsibility to raise awareness of the role and potential of dedicated CAQDAS packages, to debate their use openly, and to appropriately equip those who decide to use them with the means to do so powerfully.

 

CAQDAS challenges

The choices and controversies discussed so far leave us facing three key challenges:

1. How we can transcend polarizing discourses in order that the qualitative research community does not become fractured
2. How we will respond to continued technological and methodological developments on our field
3. How we can adequately equip researchers to make informed choices between and within technologies

The first is discussed by Jackson [6] and Jackson, Paulus & Woolf [5], and the second by Silver & Bulloch [7]. The third challenge is the focus of my current work.

 

The Five-Level QDA method

Preparing researchers to make informed choices requires teaching CAQDAS within appropriate methodological contexts, rather than as independent, decontextualized training in how to operate the  software. Integrating software instruction with education in methodology is both necessary and challenging, but the challenges are not insurmountable [8]. A pedagogy that transcends methodologies, analytic techniques, software packages and teaching modes could prompt a step-change in the way qualitative research is taught in the current digital environment. Nick Woolf and I designed the Five-Level QDA® method to explicitly address these challenges. It involves learning to “harness CAQDAS packages powerfully”, which means using the chosen program from start to finish while remaining true throughout to the iterative and emergent spirit of qualitative and mixed methods research. It explicates the unconscious practices of expert CAQDAS users, and involves a different way of harnessing computer software from a taken-for-granted or common sense approach of simply observing the features on a computer screen and looking for ways to use them.

The following five principles that underlie the method are fully explained in our books [9, 10, 11].

1. Strategies – what we plan to do – are distinct from tactics – how we plan to do it.
2. In qualitative research, analytic strategies and software tactics are contradictory by nature.
3. Avoidance or compromise, two common ways to deal with the contradiction between the nature of software and the nature of qualitative analysis, are problematic ways to manage the contraction.
4. Transcending the contradiction keeps analytic strategies distinct from software tactics by translating between them.
5. Each iteration of the heuristic process of translation is always driven by analytic strategies but may be informed by software tactics.

The key skill is translation. Because translation operates at the detailed level of individual analytic tasks, the method is relevant across methodologies and software programs and teaching the process is adaptable to a range of instructional designs.

It is increasingly less acceptable to teach methodology without technology. The Five-level QDA method intentionally gives a separate emphasis to analytic strategies and software tactics within a single framework which enables the teaching of methodology and technology concurrently within an instructional design that is adaptable to local contexts, as well as serving as a method to harness CAQDAS for researchers’ own projects.

 

[1] Silver C. (2018) CAQDAS at a Crossroads: Choices, Controversies and Challenges. In: Costa A., Reis L., Souza F., Moreira A. (eds) Computer Supported Qualitative Research. ISQR 2017. Advances in Intelligent Systems and Computing, vol 621. Springer, Cham

[2] Davidson, J., Gregorio, S.: Qualitative research and technology: In the midst of a revolution. In: Denzin, N K, Lincoln, Y. (ed.) Sage Handbook of Qualitative Research. pp. 627–644. Sage Publications (2011).

[3] Denzin, N., Lincoln, Y.: Introduction: The discipline and prac¬tice of qualitative research. In: Denzin, N. and Lincoln, Y. (eds.) The landscape of qualitative research. pp. 1–44. Thousand Oaks, CA: Sage. (2008).

[4] Fielding, N.G., Lee, R.M.: Computer analysis and qualitative research. Sage, London (1998).

[5] Jackson, K., Paulus, T., Woolf, N.: The Walking Dead: Unsubstantiated critiques of qualitative data analysis software (QDAS) and our failure to put them to rest. The Qualitative Report (in press).

[6] Jackson, K.: Determinism Vs. Constructivism: The polarizing discourse regarding digital tools for qualitative research and how it threatens our scholarship, (2016).

[7] Silver, C., Bulloch, S.L.: CAQDAS at a crossroads: affordances of technology in an online environment. In: Fielding, N., Lee, R., and Blank, G. (eds.) The SAGE Handbook of Online Research Methods. Sage Publications, London (2017).

[8] Silver, C., Woolf, N.: From guided-instruction to facilitation of learning : the development of Five-level QDA as a CAQDAS pedagogy that explicates the practices of expert users. Int. J. Soc. Res. Methodol. 18, 527–543 (2015).

[9] Woolf, N., Silver, C.: Qualitative Analysis using ATLAS.ti: The Five-Level QDA^® Method. Routledge. London. (2018)

[10] Woolf, N., Silver, C.: Qualitative Analysis using MAXQDA: The Five-Level QDA^® Method. Routledge. London. (2018)

[11] Woolf, N., Silver, C.: Qualitative Analysis using NVivo: The Five-Level QDA^® Method. Routledge. London. (2018)