This blog post is inspired by an article by Sooraj Shah for the BBC published about a month ago. When I read the article, I was really happy to see that someone was exploring this component of the situation. I did find myself doing an impromptu analysis of the article, comparing it to qualitative research. But that's not what it is, not what it's supposed to be. Articles like this are important and, in my opinion, aren't written enough.
However, it does give me a chance to talk about the reason I was originally drawn to information security and some work I hope to get back to doing one day. There isn't enough qualitative work done in information security, methodologically rigorous work that explores the "what" and "how" of infosec.
I love qualitative research. I spent basically three years studying it and I wish I got to do it more. Proper qualitative research is hard and takes time. I wrote a qualitative report for work earlier this year which involved 12 hour-long interviews conducted over the course of about four months. That's a pretty great timeline to recruit and interview that many participants.
So, why do I love qualitative research so much, and why do I think infosec needs more of it?
A specific type of understanding
Qualitative research allows us to really dig into a topic and ask broad, open-ended questions. Often, a set of hour-long interviews will ask just a few question; document analysis involves reading a lot of material related (and unrelated) to the area of study; observational research can takes months, years. This gets to a point I'll discuss later but the value achieved here is a type of understanding you cannot achieve without qualitative research.
While it can answer many other valuable questions, quantitative research cannot answer questions like "How do employees at a company responding to a breach cope with stress?" or "What frames do media use when reporting on cybersecurity incidents?" and leaving those questions unanswered is a major limitation to our understanding of infosec.
Saturation
My favorite idea from research is "saturation," the idea that you just keep collecting data until you stop finding anything new. This is definitely a moving target and sometimes you reach saturation because the semester was over and your term paper was due. But the underlying drive of qualitative research is to keep collecting data until you have learned as much as possible about a phenomena. That's just extremely cool.
Data sources
Qualitative research includes a broad set of methodologies. Interviews, textual analysis, observation, etc. This diversity of sources allows for breadth of understanding and the potential to collect diverse perspectives for analysis. Each source comes with its own caveats, naturally, but they all contribute to a fuller picture of the phenomena.
That's what I wanted to read after the BBC article, other viewpoints. While we were getting a more qualitative understanding of the situation, the article lacked a diversity of viewpoints because that wasn't the point. I'd love to see an in-depth project exploring the topic, talking to people in a variety of positions, corroborating some of the second-hand anecdotes.
Obviously I'm not saying there is no qualitative work being done on infosec topics, or that quantitative research isn't also immensely valuable. But we need to do more to broaden the types of research being conducted on infosec topics. Even more importantly, rigorous research of all types needs to be encouraged, read, promoted, and cited.
