“It is impossible to step into the same river twice.” -
Heraclitus
Asking the right question is the fundamental and most basic building block of a successful research project. No matter how much work you put in and no matter how much data you get back, if you begin with the wrong question you won’t get insightful results. While, that is one way to gather data, there is also the Star Trek method. Sometimes, it is also important to ask no questions and just sit back an observe. In truth, a combination of all of these methods leads to the best results.
“Ethnographic methods focus on producing sincere and introspective feedback by building rapport and engaging users as experts rather than research subjects.” - Bronwyn Larsen
If you have ever seen the TV series Star Trek then you will know what ethnographic research is. Though, it is referred to rather differently. We are introduced to a concept known as the Prime Directive. It is explained in an episode from March 1968:
“No identification of self or mission. No interference with the social development of said planet. No references to space or the fact that there are other worlds or civilizations.”
This type of research is the most difficult to arrange and difficult to analyse but it is very rewarding. The essence is to allow the user to tell the story of their interaction/engagement with the product or service in their own words and through their own actions.
Here at at Fluid UI we are currently developing our capabilities in this regard and will be able to do more of this type of research once we launch our new player app features. This will make it possible to view users actions as they engage with your app prototype in Fluid UI.
But, there are some other ways to capture this data. The simplest way is to identify a cohort of users who are most important to your product roadmap. This information can be derived from the quantitative research methods and to a lesser extent from the qualitative research methods. Once you have a selection how do you capture the data?
Essentially there are three research methods that you need to focus on and understand when you are conducting user research. You can collect statistical information which is derived from quantitative methods and you can collect more nuanced, richer and more elusive data, i.e. qualitative methods. Both are essential but the key is to get the right balance between the two. The third method is the ethnographic approach. This is an anthropological term which basically means studying the user within their own environment and without any outside interference or instruction.
You are asking the right questions, which is all I or anyone else can do. If you ask the right questions, the right answers will come.
Lieutenant Worf
In an ideal world it would be possible to correct all of the bugs and implement all of the feature requests that your user feedback throws up. However, in the real world that’s not possible so there needs to be careful thought about which elements of the user research are going to inform changes in the short term.
No matter how relevant the research it will not be possible to move more than 10% into your product roadmap via the product backlog. Asking the right questions is about focusing your research on areas that you can fix and enhance within a realistic time frame.
The right roadmap order for a company is one that prioritises the fastest growth items first at all times.
“Insufficient facts always invite danger.” - Mr. Spock
Mr Spock and Commander Data represent the more statistical approach to any problem. They use logic and facts and data to sum up a situation. Similarly, quantitative research methods set out to collect statistical data and the findings are expressed as numbers which makes it possible to make statistical findings based on the data. The results which derive from quantitative research can be very powerful because they are perceived to be based on real, factual information. While this is true and while it is very common to see such statistical information used to swing arguments in all types of situations we need to be careful when using this type of research.
The most easy to use, accessible and most importantly useful tools are:
So while Data and Spock bring facts to bear on any given problem, it is the other characters who bring the more qualitative approach. The key to remember is that is the combination of both approaches which gets the job done.
“I want to understand the world from your point of view. I want to know what you know in the way you know it. I want to understand the meaning of your experience, to walk in your shoes, to feel things as you feel them, to explain things as you explain them. Will you become my teacher and help me understand?”
- James P Spradley
Qualitative research is all about rather more intangible data. It tries to capture the story behind the numbers.
In an excellent discussion of the the importance of qualitative methods in UX design research, Bill Selman, Lead UX researcher at Mozilla, explains how we all get sucked into the homophily bubble and how we all need to escape from that bubble if we are involved in UX research.
There was one group which summed up the the problem with groupthink which can be brought on by the homophillic bubble.
The Borg are the ultimate Homophillic bubble. There are not shades there is just one homogenous group.
Borg Queen: Human sentiment. Compassion, guilt, empathy - they’re irrelevant.
Seven of Nine: Not to me.
Borg Queen: “Me”? There is no ‘me’. There is only ‘us’. One mind.
