A case study of UX research for parliament’s website

Introduction

Product team says the phrase “User experience (UX)” or rather “good UX” on a daily basis. But what makes a good UX? How to conduct UX research?  Even these days, there is no simple answer since UX design is a product of so many other disciplines and factors (as the diagram below visualizes it). For us – product teams, this means there is always something to explore left and it’s worth doing so regularly.

This time around the challenge was to study the secrets of Information Architecture (IA) through a UX research case study about the Polish parliament website.
‍

‍

Why Information Architecture?

The practice of Information Architecture takes care of arranging information on a website or in an app to make it easy to understand and scalable. Carefully thought out IA – the structure of all the information in your app or website – is a critical part of a UX research.  Why? Well, you can have the most fabulous content, but it is a sound structure that will make it understandable and easy to discover. As nicely put by Prototypr experts, if you do the thinking, your users won’t have to. That complies with one of the key goals of UX design – reducing users cognitive load. If the users understand your product with ease and without too much thinking, it is quite likely that (1) users will achieve their goal or, even better, (2) users will achieve their goal without getting super-frustrated. If you meet the 2nd scenario, your chances of getting these users to keep on using your product are growing.

‍

Why the government website?

Transparency of information is one of the pillars of constitutional democracy – citizens of any democratic country should have full access to the details of what is going on in their country. In the era of modern technology – where all the information is always just a click away, it would seem that providing citizens with that access, should be easier than ever. After all, it should just be a matter of fitting all that information on a website or in an app. Unfortunately, as mentioned above, it is not that simple. With a lack of good UX design and, especially – sound information structure, transparency is at threat.

‍

The problem is common across digital services of government bodies. And, actually, if you think about it, it’s not surprising. With the overwhelming amount of information government bodies have to provide and an extremely diverse user group (often every citizen is a potential user), they’d need to continuously work on their website. Otherwise, UX issues are unavoidable. Aware of the problem and its scale, we decided to dedicate some of our time and resources to the matter, take a closer look at the issue, and propose a, somewhat, universal, solution. The solution took on a form of a framework for low-budget (preliminary) UX research & diagnosis and a plan for more extensive and more expensive, full-on research and redesign phase. 

‍

Our framework for low-budget (preliminary) UX research with the focus on IA

‍UX audit: a cognitive walkthrough conducted by a UX researcher. The audit was done in order to:

• check whether the website does, in fact, suffer from major UX and IA issues,
• highlight the most prominent problems and help set a path for research before engaging users.

Audit results: outdated, cluttered UI design, bad UX, numerous IA problems including ineffective navigation and unintuitive category names

Method’s effectiveness: A cognitive walkthrough is an effective technique when used at the beginning of a research process – however, remember it delivers nothing, but hypotheses so you have to follow-up with user research!

‍

User research: in-house true intent survey and remote tree testing

Taking much navigation - and structure-related issues, two complementing methods – a true intent survey and a remote tree testing – were viewed as the best starting points for getting a full(er) image. 

The UX research was run internally, on a small sample of participants recruited within our company. Politically conscious Polish citizens were targeted and three male and three female participants, with the average age of 30, completed the two-part study. 

• True intent survey: With what aim are citizens visiting the site? What kind of information are they expecting to find? How do they judge the current UX of the site? What are their biggest pain points? 

The survey related to users previous experiences with the website and aimed to gather information about users and their pains as well as help us understand what is it that citizens are looking for on the parliament website.
‍

Survey results

‍

Other aims mentioned:

• Looking up political programmes of different parties and their members: 17%
• Obtaining contact information to different parties and their members: 17% 
• Obtaining a list of current members of the parliament: 17%
  
  ‍

Other problems mentioned:

• Search function not working as expected: 17%
• Inefficiency when looking for relevant information – too many “clicks”: 17%
• Opened pages that were empty or gave a 400 error: 17%

Method’s effectiveness: true intent survey really gets you to understand their users through eliciting their aims, expectations and biggest pain points.

Tree testing: is current labelling on the website understandable to users? are they able to match a type of information with a section name? Are they getting lost along the way?

The tree testing, on the other hand, was used to test the current labelling and organization of information on the website.

‍

Tree testing is a known usability testing technique for evaluating the findability of topics. It is sometimes called a ‘reverse card sorting’ as it is performed on an already existing labelling system. This UX research technique is conducted on a simplified text version of a site’s structure to really focus on the labelling and avoid the bias caused by i.e. visual design. Thanks to having a clear focus on validating just information architecture, it is a great tool for verifying your initial hypothesis about the effectiveness of an IA. 

‍

Tools – an information architecture validation software Treejack

We used Treejack from Optimal Workshop, a dedicated remote tool for tree testing that helps you record detailed information about how users interact with a menu including metrics such as first-clicks, direct success and indirect success. In addition, Treejack gives you the capabilities of an online survey tool with pre-testing and post-testing questionnaires. It’s a sound solution to conduct appropriate UX research, as you can use one tool for both a tree testing and a true intent survey.

Method’s effectiveness: Treejack is a super-handy online tool that lets you perform a survey and tree testing all at once.

‍

Content-related conclusions

The results of the UX research confirmed our hypotheses – the overall UX of the website could be improved, users were not happy with navigation and the overall content structure. In addition, the results indicated that IA flaws were the main source of problems for users of the parliament website. 

Within IA issues reported, ambiguous and/or misleading labelling was shown to be problematic for users. What was described as a “complicated structure of information” was another common problem, suggesting that the content is hard to navigate through and understand. In addition, users reported that they struggled to find the information they were looking for and, as an effect, they did not always achieve their goal. The finding was confirmed by results of the navigation-based tasks. There are 2 generalizable takeaways from these findings:

1. Users should be involved in the process of labelling content as otherwise, you are likely to end up with label names that make sense to you, but not your users,
2. Involve users in the process of organizing product’s content as otherwise the chances are the structure will seem sound to you, but not your user.

Actually, these could be brought down to one –  if you want your product to make sense to and meet the needs of users, involve users in the process of creating IA. We know that it sounds like an absolute UX cliche, yet, somehow it is often overlooked by the product teams – when in a hurry to launch the product, teams cut down on user research, wasting even more resources in the long run.  

We are not just saying that we have proof – when inquired about why they would visit the parliament website, 83% (!) users shared the same general goal –  they all said they would typically visit the website to look up new bills and changes in law that are relevant to them. Yet, for some reason, this piece of content is not prioritized on the website. Such a major structural flaw can be easily avoided with user research. 

‍

In UX research, we need to dive deeper on user goals and expectations. The study clearly indicated that for most users the prime reason for visiting the website is finding information about bills or changes in the law that are specific to them as a, for instance, business owners. Apart from indicating that these are important pieces of information that should be prioritized, this finding is more generalizable as it indicates a user need for customization of content. In the context of the parliament website, this customization could be achieved by i.e. allowing users to filter or sort information depending on a socio-demographic group. 

Another takeaway: Content customization functionality can be a major factor of the UX of your digital product.

‍

Next steps

The framework we proposed has shown to be effective for preliminary UX research in terms of identifying the largest user pain points and UX issues. However, some limitations of our study, mainly: few participants, a single socio-economic group, only three tasks in the tree testing, makes it hard to generalize findings to the whole population. In addition, although we found issues such as ambiguous and/or misleading label names, solving these problems would involve further user involvement. 

We’d sum it up by saying that we came up with a framework that is great for low-cost preliminary research but needs to be followed-up with more extensive user research before a redesign can be started.

We’d suggest the following plan for further research and redesign:

1. Running a bigger, true intent UX research through survey on the general population to incorporate all socioeconomic groups. The survey design does not need to change. 
2. Analysing the results of the study – key takeaways and a prioritized list of topics and tasks that are of interest to potential users as deliverables.  
3. An open card sorting study that incorporates (at least) the topics of more interest distinguished in step 2. (Since the limit for a card-sorting study is 50-60 topics it depends on how long the list form in step 2 would be.).
4. Analysing the data and, based on the results, suggesting a new IA for the site.  
5. A closed card sorting study to evaluate the new IA suggested. 
6. Analysing the results and adjusting the IA.
7. (At least) 2 iterations of designing & evaluating – In the first iteration create new wireframes of the website. The wireframes should (1) follow the conclusions from step 2, (2) incorporate the new IA adjusted in step 6. The new designs should be evaluated, ideally through user testing involving 5 potential users. The wireframes should be adjusted and a more hi-fi design should follow. The hi-fi design, ideally in the form of interactive prototypes should once again be evaluated through a user study and adjusted according to the results.
8. Once the new designs are finalized, the changes could be implemented.

Conducting UX research is a relevant part of your product strategy, whether you're representing your government, looking for a business idea or already started building your product. An early investment in this phase will allow you to understand how to serve the experience to your users and validate your idea to the fullest. Not to mention that having this knowledge from the start will help you build fewer iterations in the future!