Year / 2023
Client / New York Public Library
Team Members / Esha M., Roey W., Yaetasmin S.
My Role / User Research, UX Ethnography, UX strategy
Navigating research in the digital age presents a myriad of challenges; accessing online resources, sifting through extensive content libraries, and seeking support can be time-consuming, often leading to intellectual roadblocks that hinder the research process. Recognizing this common struggle, the New York Public Library reached out to us to delve into the challenges researchers encounter at various stages of their research process and explore ways to provide meaningful support.
In line with this vision, our team conducted ethnographic studies focused on understanding the user experience of researchers engaging with the library's services and resources. Following the briefing with the client, i.e., the NYPL team, we defined our research objective:
To assess researchers (unlike non-researchers) utilize the New York Public Library's online and offline resources in their research processes and daily lives.
To conduct this comprehensive study, we connected with researchers and Ph.D. students spanning various fields like User Research, Psychology, Bioengineering, Library Science, etc., through in-depth interviews, participatory workshops, and usability studies. Our findings revealed three key themes shaping researchers' interactions with NYPL's services:
-
Discoverability
-
Accessibility
-
Readability
Notably, we observed that many of the challenges faced by researchers were rooted in their use of NYPL's digital infrastructure. Building on these insights, we presented the NYPL team with key findings and design directions aimed at enhancing the library's web services.
Understanding NYPL's challenges and inhibitions while servicing researchers
Established in 1895, the New York Public Library (NYPL) is a cornerstone of New York City's educational landscape, boasting nearly 53 million items across 92 locations and ranking as the second-largest public library in the United States and the fourth-largest globally. It caters to a diverse audience, including researchers and non-researchers, with a commitment to adapting its services to the digital age.
The NYPL team was facing a unique challenge in catering to a diverse user base while providing access to its extensive holdings. Although the library aims to serve everyone, certain materials were assumed to be utilized by specific patrons, particularly researchers, due to the nature of the content. This assumption created a paradox as the library strived to be inclusive to all types of users while recognizing distinct user behaviors among researchers.
Designing website features catalog filters, search functions, and request forms was tricky because of the wide-ranging user spectrum, even within the category of "researchers". The unspoken assumption was that researchers, in general, possessed a certain level of expertise and preferred more scholarly search methods while utilizing online resources.
From the user's standpoint, interactions with NYPL's digital products primarily involved searching keywords, refining search results, determining how to access and browse materials, and maintaining a repository of the research material . Consequently, our research questions unraveled how researchers might perform these actions differently from non-researchers and what factors might influence their search/browsing habits.
Defining the scope of the ethnographic study and selecting appropriate research methodologies.
Research Objective
To evaluate how researchers/
non-researchers utilize current New York Public Library’s online and offline resources during their research process/daily life
Research Questions
-
How might we define parameters for distinguishing 'researchers' from 'non-researchers'?
-
What are the resources/ services offered by NYPL commonly used by researchers across diverse fields?
-
Why do researchers choose to utilize them?
-
What does their research process typically look like? What do they prioritize during their research process?
-
What are the challenges/roadblocks users face when using NYPL resources?
-
How does the research process differ from online to in-person (and which is preferred/more utilized and why)?
-
According to researchers, what are the other platforms outside of NYPL that are going the extra mile to aid their work?
Research Participants /
For this study, outlining a clear definition and criteria for distinguishing researchers from non-researchers was highly important. At NYPL, the classification of a "researcher" is defined by the "intention to find authoritative material," and not just by their background, experience, or training. This suggested that anyone with the intention, regardless of their experience, is considered a researcher. Given this distinction, recruiting participants from diverse fields and gathering information about their research practices based on intention, accommodating varying levels of experience was crucial to consider. We connected with 5 researchers from academic and applied research backgrounds -
-
Psychology
-
Medical Science
-
Art & History
-
Design Research
-
Library Science
Research Methodologies /
> An On-site observation study (~6h) at NYPL with field-notes
( Location: New York Public Library - Stephen A. Schwarzman Building )
-
Conducting an observational study at an NYPL center helped us gather behavioral insights on how the visitors were utilizing the library space and interacting with their immediate environments. It further helped us build a hypothesis for identifying 'researchers' from others in the room based on the observed ethnographic cues. These observations were crucial for mapping an iceberg model ; what researchers were 'doing' against what they were 'saying' and feeling which the in-depth interviews aimed to further capture.
> In-depth interviews followed involving usability testing of the NYPL desktop website with researchers from different fields
( 5 Participants, available remotely/in-person )
-
In-depth interviews provided us deeper and more concrete insights into researchers' needs, motivations, expectations, and overall decision-making while conducting research and utilizing NYPL's online and offline resources. After gaining an understanding of their research process during the interview session, we conducted a semi-structured usability evaluation of the NYPL website where the participants walked us through the user flows they typically followed while using the NYPL website. These insights uncovered user scenarios and mental models related to how the NYPL website supported users in conducting research efficiently.
> An Ad-HOC interview with a librarian at NYPL.
( 1 Participant, available on-site )
-
Conducting brief, spontaneous 5-minute Ad-HOC interviews with librarians at the NYPL center provided insights into the in-person services preferred by researchers. We discovered the various reasons users approached librarians, observed their interaction patterns with library assistants, and identified common needs/issues for which they sought assistance.
> Participatory workshop with researchers utilizing 'Dot Voting' and 'Forced Ranking' frameworks
( 5 Participants )
-
The participatory workshop was conducted to validate our research findings - challenges and potential design directions for improving the usability of NYPL's web services. We asked researchers to vote and rank both existing problems and their feasible solutions to suit their priorities.
Fig 1. Screengrab of our observation notes and interview summary on Miro
With users' consent, we recorded and transcribed each eye-tracking session to analyze the pain/gain points in the user journeys, aligning with the data from web analytics. The results were analyzed using 'heatmapping,' 'gaze plotting,' and 'scroll mapping' techniques to evaluate user behavior when interacting with the Poster House website on mobile devices.
Fig 2. Results of Dot Voting and Forced Ranking conducted during the participatory workshop with researchers
FINDINGS /
RECOMMENDATIONS
Before diving deep into the key insights of this ethnographic study, here are
some overall findings to make a note of :
1. Researchers’ work is best supported by environments that foster deep focus and allow them to work in solitude.
"Most of the research work I do is from my bedroom or the school library - I need to declutter and peacefully to sit in one place, to actually get work done."
2. Researchers have a heavy reliance on digital infrastructures to conduct, gather, and organize research.
"I prefer online resources because it saves me the time and effort of commuting. Besides, everything is available online."
"I’d like it if when you’re reading on an iPad, you can write notes with your pencil and annotate."
3. Using the NYPL Website creates several pain points that hinders the discoverability, accessibility and readability of resources.
"I wonder if the librarians use a better system because they can always find my wanted information that I can’t."
"I don’t want to keep track of my email to see what I’ve requested and received."
User Persona : Delineating the "ideal researcher" who can utilize NYPL's services
Deep-diving into the 3 paradigms of our findings:
Discoverability, Accessibility and Readability
01 Discoverability
'Discoverability' refers to a web service's ability to enable users to find specific content that meets their needs accurately and urgently. This theme addresses the challenges faced by researchers in finding research material on the NYPL website. Among the various navigation flows for locating desired material, we observed that the search bar was the most frequently used feature. While researchers claimed it provided the quickest and most streamlined approach to finding content, the participatory workshop highlighted that within the processes of searching and filtering search results, several challenges required immediate attention and resolution. In this section, we detail the uncovered pain points and the design solutions proposed to the NYPL team.
Fig 3. Dot Voting results showing that filtering and accessing research materials were top priority
Pain points /
Struggling to use the search feature on the NYPL website to its full potential:
> Keyword searches were not always accurate and often, the results didn’t match their expectations.
> Becoming acquainted with the functionalities of the advanced search feature proved to be a notably time-consuming process for researchers. They claimed that the interface lacked intuitiveness because of its rigid appearance, consequently diminishing the overall user-friendliness of the experience.
Experiencing limitations while using filters to locate research material:
> Researchers found it especially difficult to discover and search for specific digital artifacts when the title or author of that artifact was unknown.
> Researchers needed better classification to organize the search results.
Fig 4 UI of the filter categories and advanced search feature
For improving discoverability /
> Redesigning the search feature interface with a contemporary and streamlined visual design, incorporating filtering categories presented as "tabs" to enhance initial accessibility for users at a glance. (See Fig. 5)
> Reorganizing text-heavy pages in a visually appealing manner to support scroll+ pan and converting important links to prominent Call-to-action buttons. (See Fig X)
> Considering future opportunities to conduct a user study focused on filters to introduce more role-specific and field-specific filters.
> Presenting new users with a pop-up tutorial for utilizing NYPL's core web services in their best interests. (See Fig. 6)
Fig 5. Recommended solution for the search feature interface
Fig 6. Mock-up of the tutorial pop up on the Research page
02 Accessibility
The 'Accessibility' of research material was one researcher's major concerns while conducting research. Right from pay-walled materials to third-party log-ins and inconsistent categories, there's nothing else that frustrates researchers as much as encountering bounce points. The lack of accessibility impacts everyone. This includes individuals with disabilities who may face barriers in accessing information, and services. It also affects those who may temporarily experience limitations, such as individuals with temporary injuries or situational impairments.
Moreover, the lack of accessibility can have broader societal impacts by excluding certain groups from fully participating in various aspects of life, hindering diversity, and perpetuating inequalities. Evaluating how accessible the NYPL was for researchers and thereby bridging any accessibility-related gaps was crucial to ensuring the website aligned with NYPL's mission and vision.
Fig 7. Dot Voisting results for Research support and Accessing research materials
Pain points /
Expecting the digital infrastructure of the library to support their personal struggles and disabilities:
> Introverted researchers wished to minimize social while availing the library services and hence try prefer virtual support over in-person assistance by a librarian.
> Researchers with neurodivergence such as ADHD expressed the needed better organization dashboards to record and track their research progress.
> Researchers noted that the library database should reflect a wider range of literature representing marginalized communities.
Experiencing limitations/ interruptions in the process of sourcing research material:
> Researchers complained about having to switch in and out of the NYPL website for sourcing articles due to the excessive provision of external links.
> They also complained about the multiple logins and signups at different steps in the user flow for accessing the same material.
Users are directed from the NYPL to external websites to access research material causing multiple roadblocks in the user flow and a loss of trust.
"It is only letting me download the citation, I can’t access the actual article unless I access through a separate website even if I have the NYPL membership."
Fig 8. Screengrab of the 3rd-Party login page that users were directed to
> Some materials and categories were locked without any explanation. Researchers claimed that the lack of transparency directly impacted the credibility of the website. They wanted to know WHY an article was not available for open access.
> Some researchers found online databases limited while researching niche topics.
> Some researchers complained about the lack of a variety of digital resources such as books/audiobooks on queer literary materials.
For improving accessibility /
> Prioritizing nesting user activities within the NYPL web interface where researchers can access almost all content using the NYPL membership only.
> Introducing a dashboard where students can track and organize their research materials by project lists in the 'My Account' section of the main menu (See fig. X)
> Pushing for special initiatives for improving the representation of marginalized communities by bringing to light a wider collection of literature across all formats (audio, video, text, etc).
> Strengthening research support on the website by acknowledging that interaction with library staff should be researchers' last resort.
Fig 9. A mock-up of the dashboard for organizing and tracking research projects
03 Readability
"Readability" refers to the organization of information on the NYPL website. This includes the labels for different sections and pages, links, text, mixed media like images and videos, and other types of information visualizations. For evaluating readability, we looked into the visual hierarchy and placement of information to best aid users' lossless comprehension. We observed that wherever readability was compromised, users experienced confusion and a lack of clarity. Subsequently, it resulted in them feeling stuck during their research process, making it likely to affect the bounce rate. To troubleshoot this, users attempted to seek tech support available on the website, approach library assistants or resorted to other websites to source research materials.
"The pages are text-heavy and important links get lost in the information being provided."
Pain points /
Difficulty navigating to desired sections while browsing through information across various pages:
> According to researchers, the interface of the homepage did not correlate well with the research process steps. For example, 'Location services' and 'Events' precede “Get Research Support” in terms of visibility through the navigational flow.
Fig 10. Issues with the information hierarchy on the Research page
> Important Call-to-action buttons were not prominently displayed on the website eg. ‘download pdf’ was located in the sidebar and wasn't always available.
> The DOI was not hyperlinked causing researchers to take additional steps like copy/pasting it in a new browser.
> Despite the vastness of the library's collection, issues like lack of available transcripts may end up isolating users with disabilities.
“ Some of the oral history records do not have all the transcripts uploaded which might be an issue for people with disabilities. ”
For improving readability /
> Ensuring that important research-specific services are illustrated early on in the webflow by attributing a dedicated section on the homepage for them. (See fig. X)
> Including advanced search filters along with a citation tool from the beginning of the search journey to avoid multiple steps. (See fig. x)
> Potential to conduct an in-depth card sorting test to evaluate the efficiency of existing labels on the website for each section.
Fig 11. A mock-up highlighting the organization of text-heavy pages
Fig. 12. A Mock-up of moving important research-specific services up
In conclusion /
The foundation of any meaningful design for a product or service lies in a deep understanding of the contexts, needs, and behaviors of the eventual users. Ethnography plays a crucial role in this regard. In a collaborative 4-month project with the New York Public Library, we embarked on an exploration of the NYPL community. Our goal was to identify their needs, uncover strategies for innovation, and shape design decisions through a practical application of ethnographic methods.
Our project primarily focused on quantitative research methodologies, including observational studies, in-depth interviews, usability evaluations, and participatory workshops. From these efforts, three key themes for our research findings emerged: discoverability, accessibility, and readability. These themes were recognized as critical criteria that directly impacted researchers using the NYPL website.
In presenting our findings to the larger NYPL team, we incorporated design solutions to enhance the overall website experience. The team appreciated our insights, leading to further discussions on strengthening support for individuals with disabilities. Our research presentation emphasized a narrative that positioned the NYPL as a research library for the unaffiliated, aligning with their core mission and vision of inclusion and equity.
Following our recommendations, the visual hierarchy of the Research page was redesigned to prioritize research resources and services. We also ideated on features such as a tracking system for organizing research materials project-wise. Throughout this project, I refined my professional skills in scoping a research study, planning fieldwork, conducting ethical observational research, moderating in-depth interviews, designing frameworks for participatory research, and analyzing and presenting research findings.