SmartScholar
Helping researchers manage sources more productively
Role
Sole UX Designer and Researcher
Timeline
April 2022 - October 2022
Featured Skills
UX Design
UX Research
UI Design
Prototyping
Usability Testing
Tool
Figma
Overview
Background
When people write research papers, they use secondary sources which are analyses of the research topic by other scholars. Secondary sources can include journal articles, books, and documents in other formats. The practice of storing, organizing, and using sources for research is known as reference management, or simply, managing sources.
Reference management is practiced by anyone who conducts scholarly research, which includes undergraduate students, graduate students, professors, and scientists.
Problem
Managing sources can be an overwhelming, disorganized, and fragmented process.
Researchers make do with storing PDFs on their desktop, reading articles and not remembering what they read, having notes scattered across different documents, and switching between multiple windows. They spend unnecessary energy navigating this disjointed digital landscape. Researchers' inability to effectively manage sources can lead to stress and poor academic performance.
Objective
Design a system that makes managing sources more efficient for people who write research papers.
Solution: SmartScholar
A desktop app that allows researchers to organize sources, process information from the sources, and incorporate the information into their own writing in one streamlined platform. SmartScholar goes beyond existing products on the market by making information more accessible when users write the paper, allowing them to make connections between sources and develop their own analyses more easily.
Save and organize sources
Save sources to the library using the SmartScholar browser extension. The bibliographic information and the file will be imported into the source library. You have an overview of all the sources that you save and you can sort them by identifiers.
Annotate sources
Once you deem a piece of information important, you can highlight it, create an excerpt out of it, or write a note on the page. Creating excerpts and notes, which I call "Annotations," allows you to extract and compile information so you can access it when writing the paper without having to locate it inside the source. You can add tags to annotations to make them easier to retrieve when writing the paper.
Evaluate sources
You can leave a rating for each source to sort them by priority. You can also leave a takeaway to help you keep track of what each source is talking about.
Refer to your annotations while writing the paper
Your annotations and writing document are located side-by-side, allowing you to easily reference the sources while writing the paper. With the excerpts and notes removed from their original document and compiled in one place, you can form connections across multiple sources and generate insights. To view the specific set of annotations that you want to see, you can search for a keyword or apply a filter such as Tag.
Discovery research
Studies show that organizing information in a useful way is a challenge for researchers.
I conducted secondary research to learn more about how researchers practice reference management and what pain points they face. Studies show that researchers struggle to organize information in a way that helps them make connections across sources and form their own analyses, especially if they're working with a large quantity of sources.
Researchers use a wide variety of tools and methods to manage source. Reference management software (RMS) is the main technological solution that exists. These are apps, such as Zotero or Mendeley, that enable researchers to organize references and create bibliographies, among other capabilities. However, they are less helpful when it comes to research synthesis.
Researchers have unique way of writing research papers, but share common practices.
I conducted user interviews with 5 participants who had significant experience with research papers. I recruited them through a screener survey. 3 participants had written a thesis during their undergrad, 1 participant had written research papers throughout their master’s program, and 1 participant was actively publishing scientific papers as a PhD candidate.
My goal was to learn about researchers’ behaviors, needs, and challenges when it came to managing sources for research papers. Each interview began with the question “Can you tell me about the last time that you wrote a research paper?”
Research Synthesis
Users manage sources across 4 phases, facing multiple pain points along the way.
In order to improve the way that researchers manage sources, I needed to understand how they do it. With so much data collected from the interviews, I conducted affinity mapping to identify common patterns and I created a journey map to organize this information.
Step
Gathering Sources
Evaluating Sources
Processing Sources
Integrating Sources
Action
After coming across a useful source, researchers will save it so that they can refer to it later on.
As researchers read the source, they will decide how relevant it is for the paper and what purpose it will serve. They may take notes or use another method to keep track of the sources.
As researchers read the source, first they will label important information (usually by highlighting). Next, they will extract that info and store it somewhere so it can be referred to later. E.g., they might copy and paste the text and compile these annotations in a document.
Researchers will locate the information that they want to use in the paper, which they collected in Step 3, and then write their own words while referencing that information.
Pain Point
Saving sources in a disorganized manner makes them harder to locate.
Keeping track of what each source is talking about and why it matters can be overwhelming, especially when you're working with a lot of sources.
Note-taking, highlighting, and extracting important information from the source can be tedious. There’s no way useful way to organize this information.
There’s no seamless way to access the information that you want to reference while writing the paper.
Need
Be able to organize sources in one location and quickly retrieve what they need.
Be able to keep track of what the sources are talking about and prioritize them.
Be able to easily label and extract important information from sources.
Be able to easily access the information that they need to write the paper without having multiple windows open.
Quote
“I just downloaded sources to my desktop. Things got messy because I was looking at a lot of things. If you don't name the sources it's hard to find them. I would always switch back and forth and try to figure out “was it in this one or that one?”
“I wish there was a better system for prioritizing sources. There are sources that I’m definitely going to use for my thesis and sources that are not as relevant.”
“I read through all the sources and I picked out critical paragraphs. I put this information in the notes section of Zotero. Note-taking is a necessary if not tedious and painful part of the research process.”
“I wanted to see what my sources had to say about the thing I was writing about. If I wanted to make a connection between two sources, I wanted to see that information near each other. I just had all the info compiled in a Word Doc. It was not efficient at all.”
Researchers' pain points revolve around the organization, prioritization, and synthesis of information.
Having sources saved in a disorganized way.
Not being able to keep track of what the sources are talking about.
Not being able to easily extract information from a source.
Not being able to organize information from sources in a useful way.
Not being able to easily access information that they need when writing.
Design Challenge
Researchers struggle to organize and process information in a way that helps them write the paper. How might we help them...
Save and organize sources so that they can be easily retrieved?
Keep track of what each source is talking about and how it will be used in the paper?
Process sources in a way that is efficient and conducive to generating knowledge?
Organize information from sources so that relevant information is accessible when they write their own analysis?
Having gathered all of these insights from research, it was time for me to translate them into a design solution.
Concept Development
I came up with design principles that define what a successful user experience means for SmartScholar.
Accessibility of Information
Information should be accessible to the researcher. After you collect all the important information from the sources, you need to make connections and develop your own analyses from this information. Making information accessible requires careful consideration about how information should be organized and retrieved.
Flexibility
Writing a research paper, and therefore, managing sources, is a fluid, reiterative, and non-linear process. Researchers all have their own way of writing papers and do things in the order that makes sense to them. Instead of imposing a rigid framework, the app should give researchers the flexibility to go back and forth between steps.
I organized the SmartScholar user journey into 3 phases: Discovery, Annotating, and Writing.
Each phase represents an act of information prioritization. As described by one interviewee, "You collect sources and you collect what is important about them and then you collect what is important about the previous things."
Discovery
What sources are important?
Annotating
What's important about those sources?
Writing
What's important about the previous information?
What solution features make sense?
Saving sources with a browser extension and organizing them in a library are features that I adopted from existing reference management software. The challenge for me was figuring out how a user would seamlessly access and refer to the information that they deemed important when they need to write the paper. Without many existing UX patterns to borrow from, I did multiple rounds of sketching to try to figure out what the solution would look like.
Creating user flows helped me anticipate the paths that user would take to accomplish their tasks.
Flow 1: Creating & Opening Projects
Flow 2: Saving and Organizing Sources (Discovery Phase)
Flow 3: Evaluating and Processing Sources (Annotating Phase)
Flow 4: Integrating Sources (Writing Phase)
Design development
I created a set of wireframes to refine the solution features and validate them with potential users.
Flow 1: Creating and Opening Projects
You can create a new project or open an existing one.
Flow 2: Saving and Organizing Sources (Discovery Phase)
You can save sources to the library using the browser extension to gather them in one place and organize them to easily locate what you need.
Flow 3: Evaluating and Processing Sources (Annotating Phase)
Once you deem a piece of information important, you can highlight it, create an excerpt out of it, or write a note on the page. Creating excerpts and notes, which I call "Annotations," allows you to extract and compile information that you deem important so you can access it when writing the paper without having to locate it inside the source. You can add tags to annotations to make them easier to retrieve when writing the paper. You can also label the priority of each source as high, medium, or low.
Flow 4: Integrating Sources (Writing Phase)
You can easily reference the sources while writing the paper by viewing the Annotations and writing document side-by-side. With excerpts and notes removed from the original document and compiled in one place, you can form connections across multiple sources, generating your own analyses. To view a specific set of annotations, you can filter them by criteria such as tag or search for keywords.
Testing designs with 5 participants revealed usability issues and edge cases.
My goals for testing were to 1) identify participants' initial impression of the app, 2) uncover usability issues with the interactions, and 3) assess the value of the features before moving on to high-fidelity design. My participants represented my target users—they all had experience writing research papers at the undergraduate level, including 2 at the PhD level.
Finding #1: The “Add to Annotations” feature is not discoverable.
Before
Participants were not aware of how to create an annotation because the “Add to Annotations” button could only be accessed by right clicking.
After
I improved the discoverability of the "Add to Annotations" feature by making the button permanently fixed in one of the headers above the PDF. When a piece of text is selected, the button becomes active. Even when no text is selected, this feature will still be visible to the user.
Finding #2: Not all sources exist as “good” PDFs.
Feature exploration
The app assumes that users will be working with “good” PDFs, but realistically, users might be working with scanned PDFs which can’t be searched or easily annotated. I designed a text recognition feature to make it possible to work with scanned PDFs. To limit the project scope, I chose not to include this feature in the MVP.
I created a high fidelity prototype and conducted testing with 5 more participants.
My goal was to uncover usability issues and see how participants interpreted the app.
Finding #1: The select priority feature doesn’t make sense.
Before
Users could select the priority of the source (high, medium, low) from the drop down menu to sort the sources by priority. However, the location of the “priority” button makes it appear related to the specific page rather than the source as a whole. Participants also wanted to be able to leave a takeaway about the source (e.g. a note about how the source is helpful), not just label the relative usefulness of the source.
After
Instead of having the user select the priority of the source from the drop down menu, I created a pop up window that prompts them to rate the usefulness of the source and leave a takeaway. They can sort the sources by rating to decide which ones to keep, annotate, or archive. The rating will show up under the "Rating" column and both the and takeaway can be viewed again under “View Source Info.”
Finding #2: Users want to highlight the source before annotating it.
Before
I designed the flow so that the text that the user creates an annotation from would be automatically highlighted. However, participants said that in real life they would highlight the text before annotating.
After
I made highlighting a separate step that precedes annotating. After highlighting all the desired text, the user can click on a piece of highlighted text and click “add to annotations” to create an annotation. The “add to annotations” button will only be in the active state if a piece of highlighted text is clicked on.
Finding #3: There’s friction when navigating back to the Discovery Phase from the Annotating Phase.
Before
When asked to go back to the "source library," some participants struggled because they did not associate the word "Discover" with the act of saving and organizing sources.
After
I renamed “Discover” to “Organize” to make the purpose of this page more understandable.
I iterated the designs and conducted a 2nd round of usability testing with 5 different participants
My goal was to see whether or not the previously identified issues were resolved.
Finding #1: The select priority feature doesn’t make sense.
Result: Partially resolved
All participants were able to rate the source and write a takeaway. However, some were confused about the purpose of this feature and were unsure of where they could access this information again.
Recommendation
Communicate more clearly what the purpose of leaving a rating and takeaway is and where this information goes. Instructions should say something along the lines of “Write down anything you find helpful about the source so you can review it later under View Source Info.”
Finding #2: Users want to highlight the source before annotating it.
Result: Not resolved
All participants were able to highlight text, however, some felt that the highlighter icon did not look like a highlighter. Two participants struggled to create an annotation. One was not aware that she had to select the highlighted text before clicking on “add to annotations”. Another felt that the instructions for creating an annotation were confusing.
Recommendation
Find a different icon to represent the highlighter. Rewrite the instructions for more clarity and accuracy.
Finding #3: There’s friction when navigating back to the Discovery Phase from the Annotating Phase.
Result: Partially resolved
All participants were able to navigate from the Annotating Phase back to the Discovery Phase. However, one participant was confused by the naming of the top buttons (Organize, Annotate, Write). She felt that because these buttons are named with verbs, she didn’t perceive them to be pages that she could navigate to.
Recommendation
Find a different icon to represent the highlighter. Rewrite the instructions for more clarity and accuracy.
Final Design
Save and organize sources in one place.
Save sources using the browser extension
Save sources to the library using the SmartScholar browser extension. The bibliographic information and the file will be imported into the source library. You have an overview of all the sources that you save and you can sort them by identifiers.
Organize and retrieve sources from the source library
The source library provides you with an overview of all of your sources in one place so you quickly retrieve what they need. You can also organize the sources into folders and sort them by different identifiers.
Annotate and evaluate sources.
Create an excerpt
Use the “Add to Annotations” button to create an excerpt out of any text that you want to refer to later.
Create a note
Using the sticky note icon, create a freestanding note to jot down a takeaway about any text on the page.
Add tags to annotations
Creating excerpts and notes, which I call "Annotations," allows you to extract the information that you deem important so that you can access it when writing the paper without having to locate it inside the source.
You can add tags to an annotation to label it in a way that’s useful to you.
Rate how helpful the source is and write a takeaway
Once you reach the end of the source, the pop up window will prompt you to leave a rating and takeaway about the source to help you prioritize your sources and keep track of what they're talking about.
Filter your annotations to see the ones you need
Retrieve a specific set of annotations that’s relevant to your writing by searching for a keyword or applying a filter such as tag.
Write your paper with your annotations side-by-side.
Your annotations and writing document are located side-by-side, allowing you to easily reference the sources while writing the paper. With the excerpts and notes removed from their original document and compiled in one place, you can form connections across multiple sources and generate insights.
Explore the Prototype
View Prototype in Figma
Conclusion
Impact
My goal was to propose a system that makes managing sources more productive for people who write research papers. My discovery research validated the need for an app that helps researchers organize and process information from sources in a way that helps them write the paper. All 15 participants that I tested my solution with expressed the belief that an app like SmartScholar would have made it much easier to write research papers, confirming SmartScholar’s potential to be adopted by researchers if it were a real app.
Recommended Next Steps
Accommodate sources in different formats. Researchers aren’t always working with perfect PDFs. Not being able to annotate a scanned PDF would break the entire user experience. SmartScholar as a real app would need a text recognition feature to accommodate such sources.
Provide more flexibility. Allow researchers to open the Organizing, Annotating, and Writing Phases as separate tabs.
Create a way for the user to cite their sources (in-text citations and bibliography).
Additional testing for different types of users to accommodate different needs.
