Reaction

Abstract

Simple up/downvotes, arguably the most widely used reaction design across social media platforms, allow users to efficiently express their opinions and quickly evaluate others’ opinions from aggregated votes. However, such design forces users to project their diverse opinions onto dichotomized reactions and provides limited information to readers on why a comment was up/downvoted. We explore user-generated labels (UGLs) as an alternative reaction design to capture the rich context of user reactions to comments. We conducted a between-subjects study with 218 participants to understand how people use and are influenced by UGLs compared to up/downvotes. Specifically, we examine how UGLs affect users’ ability to express and perceive diverse opinions. Participants generated 234 unique labels on diverse aspects of a comment. Leaving more reactions than participants in the up/downvotes condition, participants reported that the ability to express their opinions improved with UGLs. UGLs also enabled participants to better understand the multifacetedness of public evaluation of a comment.

Motivation

Prior research has shown that reactions play an important role by influencing users’ perception of each comment, perceived public opinion, and their opinion on the issue.

To understand how well current reaction designs capture and deliver users’ opinions, we conducted a formative study (N=10) and observed how participants use up/downvotes (the most widely used form of reaction) in an online discussion.

We observed that people leave up/downvotes for diverse reasons and with different thresholds. However, when interpreting others’ reactions, they make a shallow assumption that up or downvote means agreement or disagreement. Also, we observed that people choose not to leave any reaction when they cannot precisely express their opinion or think the impact of their reaction is small.

Our observations highlight the need for better design reactions that 1) capture and deliver diverse opinions and 2) encourage users to react.

User-Generated Labels

We introduce the concept of user-generated labels (UGLs) as an alternative reaction design that captures diverse and precise reactions to a comment. With UGLs, users can describe their thoughts on a comment in short text and vote on labels created by others.

UGLs can capture diverse and dynamic reactions and increase the perceived contribution of users’ reactions by allowing them to make their own labels. Also, by clicking on existing UGLs, users can precisely express their reactions in a light-weighted manner.

Create text-based reactions

Vote on others’ reactions

Demo

You can experience the UGLs in this link.

Please visit our demo and explore how UGLs change your experience in online discussion.

Results

Through a between-subjects study (N=218), we compared UGLs to Up/downvotes. Participants were invited to join an online discussion on one of four social issues (capital punishment, affirmative action, animal testing, consumer data). Participants were guided to comment, reply or make reactions to others' comments as much as they would like to. Below we highlight our key findings.

(1) Participants generated UGLs on diverse aspects of a comment.

Participants (N=109) in the UGL condition generated 234 unique UGLs. Generated UGLs captured diverse aspects of evaluation, such as the level of agreement, the strength of argument, the writing style of a comment, or judgment on a commenter.

The categories of UGLs with descriptions, examples, the numbers of UGLs generated, and the number of votes.

(2) Participants with UGLs left more reactions than participants with up/downvotes.

Participants in UGL condition left more reactions to a comment, and reacted to more comments than participants with up/downvotes. Participants noted that they could express their opinion on each comment precisely, even when they have mixed evaluation (both positive and negative) of a comment.

The average number of up/downvotes (Binary) and generated/voted UGLs (UGL) on the six initial comments.

(3) Participatns with UGLs held higher percieved accuracy and uniqueness of their reactions, better understood others' opinions, and were more influenced by others' reactions.

Perceived accuracy and uniqueness of own reaction and interpretability and influence of others’ reactions (scale 1-7)

(4) Participants with UGLs better understood the multifacetedness of public evaluation of a comment.

In an open-ended question that participants were asked to think about others’ reasons for upvote. Only around 30% of Binary participants mentioned the reasons other than simple ‘agreement’ In the UGL group, 50% of participants were able to identify multiple reasons for upvotes.

Paper

Link to the PDF

ACM DL Link

ACM Reference Format

Eun-Young Ko, Eunseo Choi, Jeong-woo Jang, and Juho Kim. 2022. Capturing Diverse and Precise Reactions to a Comment with User-Generated Labels. In Proceedings of the ACM Web Conference 2022 (WWW ’22), April 25–29, 2022, Virtual Event, Lyon, France. ACM, New York, NY, USA, 10 pages. https://doi.org/10.1145/3485447.3512243

BibTex

@inproceedings{10.1145/3485447.3512243,
    author = {Ko, Eun-Young and Choi, Eunseo and Jang, Jeong-woo and Kim, Juho},
    title = {Capturing Diverse and Precise Reactions to a Comment with User-Generated Labels},
    year = {2022},
    isbn = {9781450390965},
    publisher = {Association for Computing Machinery},
    address = {New York, NY, USA},
    url = {https://doi.org/10.1145/3485447.3512243},
    doi = {10.1145/3485447.3512243},
    booktitle = {Proceedings of the ACM Web Conference 2022},
    pages = {1731–1740},
    numpages = {10},
    keywords = {online discussion, social computing, computer-mediated communication, reaction buttons, social media},
    location = {Virtual Event, Lyon, France},
    series = {WWW '22}
}