publications

2024

1. A computational social science approach to understanding predictors of chafee service receipt

   Jason Yan, Seventy F Hall, Melanie Sage, and 2 more authors

   Children and Youth Services Review, 2024

   Abs Link to Paper Code

   The John H. Chafee Foster Care Program for Successful Transition to Adulthood (CFCIP) allocates funding to provide services to youth who are likely to age out of foster care. These services, covering everything from mentoring to financial aid, are expected to be distributed in ways that prepare youth for life after care. One natural question to ask is, which youth receive Chafee services? The present work makes use of the National Youth in Transition Database (NYTD), a large-scale administrative dataset that tracks services allocated to youth that use CFCIP funds to answer this question. Specifically, we conduct a forensic social science analysis of the NYTD data. To do so, we first use computational methods to help us uncover the factors that best predict which youth will receive services associated with service receipt. We find that the majority of variables in the Adoption and Foster Care Analysis and Reporting System (AFCARS) and NYTD have limited or no utility in predicting Chafee service receipt, and that a subset of three variables—youth age, youth time in care, and the state in which a youth is in care—explain almost all variability in service receipt. We conclude with a discussion of the implications of these and other findings on future research on Chafee service allocation, and the utility of predictive modeling in child welfare, with a particular focus on the utility of the NYTD in this context.

2. An evaluation framework for predictive models of neighbourhood change with applications to predicting residential sales in Buffalo, NY

   Jan Voltaire Vergara, Maria Y Rodriguez, Jonathan Phillips, and 4 more authors

   Urban Studies, 2024

   Abs Link to Paper Code

   New data and technologies, in particular machine learning, may make it possible to forecast neighbourhood change. Doing so may help, for example, to prevent the negative impacts of gentrification on marginalised communities. However, predictive models of neighbourhood change face four challenges: accuracy (are they right?), granularity (are they right at spatial or temporal scales that actually matter for a policy response?), bias (are they equitable?) and expert validity (do models and their predictions make sense to domain experts?). The present work provides a framework to evaluate the performance of predictive models of neighbourhood change along these four dimensions. We illustrate the application of our evaluation framework via a case study of Buffalo, NY, where we consider the following prediction task: given historical data, can we predict the percentage of residential buildings that will be sold or foreclosed on in a given area over a fixed amount of time into the future?

3. The Context of Care: Abstraction Hierarchy Modeling of Therapeutic Foster Care Programs

   Connor Wurst, Huei-Yen Winnie Chen, Ann M Bisantz, and 4 more authors

   Journal of Cognitive Engineering and Decision Making, 2024

   Abs Link to Paper

   Therapeutic foster care agencies provide temporary placements and a range of services to at-risk youth to help ensure their safety, permanency, and wellbeing. The practitioners that plan such care operate under heavy caseloads, limited resources, and high stakes. There is significant interest in supporting these practitioners with various technological interventions, but their work and the context around it is still poorly understood. This study aims to better understand the current assessment and treatment planning work in therapeutic foster care. We used the abstraction hierarchy modeling approach to outline the purposes, values, constraints, processes, and tools that define the workplace ecology encountered by care coordinators and clinicians from therapeutic foster care programs at Hillside, a collaborating human service organization. The resulting abstraction hierarchy was closely examined to identify areas for interventions and design implications.

4. Strategic Behavior in Two-sided Matching Markets with Prediction-enhanced Preference-formation

   Stefania Ionescu, Yuhao Du, Kenneth Joseph, and 1 more author

   Advances in Neural Information Processing Systems, 2024

   Abs Link to Paper Code

   Two-sided matching markets have long existed to pair agents in the absence of regulated exchanges. A common example is school choice, where a matching mechanism uses student and school preferences to assign students to schools. In such settings, forming preferences is both difficult and critical. Prior work has suggested various prediction mechanisms that help agents make decisions about their preferences. Although often deployed together, these matching and prediction mechanisms are almost always analyzed separately. The present work shows that at the intersection of the two lies a previously unexplored type of strategic behavior: agents returning to the market (e.g., schools) can attack future predictions by interacting short-term non-optimally with their matches. Here, we first introduce this type of strategic behavior, which we call an adversarial interaction attack. Next, we construct a formal economic model that captures the feedback loop between prediction mechanisms designed to assist agents and the matching mechanism used to pair them. Finally, in a simplified setting, we prove that returning agents can benefit from using adversarial interaction attacks and gain progressively more as the trust in and accuracy of predictions increases. We also show that this attack increases inequality in the student population.

5. Who supports Bernie? Analyzing identity and ideological variation on Twitter during the 2020 democratic primaries

   Stef M Shuster, Celeste Campos-Castillo, Navid Madani, and 1 more author

   Plos one, 2024

   Abs Link to Paper Code

   Using a novel dataset of 590M messages by 21M users, we present the first large-scale examination of the behavior of likely Bernie supporters on Twitter during the 2020 U.S. Democratic primaries and presidential election. We use these data to dispel empirically the notion of a unified, stereotypical Bernie supporter (e.g., the “Bernie Bro”). Instead, our work uncovers significant variation in the identities and ideologies of Bernie supporters who were active on Twitter. Our work makes three contributions to the literature on social media and social movements. Methodologically, we present a novel mixed methods approach to surface identity and ideological variation within a movement via use of patterns in who retweets whom (i.e. who retweets which other users) and who retweets what (i.e. who retweets which specific tweets). Substantively, documentation of these variations challenges a trend in the social movement literature to assume actors within a particular movement are unified in their ideology, identity, and values.

6. Partisanship and Risk Talk on Twitter: A Multipronged Analysis of the Prominence, Targets, and Drivers of Risk-Related Expression by Democrats versus Republicans during the COVID-19 Pandemic

   Yini Zhang, Jody CS Wong, Zijian An, and 3 more authors

   Journal of Quantitative Description: Digital Media, 2024

   Abs Link to Paper

   There is abundant evidence that political polarization is associated with perceptions, beliefs, and behaviors related to the COVID-19 pandemic. This article turns to online expression and investigates the prominence, targets, and drivers of risk talk about the pandemic from Democrats and Republicans on Twitter/X, combining a user-level analysis, a content-level analysis, and manual coding of key accounts. We find that risk talk accounted for a greater share of pandemic-related discussion for Republicans compared to Democrats. Also, the specific targets of risk talk differed: Republicans focused more on risks related to public health guidelines and policies, vaccines, science, and the information environment, whereas Democrats emphasized risks associated with the coronavirus, politicians, and the economy. Furthermore, risk talk on both sides was spearheaded by news media, politicians, and activists; yet, this tendency to retweet opinion leaders was stronger among Republicans. These findings reveal nuanced partisan differences in risk-related opinion expression regarding the pandemic and point to the critical role of media and political elites within partisan groups in driving such opinion expression. Methodologically, our findings demonstrate how a multipronged analytic approach can help us identify meaningful expression patterns on social media.

7. Curated and Asymmetric Exposure: A Case Study of Partisan Talk during COVID on Twitter

   Zijian An, Jessica Breuhaus, Jason Niu, and 2 more authors

   In Proceedings of the International AAAI Conference on Web and Social Media, 2024

   Abs Link to Paper

   Social media has been at the center of discussions about political polarization in the United States. However, scholars are actively debating both the scale of political polarization online, and how important online polarization is to the offline world. One question at the center of this debate is what interactions across parties look like online, and in particular 1) whether increasing the number of such interactions is likely to increase or reduce polarization, and 2) what technological affordances may make it more likely that these cross-party interactions benefit, rather than detract from, existing political challenges. The present work aims to provide insights into the latter; that is, we focus on providing a better understanding of how a set of 400,000 partisan users on a particular social media platform, Twitter, used the platform’s affordances to interact within and across parties in a large dataset of tweets about COVID in 2021. Our findings suggest that Republican use of cross-party interaction were both more potent and potentially more strategic during COVID, that cross-party interaction was driven heavily by a small set of users and conversations, and that there exist non-obvious indirect pathways to cross-party exposure when different modes of interaction are chained together (especially retweets of quotes). These findings have implications beyond Twitter, we believe, in understanding how affordances of platforms can help to shape partisan exposure and interaction.

2023

1. Democrats are better than Republicans at discerning true and false news but do not have better metacognitive awareness

   Mitch Dobbs, Joseph DeGutis, Jorge Morales, and 2 more authors

   Communications Psychology, 2023

   Abs Link to Paper

   Insight into one’s own cognitive abilities is one important aspect of metacognition. Whether this insight varies between groups when discerning true and false information has yet to be examined. We investigated whether demographics like political partisanship and age were associated with discernment ability, metacognitive efficiency, and response bias for true and false news. Participants rated the veracity of true and false news headlines and provided confidence ratings for each judgment. We found that Democrats and older adults were better at discerning true and false news than Republicans and younger adults. However, all demographic groups maintained good insight into their discernment ability. Although Republicans were less accurate than Democrats, they slightly outperformed Democrats in metacognitive efficiency when a politically equated item set was used. These results suggest that even when individuals mistake misinformation to be true, they are aware that they might be wrong.

2. Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages

   Yingjie Hu, Gengchen Mai, Chris Cundy, and 6 more authors

   International Journal of Geographical Information Science, 2023

   Abs Link to Paper

   Social media messages posted by people during natural disasters often contain important location descriptions, such as the locations of victims. Recent research has shown that many of these location descriptions go beyond simple place names, such as city names and street names, and are difficult to extract using typical named entity recognition (NER) tools. While advanced machine learning models could be trained, they require large labeled training datasets that can be time-consuming and labor-intensive to create. In this work, we propose a method that fuses geo-knowledge of location descriptions and a Generative Pre-trained Transformer (GPT) model, such as ChatGPT and GPT-4. The result is a geo-knowledge-guided GPT model that can accurately extract location descriptions from disaster-related social media messages. Also, only 22 training examples encoding geo-knowledge are used in our method. We conduct experiments to compare this method with nine alternative approaches on a dataset of tweets from Hurricane Harvey. Our method demonstrates an over 40% improvement over typically used NER approaches. The experiment results also show that geo-knowledge is indispensable for guiding the behavior of GPT models. The extracted location descriptions can help disaster responders reach victims more quickly and may even save lives.

3. Field-specific ability beliefs as an explanation for gender differences in academics’ career trajectories: Evidence from public profiles on ORCID. Org.

   Aniko Hannak, Kenneth Joseph, Daniel B Larremore, and 1 more author

   Journal of Personality and Social Psychology, 2023

   Abs Link to Paper Code

   Academic fields exhibit substantial levels of gender segregation. Here, we investigated differences in field-specific ability beliefs (FABs) as an explanation for this phenomenon. FABs may contribute to gender segregation to the extent that they portray success as depending on “brilliance” (i.e., exceptional intellectual ability), which is a trait culturally associated with men more than women. Although prior work has documented a relation between academic fields’ FABs and their gender composition, it is still unclear what the underlying dynamics are that give rise to gender imbalances across academia as a function of FABs. To provide insight into this issue, we custom-built a new data set by combining information from the author-tracking service Open Researcher and Contributor ID (ORCID) with information from a survey of U.S. academics across 30 fields. Using this expansive longitudinal data set (Ns = 86,879–364,355), we found that women were underrepresented among those who enter fields with brilliance-oriented FABs and overrepresented among those who exit these fields. We also found that FABs’ association with women’s transitions across academic fields was substantially stronger than their association with men’s transitions. With respect to mechanisms, FABs’ association with gender segregation was partially explained by the fact that women encounter more prejudice in fields with brilliance-oriented FABs. With its focus on the dynamic patterns shaping segregation and its broad scope in terms of geography, career stage, and historical time, this research makes an important contribution toward understanding the factors driving gender segregation in academia.

4. A Systematic Review of Sophisticated Predictive and Prescriptive Analytics in Child Welfare: Accuracy, Equity, and Bias

   Seventy F Hall, Melanie Sage, Carol F Scott, and 1 more author

   Child and Adolescent Social Work Journal, 2023

   Abs Link to Paper

   Child welfare agencies increasingly use machine learning models to predict outcomes and inform decisions. These tools are intended to increase accuracy and fairness but can also amplify bias. This systematic review explores how researchers addressed ethics, equity, bias, and model performance in their design and evaluation of predictive and prescriptive algorithms in child welfare. We searched EBSCO databases, Google Scholar, and reference lists for journal articles, conference papers, dissertations, and book chapters published between January 2010 and March 2020. Sources must have reported on the use of algorithms to predict child welfare-related outcomes and either suggested prescriptive responses, or applied their models to decision-making contexts. We calculated descriptive statistics and conducted Mann-Whitney U tests, and Spearman’s rank correlations to summarize and synthesize findings. Of 15 articles, fewer than half considered ethics, equity, or bias or engaged participatory design principles as part of model development/evaluation. Only one-third involved cross-disciplinary teams. Model performance was positively associated with number of algorithms tested and sample size. No other statistical tests were significant. Interest in algorithmic decision-making in child welfare is growing, yet there remains no gold standard for ameliorating bias, inequity, and other ethics concerns. Our review demonstrates that these efforts are not being reported consistently in the literature and that a uniform reporting protocol may be needed to guide research. In the meantime, computer scientists might collaborate with content experts and stakeholders to ensure they account for the practical implications of using algorithms in child welfare settings.

5. The Context of Care: Abstraction Hierarchy Modeling of Therapeutic Foster Care Programs

   Connor Wurst, Huei-Yen Winnie Chen, Ann M Bisantz, and 4 more authors

   Journal of Cognitive Engineering and Decision Making, 2023

   Abs Link to Paper

   Therapeutic foster care agencies provide temporary placements and a range of services to at-risk youth to help ensure their safety, permanency, and wellbeing. The practitioners that plan such care operate under heavy caseloads, limited resources, and high stakes. There is significant interest in supporting these practitioners with various technological interventions, but their work and the context around it is still poorly understood. This study aims to better understand the current assessment and treatment planning work in therapeutic foster care. We used the abstraction hierarchy modeling approach to outline the purposes, values, constraints, processes, and tools that define the workplace ecology encountered by care coordinators and clinicians from therapeutic foster care programs at Hillside, a collaborating human service organization. The resulting abstraction hierarchy was closely examined to identify areas for interventions and design implications.

6. Strategic Behavior in Two-sided Matching Markets with Prediction-enhanced Preference-formation

   Stefania Ionescu, Yuhao Du, Kenneth Joseph, and 1 more author

   In Thirty-seventh Conference on Neural Information Processing Systems, 2023

   Abs Link to Paper Code

   Two-sided matching markets have long existed to pair agents in the absence of regulated exchanges. A common example is school choice, where a matching mechanism uses student and school preferences to assign students to schools. In such settings, forming preferences is both difficult and critical. Prior work has suggested various prediction mechanisms that help agents make decisions about their preferences. Although often deployed together, these matching and prediction mechanisms are almost always analyzed separately. The present work shows that at the intersection of the two lies a previously unexplored type of strategic behavior: agents returning to the market (e.g., schools) can attack future predictions by interacting short-term non-optimally with their matches. Here, we first introduce this type of strategic behavior, which we call an adversarial interaction attack. Next, we construct a formal economic model that captures the feedback loop between prediction mechanisms designed to assist agents and the matching mechanism used to pair them. Finally, in a simplified setting, we prove that returning agents can benefit from using adversarial interaction attacks and gain progressively more as the trust in and accuracy of predictions increases. We also show that this attack increases inequality in the student population.

7. Just another day on Twitter: a complete 24 hours of Twitter data

   Juergen Pfeffer, Daniel Matter, Kokil Jaidka, and 8 more authors

   In Proceedings of the International AAAI Conference on Web and Social Media, 2023

   Abs Link to Paper

   At the end of October 2022, Elon Musk concluded his acquisition of Twitter. In the weeks and months before that, several questions were publicly discussed that were not only of interest to the platform’s future buyers, but also of high relevance to the Computational Social Science research community. For example, how many active users does the platform have? What percentage of accounts on the site are bots? And, what are the dominating topics and sub-topical spheres on the platform? In a globally coordinated effort of 80 scholars to shed light on these questions, and to offer a dataset that will equip other researchers to do the same, we have collected all 375 million tweets published within a 24-hour time period starting on September 21, 2022. To the best of our knowledge, this is the first complete 24-hour Twitter dataset that is available for the research community. With it, the present work aims to accomplish two goals. First, we seek to answer the aforementioned questions and provide descriptive metrics about Twitter that can serve as references for other researchers. Second, we create a baseline dataset for future research that can be used to study the potential impact of the platform’s ownership change.

2022

1. NELA-Local: A Dataset of US Local News Articles for the Study of County-Level News Ecosystems

   Benjamin D Horne, Maurício Gruppi, Kenneth Joseph, and 3 more authors

   In Proceedings of the International AAAI Conference on Web and Social Media, 2022

   Abs

   Despite the influence that image-based communication has on online discourse, the role played by images in disinformation is still not well understood. In this paper, we present the first large-scale study of fauxtography, analyzing the use of manipulated or misleading images in news discussion on online communities. First, we develop a computational pipeline geared to detect fauxtography, and identify over 61k instances of fauxtography discussed on Twitter, 4chan, and Reddit. Then, we study how posting fauxtography affects engagement of posts on social media, finding that posts containing it receive more interactions in the form of re-shares, likes, and comments. Finally, we show that fauxtography images are often turned into memes by Web communities. Our findings show that effective mitigation against disinformation need to take images into account, and highlight a number of challenges in dealing with image-based disinformation

2. A Data-Driven Simulation of the New York State Foster Care System

   Yuhao Du, Stefania Ionescu, Melanie Sage, and 1 more author

   In 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022

   Abs Link to Paper

   We introduce an analytic pipeline to model and simulate youth trajectories through the New York state foster care system. Our goal in doing so is to forecast how proposed interventions may impact the foster care system’s ability to achieve it’s stated goals before these interventions are actually implemented and impact the lives of thousands of youth. Here, we focus on two specific stated goals of the system: racial equity, and, as codified most recently by the 2018 Family First Prevention Services Act (FFPSA), a focus on keeping all youth out of foster care. We also focus on one specific potential intervention— a predictive model, proposed in prior work and implemented elsewhere in the U.S., which aims to determine whether or not a youth is in need of care. We use our method to explore how the implementation of this predictive model in New York would impact racial equity and the number of youth in care. While our findings, as in any simulation model, ultimately rely on modeling assumptions, we find evidence that the model would not necessarily achieve either goal. Primarily, then, we aim to further promote the use of data-driven simulation to help understand the ramifications of algorithmic interventions in public systems.

3. Insidious Nonetheless: How Small Effects and Hierarchical Norms Create and Maintain Gender Disparities in Organizations

   Yuhao Du, Jessica Nordell, and Kenneth Joseph

   Socius : sociological research for a dynamic world, 2022

   Abs Link to Paper Code

   The term glass ceiling is applied to the well-established phenomenon in which women and people of color are consistently blocked from reaching the uppermost levels of the corporate hierarchy. Focusing on gender, we present an agent-based model that explores how empirically established mechanisms of interpersonal discrimination coevolve with social norms at both the organizational (meso) and societal (macro) levels to produce this glass ceiling effect for women. Our model extends the understanding of how the glass ceiling arises and why it can be resistant to change. We do so by synthesizing existing psychological and structural theories of discrimination into a mathematical model that quantifies explicitly how complex organizational systems can produce and maintain inequality. We discuss implications of our findings for both intervention and future empirical analyses and provide open-source code for those wishing to adapt or extend our work.

4. A Qualitative, Network-Centric Method for Modeling Socio-Technical Systems, with Applications to Evaluating Interventions on Social Media Platforms to Increase Social Equality

   Kenneth Joseph, Huei-Yen Winnie Chen, Stefania Ionescu, and 4 more authors

   Applied Network Science, 2022

   Abs Link to Paper

   We propose and extend a qualitative, complex systems methodology from cognitive engineering, known as the abstraction hierarchy, to model how potential interventions that could be carried out by social media platforms might impact social equality. Social media platforms have come under considerable ire for their role in perpetuating social inequality. However, there is also significant evidence that platforms can play a role in reducing social inequality, e.g. through the promotion of social movements. Platforms’ role in producing or reducing social inequality is, moreover, not static; platforms can and often do take actions targeted at positive change. How can we develop tools to help us determine whether or not a potential platform change might actually work to increase social equality? Here, we present the abstraction hierarchy as a tool to help answer this question. Our primary contributions are two-fold. First, methodologically, we extend existing research on the abstraction hierarchy in cognitive engineering with principles from Network Science. Second, substantively, we illustrate the utility of this approach by using it to assess the potential effectiveness of a set of interventions, proposed in prior work, for how online dating websites can help mitigate social inequality.

5. Critical Decision Method Interviews to Understand the Initial Treatment Planning Process in Foster Care

   Connor Wurst, Huei-Yen Winnie Chen, Melanie Sage, and 2 more authors

   Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Sep 2022

   Abs Link to Paper

   In foster care settings, the treatment plan captures goals and interventions for youth in care. The first version of this plan is typically due 30 days after the youth is enrolled in the foster care program, leading to a challenging month of assessing the case and developing the treatment plan. This study utilized Critical Decision Method interviews with care coordinators and clinicians to understand the decision-making involved in balancing assessment tasks, and the barriers to using assessment to inform treatment. The interviews were coded to identify major themes including information sources and constraints. These identified themes and general understanding of the problem space will drive future work developing interventions to improve the workflow process and drive better outcomes for youth in foster care.

6. Local News Online and COVID in the US: Relationships among Coverage, Cases, Deaths, and Audience

   Kenneth Joseph, Benjamin D Horne, Jon Green, and 1 more author

   In Proceedings of the International AAAI Conference on Web and Social Media, Sep 2022

   Abs Link to Paper Code

   We present analyses from a real-time information monitoring system of online local news in the U.S. We study relationships among online local news coverage of COVID, cases and deaths in an area, and properties of local news outlets and their audiences. Our analysis relies on a unique dataset of the online content of over 300 local news outlets, encompassing over 750,000 articles over a period of 10 months spanning April 2020 to February 2021. We find that the rate of COVID coverage over time by local news outlets was primarily associated with death rates at the national level, but that this effect dissipated over the course of the pandemic as news about COVID was steadily displaced by sociopolitical events, like the 2020 U.S. elections. We also find that both the volume and content of COVID coverage differed depending on local politics, and outlet audience size, as well as evidence that more vulnerable populations received less pandemic-related news.

7. Mutual Information Scoring: Increasing Interpretability in Categorical Clustering Tasks with Applications to Child Welfare Data

   Pranav Sankhe, Seventy F Hall, Melanie Sage, and 3 more authors

   In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, Sep 2022

   Abs Link to Paper Code

   Youth in the American foster care system are significantly more likely than their peers to face a number of negative life outcomes, from homelessness to incarceration. Administrative data on these youth have the potential to provide insights that can help identify ways to improve their path towards a better life. However, such data also suffer from a variety of biases, from missing data to reflections of systemic inequality. The present work proposes a novel, prescriptive approach to using these data to provide insights about both data biases and the systems and youth they track. Specifically, we develop a novel categorical clustering and cluster summarization methodology that allows us to gain insights into subtle biases in existing data on foster youth, and to provide insight into where further (often qualitative) research is needed to identify potential ways of assisting youth.

2021

1. (Mis) Alignment between Stance Expressed in Social Media Data and Public Opinion Surveys

   Kenneth Joseph, Sarah Shugars, Ryan Gallagher, and 4 more authors

   In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Sep 2021

   Abs Link to Paper

   Stance detection, which aims to determine whether an individual is for or against a target concept, promises to uncover public opinion from large streams of social media data. Yet even human annotation of social media content does not always capture "stance" as measured by public opinion polls. We demonstrate this by directly comparing an individual’s self-reported stance to the stance inferred from their social media data. Leveraging a longitudinal public opinion survey with respondent Twitter handles, we conducted this comparison for 1,129 individuals across four salient targets. We find that recall is high for both "Pro" and "Anti" stance classifications but precision is variable in a number of cases. We identify three factors leading to the disconnect between text and author stance: temporal inconsistencies, differences in constructs, and measurement errors from both survey respondents and annotators. By presenting a framework for assessing the limitations of stance detection models, this work provides important insight into what stance detection truly measures.

2. Pandemics, Protests, and Publics: Demographic Activity and Engagement on Twitter in 2020

   Sarah Shugars, Adina Gitomer, Stefan McCabe, and 6 more authors

   Journal of Quantitative Description: Digital Media, Sep 2021

   Abs Link to Paper

   As an integral component of public discourse, Twitter is among the main data sources for scholarship in this area. However, there is much that scholars do not know about the basic mechanisms of public discourse on Twitter, including the prevalence of various modes of communication, the types of posts users make, the engagement those posts receive, or how these things vary with user demographics and across different topical events. This paper broadens our understanding of these aspects of public discourse. We focus on the first nine months of 2020, studying that period as a whole and giving particular attention to two monumentally important topics of that time: the Black Lives Matter movement and the COVID-19 pandemic. Leveraging a panel of 1.6 million Twitter accounts matched to U.S. voting records, we examine the demographics, activity, and engagement of 800,000 American adults who collectively posted nearly 300 million tweets during this time span. We find notable variation in user activity and engagement, in terms of modality (e.g., retweets vs. replies), demographic subgroup, and topical context. We further find that while Twitter can best be understood as a collection of interconnected publics, neither topical nor demographic variation perfectly encapsulates the "Twitter public." Rather, Twitter publics are fluid, contextual communities which form around salient topics and are informed by demographic identities. Together, this paper presents a disaggregated, multifaceted description of the demographics, activity, and engagement of American Twitter users in 2020.

3. Sizing up “Media Twitter”: Exploring Population Extent, Beats, and Utility of Social Media

   J Wihbey, Kenneth Joseph, and Daniela Rincon Reyes

   In ICADAT’21, Sep 2021

   Abs Link to Paper

   This mixed-methods study explores which types of journalists are on social media and what forms of knowledge-related utility they find there for their practice. We leverage computational techniques to identify more than 166,000 journalist profiles, in English, on Twitter and to examine their beats, follower counts, and volume of activity. We pair this with findings from an original 2019 survey with policy-oriented journalists (N = 450) who work on a variety of beats. Two-thirds of journalists believe social media tools help them frequently in their reporting work across many dimensions. Regression analysis finds significant associations with the technology and international affairs beats, as well as among younger journalists and those with a national audience. Our Twitter analysis, based on a dataset of 2.5 billion tweets collected in mid-2020, finds that the beats of politics, international affairs, and technology see the highest relative number of journalists on Twitter. The findings furnish a descriptive, quantitative picture of “Media Twitter” and speak to questions about social media’s place in journalism.

4. Formative Modeling of Foster Care Work: A Cognitive Work Analysis Approach

   Connor Wurst, Huei-Yen Winnie Chen, and Kenneth Joseph

   In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Sep 2021

   Abs Link to Paper

   In this paper we present the promise of the Cognitive Work Analysis (CWA) methodology, particularly abstraction hierarchy modeling, in the foster care domain. There is increasing interest in applying machine learning decision aids to foster care decision making, but that interest is accompanied by concerns that those aids may perpetuate systemic bias or be largely context-blind. Modeling the work conducted at different levels of the domain offers unique insights into where bias may enter the system as well as possible design implications for these future decision aids. This project models two major areas of work in the domain, management of individual cases and management of overall programs offered. These work areas are then considered in the first 3 levels of the abstraction hierarchy to display the promise that this model can hold for the domain in future work, particularly when supported with more naturalistic studies.

5. An Agent-Based Model to Evaluate Interventions on Online Dating Platforms to Decrease Racial Homogamy

   Stefania Ionescu, Aniko Hannak, and Kenneth Joseph

   Proceedings of the 2021 ACM conference on fairness, accountability, and transparency., Sep 2021

   Abs Link to Paper

   Perhaps the most controversial questions in the study of online platforms today surround the extent to which platforms can intervene to reduce the societal ills perpetrated on them. Up for debate is whether there exist any effective and lasting interventions a platform can adopt to address, e.g., online bullying, or if other, more far-reaching change is necessary to address such problems. Empirical work is critical to addressing such questions. But it is also challenging, because it is time-consuming, expensive, and sometimes limited to the questions companies are willing to ask. To help focus and inform this empirical work, we here propose an agent-based modeling (ABM) approach. As an application, we analyze the impact of a set of interventions on a simulated online dating platform on the lack of long-term interracial relationships in an artificial society. In the real world, a lack of interracial relationships are a critical vehicle through which inequality is maintained. Our work shows that many previously hypothesized interventions online dating platforms could take to increase the number of interracial relationships from their website have limited effects, and that the effectiveness of any intervention is subject to assumptions about sociocultural structure. Further, interventions that are effective in increasing diversity in long-term relationships are at odds with platforms’ profit-oriented goals. At a general level, the present work shows the value of using an ABM approach to help understand the potential effects and side effects of different interventions that a platform could take.

6. Who Gets What, According to Whom? An Analysis of Fairness Perceptions in Service Allocation

   Jacqueline Hannan, Huei-Yen Winnie Chen, and Kenneth Joseph

   In AIES’21, Sep 2021

   Abs Link to Paper Code

   Algorithmic fairness research has traditionally been linked to the disciplines of philosophy, ethics, and economics, where notions of fairness are prescriptive and seek objectivity. Increasingly, however, scholars are turning to the study of what different people perceive to be fair, and how these perceptions can or should help to shape the design of machine learning, particularly in the policy realm. The present work experimentally explores five novel research questions at the intersection of the "Who," "What," and "How" of fairness perceptions. Specifically, we present the results of a multi-factor conjoint analysis study that quantifies the effects of the specific context in which a question is asked, the framing of the given question, and who is answering it. Our results broadly suggest that the "Who" and "What," at least, matter in ways that are 1) not easily explained by any one theoretical perspective, 2) have critical implications for how perceptions of fairness should be measured and/or integrated into algorithmic decision-making systems.

7. An Analysis of Replies to Trump’s Tweets

   Zijian An, and Kenneth Joseph

   In Proceedings of the International AAAI Conference on Web and Social Media, Sep 2021

   Abs Link to Paper Code

   Donald Trump has tweeted thousands of times during his presidency. These public statements are an increasingly important way through which Trump communicates his political and personal views. A better understanding of the way the American public consumes and responds to these tweets is therefore critical. In the present work, we address both consumption of and response to Trump’s tweets by studying replies to them on Twitter. With respect to response, we find that a small number of older, white, left-leaning, and female Americans are responsible for the vast majority of replies to Trump’s tweets. These individuals also attend to a broader range of Trump’s tweets than the rest of the individuals we study. With respect to consumption, we note that Trump’s tweets are often viewed not in isolation, but rather in the context of a set of algorithmically-curated replies. These replies may therefore color the way Americans consume Trump’s tweets. To this end, we find some evidence that Twitter accounts see replies in line with their political leanings. However, we show that this can be largely, although not entirely, attributed to the fact that Twitter is more likely to show replies by accounts a user follows. As a basis for comparison, all results for Trump are compared and contrasted with replies to Joe Biden’s tweets.

8. Friend or Foe: A Review and Synthesis of Computational Models of the Identity Labeling Problem

   Kenneth Joseph, and Jonathan Howard Morgan

   The Journal of Mathematical Sociology, Sep 2021

   Abs Link to Paper Code

   We introduce the identity labeling problem – given an individual in a social situation, can we predict what identity(ies) they will be labeled with by someone else? This problem remains a theoretical gap and methodological challenge, evidenced by the fact that models of social-cognition often sidestep the issue by treating identities as already known. We build on insights from existing models to develop a new framework, entitled Latent Cognitive Social Spaces, that can incorporate multiple social cues including sentiment information, socio-demographic characteristics, and institutional associations to estimate the most culturally expected identity. We apply our model to data collected in two vignette experiments, finding that it predicts identity labeling choices of participants with a mean absolute error of 10.9%, a 100% improvement over previous models based on parallel constraint satisfaction and affect control theory.

9. A Method to Analyze Multiple Social Identities in Twitter Bios

   Arjunil Pathak, Navid Madani, and Kenneth Joseph

   Proceedings of the ACM on Human-Computer Interaction, Sep 2021

   Abs Link to Paper Code

   Twitter users signal social identity in their profile descriptions, or bios, in a number of important but complex ways that are not well-captured by existing characterizations of how identity is expressed in language. Better ways of defining and measuring these expressions may therefore be useful both in understanding how social identity is expressed in text, and how the self is presented on Twitter. To this end, the present work makes three contributions. First, using qualitative methods, we identify and define the concept of a personal identifier, which is more representative of the ways in which identity is signaled in Twitter bios. Second, we propose a method to extract all personal identifiers expressed in a given bio. Finally, we present a series of validation analyses that explore the strengths and limitations of our proposed method. Our work opens up exciting new opportunities at the intersection between the social psychological study of social identity and the study of how we compose the self through markers of identity on Twitter and in social media more generally.

2020

1. NeuroTPR: A Neuro-Net Toponym Recognition Model for Extracting Locations from Social Media Messages

   Jimin Wang, Yingjie Hu, and Kenneth Joseph

   Transactions in GIS, Sep 2020

   Abs Link to Paper

   Social media messages, such as tweets, are frequently used by people during natural disasters to share real-time information and to report incidents. Within these messages, geographic locations are often described. Accurate recognition and geolocation of these locations are critical for reaching those in need. This article focuses on the first part of this process, namely recognizing locations from social media messages. While general named entity recognition tools are often used to recognize locations, their performance is limited due to the various language irregularities associated with social media text, such as informal sentence structures, inconsistent letter cases, name abbreviations, and misspellings. We present NeuroTPR, which is a Neuro-net ToPonym Recognition model designed specifically with these linguistic irregularities in mind. Our approach extends a general bidirectional recurrent neural network model with a number of features designed to address the task of location recognition in social media messages. We also propose an automatic workflow for generating annotated data sets from Wikipedia articles for training toponym recognition models. We demonstrate NeuroTPR by applying it to three test data sets, including a Twitter data set from Hurricane Harvey, and comparing its performance with those of six baseline models.

2. MDR Cluster-Debias: A Nonlinear Word Embedding Debiasing Pipeline

   Yuhao Du, and Kenneth Joseph

   In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, Sep 2020

   Abs Link to Paper

   Existing methods for debiasing word embeddings often do so only superficially, in that words that are stereotypically associated with, eg, a particular gender in the original embedding space can still be clustered together in the debiased space. However, there has yet to be a study that explores why this residual clustering exists, and how it might be addressed. The present work fills this gap. We identify two potential reasons for which residual bias exists and develop a new pipeline, MDR Cluster-Debias, to mitigate this bias. We explore the strengths and weaknesses of our method, finding that it significantly outperforms other existing debiasing approaches on a variety of upstream bias tests but achieves limited improvement on decreasing gender bias in a downstream task. This indicates that word embeddings encode gender bias in still other ways, not necessarily captured by upstream tests.

3. Understanding Visual Memes: An Empirical Analysis of Text Superimposed on Memes Shared on Twitter

   Yuhao Du, Muhammad Aamir Masood, and Kenneth Joseph

   In Proceedings of the International AAAI Conference on Web and Social Media, Sep 2020

   Abs Link to Paper

   Visual memes have become an important mechanism through which ideologically potent and hateful content spreads on today’s social media platforms. At the same time, they are also a mechanism through which we convey much more mundane things, like pictures of cats with strange accents. Little is known, however, about the relative percentage of visual memes shared by real people that fall into these, or other, thematic categories. The present work focuses on visual memes that contain superimposed text. We carry out the first large-scale study on the themes contained in the text of these memes, which we refer to as image-with-text memes. We find that 30% of the image-with-text memes in our sample which have identifiable themes are politically relevant, and that these politically relevant memes are shared more often by Democrats than Republicans. We also find disparities in who expresses themselves via image-with-text memes, and images in general, versus other forms of expression on Twitter. The fact that some individuals use images with text to express themselves, instead of sending a plain text tweet, suggests potential consequences for the representativeness of analyses that ignore text contained in images.

4. Who Says What with Whom: Using Bi-Spectral Clustering to Organize and Analyze Social Media Protest Networks

   Kenneth Joseph, Ryan J Gallagher, and Brooke Foucault Welles

   Computational Communication Research, Sep 2020

   Abs Link to Paper Code

   Social media protest networks involve many participants, from long-time activists to individuals who are engaged only in a particular protest event. We propose a new approach to studying how various communities of users participate in protest events. Our approach combines two methodological innovations. First, rather than study tweets central to one event, we collect full timelines of user activity leading up to participation in a focal event. Second, we propose bi-spectral clustering as a scalable computational method for rapidly identifying communities of users by the words (hashtags) they use. Using a large sample of tweets from users who discussed the 2016 protests in Charlotte, North Carolina following the extrajudicial killing of Keith Lamont Scott as a case study, we demonstrate how bi-spectral clustering can be applied to sort, sample, and identify ideologically and thematically coherent clusters whose members participated in the protest on Twitter. Our proposed approach provides another tool in the mixed methods scholar’s toolkit to computationally sort and cluster large-scale network data by allowing researchers to look beyond focal hashtags or keywords and situate protest messages within the broader communication context of participating users

5. When Do Word Embeddings Accurately Reflect Surveys on Our Beliefs about People?

   Kenneth Joseph, and Jonathan H Morgan

   In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Sep 2020

   Abs Link to Paper Code

   Social biases are encoded in word embeddings. This presents a unique opportunity to study society historically and at scale, and a unique danger when embeddings are used in downstream applications. Here, we investigate the extent to which publicly-available word embeddings accurately reflect beliefs about certain kinds of people as measured via traditional survey methods. We find that biases found in word embeddings do, on average, closely mirror survey data across seventeen dimensions of social meaning. However, we also find that biases in embeddings are much more reflective of survey data for some dimensions of meaning (e.g. gender) than others (e.g. race), and that we can be highly confident that embedding-based measures reflect survey data only for the most salient biases.

6. Theory In, Theory Out: The Uses of Social Theory in Machine Learning for Social Science

   Jason Radford, and Kenneth Joseph

   Frontiers in Big Data, Sep 2020

   Abs Link to Paper

   Research at the intersection of machine learning and the social sciences has provided critical new insights into social behavior. At the same time, a variety of issues have been identified with the machine learning models used to analyze social data. These issues range from technical problems with the data used and features constructed, to problematic modeling assumptions, to limited interpretability, to the models’ contributions to bias and inequality. Computational researchers have sought out technical solutions to these problems. The primary contribution of the present work is to argue that there is a limit to these technical solutions. At this limit, we must instead turn to social theory. We show how social theory can be used to answer basic methodological and interpretive questions that technical solutions cannot when building machine learning models, and when assessing, comparing, and using those models. In both cases, we draw on related existing critiques, provide examples of how social theory has already been used constructively in existing work, and discuss where other existing work may have benefited from the use of specific social theories. We believe this paper can act as a guide for computer and social scientists alike to navigate the substantive questions involved in applying the tools of machine learning to social data.

2019

1. Fake News on Twitter during the 2016 US Presidential Election

   Nir Grinberg, Kenneth Joseph, Lisa Friedland, and 2 more authors

   Science, Sep 2019

   Abs Link to Paper Code

   The spread of fake news on social media became a public concern in the United States after the 2016 presidential election. We examined exposure to and sharing of fake news by registered voters on Twitter and found that engagement with fake news sources was extremely concentrated. Only 1% of individuals accounted for 80% of fake news source exposures, and 0.1% accounted for nearly 80% of fake news sources shared. Individuals most likely to engage with fake news sources were conservative leaning, older, and highly engaged with political news. A cluster of fake news sources shared overlapping audiences on the extreme right, but for people across the political spectrum, most political news exposure still came from mainstream media outlets.

2. The Social Silos of Journalism? Twitter, News Media and Partisan Segregation

   John Wihbey, Kenneth Joseph, and David Lazer

   New Media & Society, Sep 2019

   Abs Link to Paper Code

   The present work proposes social media as a tool to understand the relationship between journalists’ social networks and the content they produce. Specifically, we ask, “what is the association between the partisan nature of the accounts journalists follow on Twitter and the news content they produce?” Using standard text scaling techniques, we analyze partisanship in a novel dataset of more than 300,000 news articles produced by 644 journalists at 25 different US news outlets. We then develop a novel, semi-supervised model of partisanship of Twitter following relationships and show a modest correlation between the partisanship of whom a journalist follows on Twitter and the content she produces. The findings provide insight into the partisan dynamics that appear to characterize the US media ecosystem in its broad contours, dynamics that may be traceable from social media networks to published stories.

3. Breaking News and Younger Twitter Users: Comparing Self-Reported Motivations to Online Behavior

   Kenneth Joseph, and John Wihbey

   In Proceedings of the 10th International Conference on Social Media and Society, Sep 2019

   Abs Link to Paper

   Breaking news represents an important avenue of information about current events, including life-or-death information during natural disasters, but it can also serve as a vector along which misinformation can rapidly spread. In the present work we study factors associated with the sharing of breaking news by young, college-aged students. Using a unique combination of survey and behavioral data, we identify traits associated with a propensity to share breaking news on Twitter.We find that individuals who share more breaking news report high levels of confidence in their ability to differentiate real from fake news and to manage information overload. However, breaking news sharing is not associated with the reported use of traditional fact-checking strategies (e.g., finding other sources for the same information) before sharing. Thus, our data are consistent with the idea that breaking news sharers tend to rely, at least in part, on confidence in their own understanding of the news when determining what breaking news to share. We contextualize these findings by studying patterns of general news sharing and non-news sharing, as well as studying connections between party affiliation and factors associated with the sharing of content.

4. Polarized, Together: Comparing Partisan Support for Trump’s Tweets Using Survey and Platform-Based Measures

   Kenneth Joseph, Briony Swire-Thompson, Hannah Masuga, and 2 more authors

   In Proceedings of the International AAAI Conference on Web and Social Media, Sep 2019

   Abs Link to Paper Code

   Using both survey- and platform-based measures of support, we study how polarization manifests for 4,313 of President Donald Trump’s tweets since he was inaugurated in 2017. We find high levels of polarization in response to Trump’s tweets. However, after controlling for mean differences, we surprisingly find a high degree of agreement across partisan lines across both survey and platform-based measures. This suggests that Republicans and Democrats, while disagreeing on an absolute level, tend to agree on the relative quality of Trump’s tweets. We assess potential reasons for this, for example, by studying how support changes in response to tweets containing positive versus negative language. We also explore how Democrats and Republicans respond to tweets containing insults of individuals with particular socio-demographics, finding that Republican support decreases when Republicans, relative to Democrats, are insulted, and Democrats respond negatively to insults of women and members of the media

2018

1. Auditing Partisan Audience Bias within Google Search

   Ronald E Robertson, Shan Jiang, Kenneth Joseph, and 3 more authors

   Proceedings of the ACM on Human-Computer Interaction, Sep 2018

   Abs Link to Paper

   There is a growing consensus that online platforms have a systematic influence on the democratic process. However, research beyond social media is limited. In this paper, we report the results of a mixed-methods algorithm audit of partisan audience bias and personalization within Google Search. Following Donald Trump’s inauguration, we recruited 187 participants to complete a survey and install a browser extension that enabled us to collect Search Engine Results Pages (SERPs) from their computers. To quantify partisan audience bias, we developed a domain-level score by leveraging the sharing propensities of registered voters on a large Twitter panel. We found little evidence for the "filter bubble” hypothesis. Instead, we found that results positioned toward the bottom of Google SERPs were more left-leaning than results positioned toward the top, and that the direction and magnitude of overall lean varied by search query, component type (e.g. "answer boxes"), and other factors. Utilizing rank-weighted metrics that we adapted from prior work, we also found that Google’s rankings shifted the average lean of SERPs to the right of their unweighted average.

2. Early Intervention Systems: Predicting Adverse Interactions between Police and the Public

   Jennifer Helsby, Samuel Carton, Kenneth Joseph, and 8 more authors

   Criminal justice policy review, Sep 2018

   Abs Link to Paper

   Adverse interactions between police and the public hurt police legitimacy, cause harm to both officers and the public, and result in costly litigation. Early intervention systems (EISs) that flag officers considered most likely to be involved in one of these adverse events are an important tool for police supervision and for targeting interventions such as counseling or training. However, the EISs that exist are not data-driven and based on supervisor intuition. We have developed a data-driven EIS that uses a diverse set of data sources from the Charlotte-Mecklenburg Police Department and machine learning techniques to more accurately predict the officers who will have an adverse event. Our approach is able to significantly improve accuracy compared with their existing EIS: Preliminary results indicate a 20% reduction in false positives and a 75% increase in true positives.

3. Mining Online Communities to Inform Strategic Messaging: Practical Methods to Identify Community-Level Insights

   Matthew Benigni, Kenneth Joseph, and Kathleen M Carley

   Computational and Mathematical Organization Theory, Sep 2018
4. The Power of Social Cognition

   Geoffrey P Morgan, Kenneth Joseph, and Kathleen M Carley

   Journal of Social Structure, Sep 2018

   Abs Link to Paper

   As human beings, we understand and make sense of the social world using social cognition. Social cognitions are cognitive processes through which we understand, process, and recall our interactions with others. Most agent-based models do not account for social cognition; rather, they either provide detailed models of task-related cognition or model many actors and focus on social processes. In general, the more cognitively realistic the models, the less they explain human social behavior and the more computationally expensive it is to model a single agent. In contrast, in this research an agent-based model containing an explicit model of social cognition is developed. Results from this model demonstrate that adding social cognition both improves the model veridicality and decreases computation costs.

2017

1. Exploring the Ideological Nature of Journalists’ Social Networks on Twitter and Associations with News Story Content

   John Wihbey, Thalita Dias Coleman, Kenneth Joseph, and 1 more author

   In DS+J Workshop at KDD’18, Sep 2017

   Abs Link to Paper

   The present work proposes the use of social media as a tool for better understanding the relationship between a journalists’ social network and the content they produce. Specifically, we ask: what is the relationship between the ideological leaning of a journalist’s social network on Twitter and the news content he or she produces? Using a novel dataset linking over 500,000 news articles produced by 1,000 journalists at 25 different news outlets, we show a modest correlation between the ideologies of who a journalist follows on Twitter and the content he or she produces. This research can provide the basis for greater self-reflection among media members about how they source their stories and how their own practice may be colored by their online networks. For researchers, the findings furnish a novel and important step in better understanding the construction of media stories and the mechanics of how ideology can play a role in shaping public information.

2. " Voters of the Year": 19 Voters Who Were Unintentional Election Poll Sensors on Twitter

   William Hobbs, Lisa Friedland, Kenneth Joseph, and 3 more authors

   In Proceedings of the International AAAI Conference on Web and Social Media, Sep 2017

   Abs Link to Paper

   Public opinion and election prediction models based on social media typically aggregate, weight, and average signals from a massive number of users. Here, we analyze political attention and poll movements to identify a small number of social "sensors" — individuals whose levels of social media discussion of the major parties’ candidates characterized the candidates’ ups and downs over the 2016 U.S. presidential election campaign. Starting with a sample of approximately 22,000 accounts on Twitter that we linked to voter registration records, we used penalized regressions to identify a set of 19 accounts (sensors) that were predictive for the candidates’ poll numbers (5 for Hillary Clinton, 13 for Donald Trump, and 1 for both). The predictions based on the activity of these handfuls of sensors accurately tracked later movements in poll margins. Despite the regressions allowing both supportive and opposition sensors, our separate models for Trump and Clinton poll support identified sensors for Hillary Clinton who were disproportionately women and for Donald Trump who were disproportionately white. The method did not predict changes in levels of undecideds and underestimated support for Donald Trump in September 2016, where the errors were correlated with discussions of protests of police shootings.

3. Online Extremism and the Communities That Sustain It: Detecting the ISIS Supporting Community on Twitter

   Matthew C Benigni, Kenneth Joseph, and Kathleen M Carley

   PloS one, Sep 2017
4. Scalable Computational Techniques for Centrality Metrics on Temporally Detailed Social Network

   Venkata MV Gunturi, Shashi Shekhar, Kenneth Joseph, and 1 more author

   Machine Learning, Sep 2017

   Abs Link to Paper

   Increasing proliferation of mobile and online social networking platforms have given us unprecedented opportunity to observe and study social interactions at a fine temporal scale. A collection of all such social interactions among a group of individuals (or agents) observed over an interval of time is referred to as a temporally-detailed (TD) social network. A TD social network opens up the opportunity to explore TD questions on the underlying social system, e.g., “How is the betweenness centrality of an individual changing with time?” To this end, related work has proposed temporal extensions of centrality metrics (e.g., betweenness and closeness). However, scalable computation of these metrics for long time-intervals is challenging. This is due to the non-stationary ranking of shortest paths (the underlying structure of betweenness and closeness) between a pair of nodes which violates the assumptions of classical dynamic programming based techniques. To this end, we propose a novel computational paradigm called epoch-point based techniques for addressing the non-stationarity challenge of TD social networks. Using the concept of epoch-points, we develop a novel algorithm for computing shortest path based centrality metric such as betweenness on a TD social network. We prove the correctness and completeness of our algorithm. Our experimental analysis shows that the proposed algorithm out performs the alternatives by a wide margin.

5. ConStance: Modeling Annotation Contexts to Improve Stance Classification

   Kenneth Joseph, Lisa Friedland, William Hobbs, and 2 more authors

   In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Sep 2017

   Code
6. Girls Rule, Boys Drool: Extracting Semantic and Affective Stereotypes from Twitter

   Kenneth Joseph, Wei Wei, and Kathleen M Carley

   In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, Sep 2017

   Abs Link to Paper Code

   Social identities carry widely agreed upon meanings, called stereotypes, that have important effects on social processes. In the present work, we develop a method to extract the stereotypes of Twitter users. Our method is grounded in two distinct strands of theory, one that represents stereotypes as identities’ affective meanings and the other that represents stereotypes as semantic relationships between identities. After validating our approach via a prediction task, we apply the model to a dataset of 45 thousand Twitter users who actively tweeted about the Michael Brown and Eric Garner tragedies. Our work provides unique insights into the stereotypes of these users, as well as providing a way of quantifying stereotypes that blends existing sociological and psychological theory in a novel, parsimonious way.

2016

1. Identifying Police Officers at Risk of Adverse Events

   Samuel Carton, Jennifer Helsby, Kenneth Joseph, and 7 more authors

   In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Sep 2016
2. Contextual Sentiment Analysis

   Will Frankenstein, Kenneth Joseph, and Kathleen M Carley

   In International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation, Sep 2016

   Abs Link to Paper

   This study examines the role of context in evaluating responses to social media posts online. Current sentiment analysis tools evaluate the content of posts without considering the broader context that the post comes from. Utilizing data from an in-person study, we examine differences between perceived sentiment evaluation when social media response posts are viewed in isolation and perceived sentiment evaluation when social media responses are viewed in the context of the original post. We find that evaluations of responses viewed in context change over 50 % of the time. We validate this finding by utilizing simulated data to show the result is not simply a result of data manipulation or noisy data; furthermore, we explore results of this finding with current sentiment analysis tools, examining this result with subsets of our data with high and low kappa values.

3. Exploring Characteristics of Suspended Users and Network Stability on Twitter

   Wei Wei, Kenneth Joseph, Huan Liu, and 1 more author

   Social network analysis and mining, Sep 2016

   Abs Link to Paper

   Social media is rapidly becoming a medium of choice for understanding the cultural pulse of a region; e.g. for identifying what the population is concerned with and what kind of help is needed in a crisis. To assess this cultural pulse, it is critical to have an accurate assessment of who is saying what. Unfortunately, social media is also the home of users who engage in disruptive, disingenuous, and potentially illegal activity. A range of users, both human and non-human, carry out such social cyber-attacks. We ask, to what extent does the presence or absence of such users influence our ability to assess the cultural pulse of a region? Our prior research on this topic showed that Twitter-based network structures and content are unstable and can be highly impacted by the removal of suspended users. Because of this, statistical techniques can be established to differentiate potential types of suspended and non-suspended users. In this extended paper, we develop additional experiments to explore the spatial patterns of suspended users, and we further consider how these users affect structural and content concentrations via the development of new metrics and new analyses. We find significant evidence that suspended users exist on the periphery of social networks on Twitter and consequently that removing them has little impact on network structure. We also improve prior attempts to distinguish among different types of suspended users by using a much larger dataset. Finally, we conduct a temporal sentiment analysis to illustrate differences between suspended users and non-suspended users on this dimension.

4. Exploring Patterns of Identity Usage in Tweets: A New Problem, Solution and Case Study

   Kenneth Joseph, Wei Wei, and Kathleen M Carley

   In WWW, Sep 2016

   Abs Link to Paper Code

   Sociologists have long been interested in the ways that identities, or labels for people, are created, used and applied across various social contexts. The present work makes two contributions to the study of identity, in particular the study of identity in text. We first consider the following novel NLP task: given a set of text data (here, from Twitter), label each word in the text as being representative of a (possibly multi-word) identity. To address this task, we develop a comprehensive feature set that leverages several avenues of recent NLP work on Twitter and use these features to train a supervised classifier. Our model outperforms a surprisingly strong rule-based baseline by 33%. We then use our model for a case study, applying it to a large corpora of Twitter data from users who actively discussed the Eric Garner and Michael Brown cases. Among other findings, we observe that the identities used by individuals differ in interesting ways based on social context measures derived from census data.

5. Relating Semantic Similarity and Semantic Association to How Humans Label Other People

   Kenneth Joseph, and Kathleen M Carley

   In Proceedings of the First Workshop on NLP and Computational Social Science, Sep 2016

   Abs Link to Paper Code

   Computational linguists have long relied on a distinction between semantic similarity and semantic association to explain and evaluate what is being learned by NLP models. In the present work, we take these same concepts and explore how they apply to an entirely different question-how individuals label other people. Leveraging survey data made public by NLP researchers, we develop our own survey to connect semantic similarity and semantic association to the process by which humans label other people. The result is a set of insights applicable to how we think of semantic similarity as NLP researchers and a new way of leveraging NLP models of semantic similarity and association as researchers of social science.

6. A Social-Event Based Approach to Sentiment Analysis of Identities and Behaviors in Text

   Kenneth Joseph, Wei Wei, Matthew Benigni, and 1 more author

   The Journal of Mathematical Sociology, Sep 2016

   Abs Link to Paper

   We describe a new methodology to infer sentiments held toward identities and behaviors from social events that we extract from a large corpus of newspaper text. Our approach draws on affect control theory, a mathematical model of how sentiment is encoded in social events and culturally shared views toward identities and behaviors. While most sentiment analysis approaches evaluate concepts on a single, evaluative dimension, our work extracts a three-dimensional sentiment “profile” for each concept. We can also infer when multiple sentiment profiles for a concept are likely to exist. We provide a case study of a large newspaper corpus on the Arab Spring, which helps to validate our approach.

2015

1. A Bayesian Graphical Model to Discover Latent Events from Twitter

   Wei Wei, Kenneth Joseph, Wei Lo, and 1 more author

   In Proceedings of the International AAAI Conference on Web and Social Media, Sep 2015

   Abs Link to Paper

   Online social networks like Twitter and Facebook produce an overwhelming amount of information every day. However, research suggests that much of this content focuses on a reasonably sized set of ongoing events or topics that are both temporally and geographically situated. These patterns are especially observable when the data that is generated contains geospatial information, usually generated by a location enabled device such as a smartphone. In this paper, we consider a data set of 1.4 million geo-tagged tweets from a country during a large social movement, where social events and demonstrations occurred frequently. We use a probabilistic graphical model to discover these events within the data in a way that informs us of their spatial, temporal and topical focus. Quantitative analysis suggests that the streaming algorithm proposed in the paper uncovers both well-known events and lesser-known but important events that occurred within the timeframe of the dataset. In addition, the model can be used to predict the location and time of texts that do not have these pieces of information, which accounts for the much of the data on the web.

2. Analysis of Music Tagging and Listening Patterns: Do Tags Really Function as Retrieval Aids?

   Jared Lorince, Kenneth Joseph, and Peter M Todd

   In International Conference on Social Computing, Behavioral-Cultural Modeling, and Prediction, Sep 2015

   Abs Link to Paper

   In collaborative tagging systems, it is generally assumed that users assign tags to facilitate retrieval of content at a later time. There is, however, little behavioral evidence that tags actually serve this purpose. Using a large-scale dataset from the social music website Last.fm, we explore how patterns of music tagging and subsequent listening interact to determine if there exist measurable signals of tags functioning as retrieval aids. Specifically, we describe our methods for testing if the assignment of a tag tends to lead to an increase in listening behavior. Results suggest that tagging, on average, leads to only very small increases in listening rates, and overall the data do not support the assumption that tags generally serve as retrieval aids.

3. Culture, Networks, Twitter and Foursquare: Testing a Model of Cultural Conversion with Social Media Data

   Kenneth Joseph, and Kathleen M Carley

   In Ninth International AAAI Conference on Web and Social Media, Sep 2015

   Abs Link to Paper Code

   Social network research often takes the view that networks chiefly influence the spread of culture, with few reciprocal effects. While some network scholars have implied a coevolutionary relationship between the two, cultural sociologists have provided increasingly convincing evidence that it is in fact cultural preferences which mediate network structure, rather than the other way around. In the present work, we attempt to validate one such model of the conversion of cultural capital to network position. We use Twitter data to extract the ego networks of individuals and foursquare check-ins to understand their cultural preferences. Our results are indicative of the importance of considering sociological models in which culture influences network structure.

4. The Fragility of Twitter Social Networks against Suspended Users

   Wei Wei, Kenneth Joseph, Huan Liu, and 1 more author

   In Proceedings of the 2015 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining 2015, Sep 2015

   Abs Link to Paper

   Social media is rapidly becoming one of the mediums of choice for understanding the cultural pulse of a region; e.g., for identifying what the population is concerned with and what kind of help is needed in a crisis. To assess this cultural pulse it is critical to have an accurate assessment of who is saying what in social media. However, social media is also the home of malicious users engaged in disruptive, disingenuous, and potentially illegal activity. A range of users, both human and non-human, carry out such social cyber-attacks. We ask, to what extent does the presence or absence of such users influence our ability to assess the cultural pulse of a region? We conduct a series of experiments to analyze the fragility of social network assessments based on Twitter data by comparing changes in both the structural and content results when suspended users are left in and taken out. Because a Twitter account can be suspended for various reasons including spamming or spreading ideas that can lead to extremism or terrorism, we separately assess the impacts of removing apparent spam bots and apparent extremists. Experimental results demonstrate that Twitter-based network structures and content are unstable, and can be highly impacted by the removal of suspended users. Further, the results exhibit regional and temporal variation that may be related to the political situation or civil unrest. We also provides guidance on the differential impact of different types of potentially suspend-able users.

2014

1. An Approach to Selecting Keywords to Track on Twitter during a Disaster

   Kenneth Joseph, Peter M Landwehr, and Kathleen M Carley

   In ISCRAM’14, Sep 2014

   Code
2. Arab Spring: From Newspaper

   Kenneth Joseph, Kathleen M Carley, David Filonuk, and 2 more authors

   Social Network Analysis and Mining, Sep 2014
3. Check-Ins in “Blau Space”: Applying Blau’s Macrosociological Theory to Foursquare Check-Ins from New York City

   Kenneth Joseph, Kathleen M. Carley, and Jason I. Hong

   ACM Transactions on Intelligent Systems and Technology, Sep 2014

   Code
4. Two 1%s Don’t Make a Whole: Comparing Simultaneous Samples from Twitter’s Streaming API

   Kenneth Joseph, Peter M Landwehr, and Kathleen M Carley

   In Social Computing, Behavioral-Cultural Modeling and Prediction, Sep 2014

   Abs Link to Paper

   We compare samples of tweets from the Twitter Streaming API constructed from different connections that tracked the same popular keywords at the same time. We find that on average, over 96% of the tweets seen in one sample are seen in all others. Those tweets found only in a subset of samples do not significantly differ from tweets found in all samples in terms of user popularity or tweet structure. We conclude they are likely the result of a technical artifact rather than any systematic bias. Practically, our results show that an infinite number of Streaming API samples are necessary to collect “most” of the tweets containing a popular keyword, and that findings from one sample from the Streaming API are likely to hold for all samples that could have been taken. Methodologically, our approach is extendible to other types of social media data beyond Twitter.

2013

1. An Agent-Based Model for Simultaneous Phone and SMS Traffic over Time

   Kenneth Joseph, Wei Wei, and Kathleen M Carley

   International conference on social computing, behavioral-cultural modeling, and prediction, Sep 2013

   Abs Link to Paper

   The present work describes a utility-based, multi-agent, dynamic network model of phone call and SMS traffic in a population. The simulation is novel in its ability to generate interactions from both an asymmetric and a symmetric media simultaneously. Within the model, we develop and test a simple extension to the theory of media multiplexity, a well-known theory of how humans use the communication media available to them with different alters (friends). Model output qualitatively matches patterns in real data at the network-level and with respect to how humans use SMS and voice calls with different alters and thus shows general support for our theoretical claim.

2. On the Coevolution of Stereotype, Culture, and Social Relationships: An Agent-Based Model

   Kenneth Joseph, Geoffrey P Morgan, Michael K Martin, and 1 more author

   Social Science Computer Review, Sep 2013

   Abs Link to Paper Code

   The theory of constructuralism describes how shared knowledge, representative of cultural forms, develops between individuals through social interaction. Constructuralism argues that through interaction and individual learning, the social network (who interacts with whom) and the knowledge network (who knows what) coevolve. In the present work, we extend the theory of constructuralism and implement this extension in an agent-based model (ABM). Our work focuses on the theory’s inability to describe how people form and utilize stereotypes of higher order social structures, in particular observable social groups and society as a whole. In our ABM, we formalize this theoretical extension by creating agents that construct, adapt, and utilize social stereotypes of individuals, social groups, and society. We then use this model to carry out a virtual experiment that explores how ethnocentric stereotypes and the underlying distribution of culture in an artificial society interact to produce varying levels of social relationships across social groups. In general, we find that neither stereotypes nor the form of underlying cultural structures alone are sufficient to explain the extent of social relationships across social groups. Rather, we provide evidence that shared culture, social relations, and group stereotypes all intermingle to produce macrosocial structure.

2012

1. Beyond “Local”,“Categories” and “Friends”: Clustering Foursquare Users with Latent “Topics”

   Kenneth Joseph, Chun How Tan, and Kathleen M Carley

   Sep 2012

   Abs Link to Paper

   In this work, we use foursquare check-ins to cluster users via topic modeling, a technique commonly used to classify text documents according to latent "themes". Here, however, the latent variables which group users can be thought of not as themes but rather as factors which drive check in behaviors, allowing for a qualitative understanding of influences on user check ins. Our model is agnostic of geo-spatial location, time, users’ friends on social networking sites and the venue categories-we treat the existence of and intricate interactions between these factors as being latent, allowing them to emerge entirely from the data. We instantiate our model on data from New York and the San Francisco Bay Area and find evidence that the model is able to identify groups of people which are of different types (e.g. tourists), communities (e.g. users tightly clustered in space) and interests (e.g. people who enjoy athletics).