Identity + Algorithms

Characteristics that make up human identity have become increasingly embedded into technological systems. Human characteristics like age, gender, race, and sexuality are being folded into the categorical structures of automated systems, such as algorithmic computer vision methods. However, these characteristics are often complex, nuanced, and fluid–and linked to social and historical instances of bias and discrimination. The simple and discrete categorization of these characteristics leads to tensions that can clash with human values and identity, and result in risky ramifications for already marginalized populations.

To mitigate the potential risks of these types of technological methods, we are researching ways to appropriately develop algorithms that are sensitive to the nuanced human identities held and expressed by the people classified. Our aim is to inform design approaches that are empowering and safe for all users.

Researchers

Jed Brubaker, Katie Gach, Aaron Jiang, Anthony Pinter, Morgan Klaus Scheuerman

Publications

From Human to Data to Dataset: Mapping the Traceability of Human Subjects in Computer Vision Datasets Scheuerman, Morgan Klaus and Weathington, Katy and Mugunthan, Tarun and Denton, Emily and Fiesler, Casey
Proc. ACM Hum.-Comput. Interact. 7, CSCW1
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3579488
Computer vision is a "data hungry" field. Researchers and practitioners who work on human-centric computer vision, like facial recognition, emphasize the necessity of vast amounts of data for more robust and accurate models. Humans are seen as a data resource which can be converted into datasets. The necessity of data has led to a proliferation of gathering data from easily available sources, including "public" data from the web. Yet the use of public data has significant ethical implications for the human subjects in datasets. We bridge academic conversations on the ethics of using publicly obtained data with concerns about privacy and agency associated with computer vision applications. Specifically, we examine how practices of dataset construction from public data-not only from websites, but also from public settings and public records-make it extremely difficult for human subjects to trace their images as they are collected, converted into datasets, distributed for use, and, in some cases, retracted. We discuss two interconnected barriers current data practices present to providing an ethics of traceability for human subjects: awareness and control. We conclude with key intervention points for enabling traceability for data subjects. We also offer suggestions for an improved ethics of traceability to enable both awareness and control for individual subjects in dataset curation practices.
Taxonomizing and Measuring Representational Harms: A Look at Image Tagging Katzman, Jared and Wang, Angelina and Scheuerman, Morgan Klaus and Blodgett, Su Lin and Laird, Kristen and Wallach, Hanna and Barocas, Solon
AAAI
- PDF
- Reference
- BibTeX

Auto-essentialization: Gender in automated facial analysis as extended colonial project Scheuerman, Morgan Klaus and Pape, Madeleine and Hanna, Alex
Big Data & Society 8, 2: 20539517211053712

PDF
Reference
BibTeX

@article{Scheuerman2021-bigdata-autoessentalization,
  title = {Auto-essentialization: Gender in automated facial analysis as extended colonial project},
  author = {Scheuerman, Morgan Klaus and Pape, Madeleine and Hanna, Alex},
  journal = {Big Data \& Society},
  volume = {8},
  number = {2},
  pages = {20539517211053712},
  year = {2021},
  publisher = {SAGE Publications Sage UK: London, England},
  tags = {identity-and-algorithms}
}

How We’ve Taught Algorithms to See Identity: Constructing Race and Gender in Image Databases for Facial Analysis Scheuerman, Morgan Klaus and Wade, Kandrea and Lustig, Caitlin and Brubaker, Jed R.
Proc. ACM Hum.-Comput. Interact. 4, CSCW1: Article 58 Best Paper Honorable Mention

PDF
Reference
BibTeX
DOI: 10.1145/3392866

@article{Scheuerman2020-cscw-databaseidentity,
  title = {How {We’ve} {Taught} {Algorithms} to {See} {Identity}: {Constructing} {Race} and {Gender} in {Image} {Databases} for {Facial} {Analysis}},
  author = {Scheuerman, Morgan Klaus and Wade, Kandrea and Lustig, Caitlin and Brubaker, Jed R.},
  doi = {10.1145/3392866},
  journal = {Proc. ACM Hum.-Comput. Interact.},
  number = {CSCW1},
  pages = {Article 58},
  volume = {4},
  year = {2020},
  tags = {identity-and-algorithms, marginalization-and-safety, positionalML},
  annote = {Best Paper Award},
  note = {Best Paper Honorable Mention}
}

How Computers See Gender: An Evaluation of Gender Classification in Commercial Facial Analysis and Image Labeling Services Scheuerman, Morgan Klaus and Paul, Jacob M and Brubaker, Jed R.
Proc. ACM Hum.-Comput. Interact. 3, CSCW: Article 144
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3359246
"Am I Never Going to Be Free of All This Crap?": Upsetting Encounters With Algorithmically Curated Content About Ex-Partners Pinter, Anthony T. and Jiang, Jialun "Aaron" and Gach, Katie Z. and Sidwell, Melanie M. and Dykes, James E. and Brubaker, Jed R.
Proc. ACM Hum.-Comput. Interact. 3, CSCW: Article 70
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3359172
Gender is not a Boolean: Towards Designing Algorithms to Understand Complex Human Identities Scheuerman, Morgan Klaus and Brubaker, Jed R.
In Participation+Algorithms Workshop at CSCW 2018.
- Abstract
- PDF
- Reference
- BibTeX
Algorithmic methods are increasingly used to identify and categorize human characteristics. A range of human identities, such as gender, race, and sexual orientation, are becoming interwoven with systems. We discuss the case of automatic gender recognition technologies that algorithmically assign binary gender categories. Based on our previous work with transgender participants, we discuss the ways current gender recognition systems misrepresent complex gender identities and undermine safety. We describe plans to build on this by conducting participatory design workshops with designers and potential users to develop improved methods for conceptualizing gender identity in algorithms.
Gender Recognition or Gender Reductionism?: The Social Implications of Embedded Gender Recognition Systems Hamidi, Foad and Scheuerman, Morgan Klaus and Branham, Stacy M.
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Abstract
- PDF
- Reference
- BibTeX
- DOI: 10.1145/3173574.3173582
Automatic Gender Recognition (AGR) refers to various computational methods that aim to identify an individual’s gender by extracting and analyzing features from images, video, and/or audio. Applications of AGR are increasingly being explored in domains such as security, marketing, and social robotics. However, little is known about stakeholders’ perceptions and attitudes towards AGR and how this technology might disproportionately affect vulnerable communities. To begin to address these gaps, we interviewed 13 transgender individuals, including three transgender technology designers, about their perceptions and attitudes towards AGR. We found that transgender individuals have overwhelmingly negative attitudes towards AGR and fundamentally question whether it can accurately recognize such a subjective aspect of their identity. They raised concerns about privacy and potential harms that can result from being incorrectly gendered, or misgendered, by technology. We present a series of recommendations on how to accommodate gender diversity when designing new digital systems.

Blog

How We’ve Taught Algorithms to See Identity

Break-ups Suck. They Could Suck Less.

Press

Even after blocking an ex on Facebook, the platform promotes painful reminders

How social media makes breakups that much worse The Problem With Putting Social Media in Charge of Our Memories