BEAAMO

The Berkeley Equity and Access in Algorithms, Mechanisms, and Optimization (BEAAMO) is a research group working broadly in the fields of algorithms and AI, with a focus on inequality and distributive justice concerns. Specifically, we draw on and contribute to techniques in areas ranging from algorithms and mechanism design, to statistical inference and machine learning, to natural language processing and human-computer interaction, all with the goal of tackling problems that impact marginalized communities. Our work at this rich interface of algorithms and inequality is also informed by close collaborations with researchers in relevant disciplines within the social sciences and humanistic studies as well as domain experts, including impacted communities, front-line workers, and relevant civil societies, community organizations, and nonprofits.

This website is work-in-progress. Follow us for updates on highlighted work related to: algorithmic assessments of social service and safety net programs, auditing and evaluation frameworks for ML and software tools used in education and criminal legal systems, technical challenges in formulating social problems as well as translating algorithmic outputs to real-world interventions, and analyzing and formulating new models for data curation, sharing, and ownership.


Members

Rediet Abebe

Assistant Professor

Meryem Essaidi

Visiting Scholar

Eve Fleisig

PhD Student (1st year)

Angela Jin

PhD Student (1st year)

Liya Mulugeta

Undergraduate student

Ezinne Nwankwo

PhD Student (1st year)

Vyoma Raman

Undergraduate Student

Ricardo Sandoval

PhD Student (incoming)

Ali Shirali

PhD Student (1st year)

Ramon Vilarino

PhD Student (1st year)

Serena Wang

PhD Student (4th year)

Naomi Yamasaki

Administrative Lead

Research Seminar


Understanding Holistic Allocation of Social Services in Sonoma County

Ezinne Nwankwo (May 2, 2022)

On the Theory of Adversarial Dynamic Data Collection

Ali Shirali (April 25, 2022)

Equity in Matching: Ramifications of Scores and Preferences in School Assignment Algorithms

Meryem Essaidi (April 18, 2022)

Abstract: Classical matching algorithms have redesigned many matching applications around the world, including that of student-to-school assignments. While these mechanisms are designed with stability and efficiency guarantees in mind, they can show inherent limitations in satisfying other desiderata. Notably, implementations of these mechanisms can lead to a segregated student bodies or inequality in assignment processes. A recent body of work aims to address these issues using diversity-aware matching mechanisms, which aim to ensure that schools are representative of communities across demographic dimensions such as gender, race, or income groups.

In our work, we discuss whether classical matching mechanisms such as the Boston Mechanism, Serial Dictatorship, and Top Trading Cycles yield equitable allocations. We ask: Do classical mechanisms yield unintended disparate allocations by different demographics? We further investigate if these issues are addressed by diversity-aware mechanisms. We present our results using a large dataset from a National School Assignment and show that both the classical matching mechanisms and the diversity-aware matching mechanisms yield large gaps across different demographics, even when disaggregating across a single demographic dimension (i.e., students' reported gender). We show that female students, and to a greater extent older female students, get assigned to universities that rank significantly worse on their preference list than their male counterparts. We discuss the role of student scores and preference elicitation practices as drivers behind this inequity.

Finally, these results motivate us to rethink the inherent properties of these mechanisms and the assumptions surrounding them. We close with a discussion around the following questions: What characterizes preferences in the real-world? Do preferences present practical difficulties? Do current preference elicitation mechanisms adequately capture people's preferences?

Data as Embodied Relationships: Some Initial Thoughts on Upholding Ontological Security in Data Governance

Zoë Bell (April 11, 2022)

Abstract: I will present the "data as social relations" or "data as democratic medium" framework for understanding data, its value, and its harms in the digital data economy from law scholar Salmoé Viljoen's "A Relational Theory of Data Governance." I will then more informally discuss some potential technical research directions I am exploring that could come from taking this alternate ontology for data seriously, instead of viewing data as information, capital, or labor describing and/or belonging to an individual. This will lead to a discussion of how "ontological security" might be upheld by data co-ops in order to maintain cultural data sovereignty, and then I will open up the conversation for thoughts on what technical tools could support this goal, and for general feedback on this possible research direction of mine.

Envisioning Methods for Scrutinizing Evidentiary Software

Angela Jin (April 4, 2022)

The US criminal legal system increasingly relies on software outputs to convict and incarcerate people. In a vast number of cases each year, the government makes these consequential decisions based on evidence from software tools that the defense counsel cannot fully cross-examine or scrutinize. This offends the commitments of the adversarial criminal legal system, which relies on the defense’s ability to probe and test the prosecution’s case to seek truth and safeguard individual rights. At present, there are no technical standards to adversarially scrutinize output from software used as evidence at trial. This gap has led a variety of government bodies, advocacy groups, and other stakeholders to call for standards and programs to systematically examine the reliability of software tools.

This talk consists of two parts. In Part I, I will discuss our FAccT proposal: robust adversarial scrutiny as a framework for questioning evidence output from statistical software, which range from probabilistic genotyping tools, to environment audio detection, and toolmark analysis. Drawing on a large body of recent work in robust machine learning and algorithmic fairness, we define and operationalize this notion of robust adversarial scrutiny for defense use. In Part II, I will discuss our next steps in determining the degree to which our proposed framework aligns with how defense lawyers’ currently think about evidentiary statistical software with respect to cases involving the clients they serve.

Inference, Measurement, and Qualitative Frameworks for Algorithmic Auditing: A Case in Probabilistic Genotyping Software

Ramon Vilarino (March 14, 2022)

Genotyping is the idea of utilizing a few specific sites of the human non-encoding genome (a.k.a. junk DNA) that are expected to vary subtly throughout the population to create a system to code and differentiate identities. The choice of sites is critical, since the number of different variations expected to happen, along with their assumed frequency in the population, determine how effective, in principle, a genotyping can be. Probabilistic Genotyping Software (PGS) names a class of genotyping systems that leverage computational tools to infer identity match between two sources of genetic evidence. Extracting information from evidence always implicates interpretation and judgment calls based on assumptions about the processes that generated the evidence. This means that we should always expect evidence analysis to be sensitive to the judgments and assumptions underlying it. There is reasonable intuition behind PGS systems and the scientific constructions that justify them are widely accepted. However, the subtleties of the judgments and assumptions underlying them seldomly find place during their use in the criminal system, making their use acceptable in an incredibly wide range of scenarios. Which brings the question: do PGS systems pose a threat to due process by leading us to misinterpreting evidence? Can we design a set of experiments that enable us to characterize and differentiate qualitative scenarios according to if and how poorly we expect the inference PGS systems to perform?

Towards a Theory of Justice for Artificial Intelligence

Iason Gabriel (March 7, 2022)

Abstract: This paper explores the relationship between artificial intelligence and principles of distributive justice. Drawing upon the political philosophy of John Rawls, it holds that the basic structure of society should be understood as a composite of socio-technical systems, and that the operation of these systems is increasingly shaped and influenced by AI. As a consequence, egalitarian norms of justice apply to the technology when it is deployed in these contexts. These norms entail that the relevant AI systems must meet a certain standard of public justification, support citizens rights, and promote substantively fair outcomes – something that requires specific attention be paid to the impact they have on the worst-off members of society.

What Makes Human Decision-Making Special?

Christopher Strong (February 28, 2022)

Abstract: As algorithmic tools are more widely deployed, there remain fundamental questions about how to evaluate them. For algorithms designed to replace tasks that humans perform, designers often attempt to compare these tools against “human performance.” These comparisons occur across many domains, for example in machine translation, health diagnostics, and criminal risk assessment. An unreliable comparison can lead to an overconfident or premature deployment of a tool, which in turn can cause widespread harm. In this talk, we examine what makes human decision-making effective, with the goal of better understanding the ways in which human decision-making displays robustness and reliability beyond what we expect to see from algorithmic counterparts. We provide a review of evaluations of algorithms and discuss conjectures that may help explain gaps we observe in human vs. algorithmic decision-making.

Auditing Algorithms: Understanding Algorithmic Systems from the Outside In

Danaë Metaxa (February 23, 2022)

Abstract: Algorithms are ubiquitous and critical sources of information online, increasingly acting as gatekeepers for users accessing or sharing information about virtually any topic, including their personal lives and those of friends and family, news and politics, entertainment, and even information about health and well-being. As a result, algorithmically-curated content is drawing increased attention and scrutiny from users, the media, and lawmakers alike. However, studying such content poses considerable challenges, as it is both dynamic and ephemeral: these algorithms are constantly changing, and frequently changing silently, with no record of the content to which users have been exposed over time. One strategy that has proven effective is the algorithm audit: a method of repeatedly querying an algorithm and observing its output in order to draw conclusions about the algorithm’s opaque inner workings and possible external impact. In this work, we present an overview of the algorithm audit methodology, including the history of audit studies in the social sciences from which this method is derived; a summary of key algorithm audits over the last two decades in a variety of domains, including health, politics, discrimination, and others; and a set of best practices for conducting algorithm audits today, contextualizing these practices using search engine audits as a case study. Finally, we conclude by discussing the social, ethical, and political dimensions of auditing algorithms, and propose normative standards for the use of this method.

Understanding Nuanced Opinions on Subjective Tasks by Capturing Annotator Disagreement

Eve Fleisig (February 14, 2022)

Abstract: To elicit judgments from multiple annotators for machine learning datasets, researchers typically use the majority vote among annotators as the ground truth. This makes sense for many tasks: if 3 out of 4 annotators say an image contains a cat, the fourth annotator was likely inattentive. But what happens if the task is more subjective, such as asking about the sentiment or offensiveness of a statement? Recent work has found that taking the majority vote on subjective tasks obscures the opinions of minority groups and masks real disagreement among annotators. For example, using majority-vote aggregated data for hate speech classification causes text in African-American English to be disproportionately marked as offensive, even when African-American annotators disagree. In this talk, I’ll give an overview of the issues stemming from aggregate annotations; examine existing ideas for capturing more nuanced opinions and propose new alternatives; and discuss ongoing work on reframing machine learning objectives for subjective tasks.

Exploring the Limitations of Group-Based Fairness in Machine Learning

Serena Wang (February 7, 2022)

Abstract: Much of the recent literature in machine learning (ML) fairness has focused on statistical group-based notions of fairness, where the goal is to achieve or equalize some model performance metric across protected groups. While this framework has received widespread mathematical attention, this talk will discuss several of its practical and philosophical limitations. First, there is a practical issue of enforcing group-based fairness constraints when the data on protected groups is incomplete or noisy. Second, even with perfect data, we discuss rule-based notions of fair treatment that group-based fairness notions still cannot philosophically capture. Finally, even the most heavily fairness-constrained ML model might still fall short in satisfying societal needs due to choices in problem formulation and downstream interventions. Thus, we argue that the typical view of the ML life cycle in ML research needs to be expanded to capture a full spectrum of societal impacts.

An Algorithmic Introduction to Saving Circles

Christian Ikeokwu (January 24, 2022)

Abstract: Rotating savings and credit associations (roscas) are informal financial organizations common in settings where communities have reduced access to formal institutions. In a rosca, a fixed group of participants regularly contribute sums of money to a pot. This pot is then allocated periodically using lottery, aftermarket, or auction mechanisms. Roscas are empirically well-studied in the economics literature. Due to their dynamic nature, however, roscas have proven challenging to study theoretically, and typical economic economic analyses stop at coarse ordinal welfare comparisons to other credit allocation mechanisms and leave much of roscas’ ubiquity unexplained. This work takes an algorithmic perspective on the study of roscas. We present worst-case welfare approximation guarantees, building on tools from the price of anarchy. These cardinal welfare analyses help rationalize the prevalence of roscas. We conclude by discussing several other promising avenues.

Summer Fellowship


BEAAMO is holding its first summer fellowship program!

During the summer of 2022, fellows will tackle questions related to auditing and evaluation of algorithmic tools used in criminal legal systems. Hoping to advance and further internationalize the debate around the use of algorithms in criminal law, we will focus on tools used both in Brazil and the US. Fellows will work closely with faculty and graduate researchers in computer science and law as well as public defenders and those working in civil society organizations.

The summer program will host participants from or based in Brazil. We welcome researchers and practitioners in the areas of computer and data science, mathematics, statistics, and closely related fields to apply. We welcome applications from candidates with various degrees of educational background, recognizing that expertise can also be demonstrated through past projects and experiences besides a specific degree. We especially welcome applicants who can contribute with unique perspectives and diverse experiences to apply.

Program Lead: Ramon Vilarino

Faculty Advisor: Rediet Abebe
Administrative Lead: Naomi Yamasaki

Application Details

Who

We invite applicants from or based in Brazil. Applicants should have demonstrated experience in computer science, data science, mathematics, statistics, or closely related fields, as well as enthusiasm to work with researchers and practitioners in law, including scholars, public defenders, and those working in civil societies and non-profit organizations.

When

Priority Application Deadline: March 2, 2022 (11:59 PM PT)

Applications will be received until the internship slots are filled, but priority will be given to those who apply by March 2nd.

Notification: March 21, 2022

Start Date: June 20, 2022

End Date: August 12, 2022

Where

We plan to hold this program in person on the University of California, Berkeley campus in Berkeley, California, USA, although plans may change depending on the ongoing pandemic.

Please use this application form.

Application details: English, Português