I am a Ph.D. student in the School of Computer Science at Carnegie Mellon University, advised by Nihar Shah. Previously, I received a B.S. in EECS from UC Berkeley, where I worked with Laura Waller on computational imaging. My research interest is in machine learning, particularly in applications to improving the process of peer review and crowdsourcing. In these applications, the goal of my research is to understand and mitigate various biases using tools from computer science and statistics, and to also have realworld impact through outreach and policies. Email: jingyanw [at] cs.cmu.edu 

 Human bias: People can be biased when they make evaluations (even if they try to be accurate and impartial). One type of such bias is the miscalibration of people, that is, the fact that people have different scales when expressing opinions.
In past work, we designed algorithms that take into account the miscalibration of people  for example, some people tend to be lenient and give higher scores, and some others tend to be strict. Our algorithms are provably more robust to people’s miscalibration, and hence make the evaluation more fair. [paper]
Another type of bias is introduced by the outcome that people have experienced. For example, when evaluating instructors' teaching quality, students often give higher ratings if they receive higher grades, and often give lower ratings if they receive lower grades. In past work, we formulated a model for this problem, and proposed an estimator that reduces the bias in a datadependent fashion, without prior knowledge on the extent of the existing bias. [paper]  Algorithmic bias: An emerging line of research on algorithmic bias addresses the question: "Can algorithms amplify biases that exist in the data?" In past work, we addressed a related but different question: "Can algorithms themselves introduce a bias, even if the data is perfectly unbiased?" Specifically, for a widelyused model (the BradleyTerry model) and algorithm (the maximum likelihood estimator) for peergrading, we mathematically showed that even under the assumption that students give unbiased peer grades, the algorithm systematically assigns lower scores to the top students and assigns higher scores to the students at bottom. In turn, we proposed a modification to the algorithm which provably reduces such bias and is also extremely simple to be implemented in practice. [paper]
 Policy bias: Going beyond technical work, I desire to extend my research to real policy improvements in practice. For example, we compiled the data and evaluated the gender distribution in awardwinning papers in 16 top computer science conferences in the past 10 years, which showed prominent differences across conferences [blog post]. We also considered the biases caused by the alphabetical ordering practice in scientific publication [blog post]. Taking cognizance of the biases arising from alphabetical ordering, the Machine Learning Department at CMU has randomized the ordering of students and faculty on its webpages.


A Heuristic for Statistical Seriation
Komal Dhull, Jingyan Wang, Nihar B. Shah, Yuanzhi Li, R. Ravi
UAI, 2021

Debiasing Evaluations That are Biased by Evaluations
Jingyan Wang, Ivan Stelmakh, Yuting Wei, Nihar B. Shah
AAAI, 2021

Stretching the Effectiveness of MLE from Accuracy to Bias for Pairwise Comparisons
Jingyan Wang, Nihar B. Shah, R. Ravi
AISTATS, 2020 [talk]

Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings
Jingyan Wang, Nihar B. Shah
AAMAS, 2019 Best student paper award at AAMAS 2019Also appeared as "Ranking and Rating Rankings and Ratings" atAAAI 2020 Sister Conference Track [slides]

The more you look, the more you see: towards general object understanding through recursive refinement
Jingyan Wang, Olga Russakovsky, Deva Ramanan
WACV, 2018 [code] [supplementary]

Having Your Cake and Eating It Too: Jointly Optimal Codes for I/O, Storage and Networkbandwidth in Distributed Storage Systems
K. V. Rashmi, Preetum Nakkiran, Jingyan Wang, Nihar B. Shah, Kannan Ramchandran
USENIX FAST, 2015 Picked as the best paper of USENIX FAST 2015 by StorageMojo

3D differential phasecontrast microscopy with computational illumination using an LED array
Lei Tian, Jingyan Wang, Laura Waller
Optics Letters, 2014

 16720 Computer Vision TA, Fall 2017