Paper: arXiv 2504.16093
Authors: Yurun Ge, Lucas Böttcher, Tom Chou, Maria R. D’Orsogna
Abstract
How to allocate limited resources to projects that will yield the greatest long-term benefits is a problem that often arises in decision-making under uncertainty. For example, organizations may need to evaluate and select innovation projects with risky returns. Similarly, when allocating resources to research projects, funding agencies are tasked with identifying the most promising proposals based on idiosyncratic criteria. Finally, in participatory budgeting, a local community may need to select a subset of public projects to fund. Regardless of context, agents must estimate the uncertain values of a potentially large number of projects. Developing parsimonious methods to compare these projects, and aggregating agent evaluations so that the overall benefit is maximized, are critical in assembling the best project portfolio. Unlike in standard sorting algorithms, evaluating projects on the basis of uncertain long-term benefits introduces additional complexities. We propose comparison rules based on Quicksort and the Bradley–Terry model, which connects rankings to pairwise “win” probabilities. In our model, each agent determines win probabilities of a pair of projects based on his or her specific evaluation of the projects’ long-term benefit. The win probabilities are then appropriately aggregated and used to rank projects. Several of the methods we propose perform better than the two most effective aggregation methods currently available. Additionally, our methods can be combined with sampling techniques to significantly reduce the number of pairwise comparisons. We also discuss how the Bradley–Terry portfolio selection approach can be implemented in practice.
Complexity vs Empirical Score
- Math Complexity: 7.5/10
- Empirical Rigor: 3.0/10
- Quadrant: Lab Rats — high math complexity, limited empirical testing
Why this score: The paper introduces significant mathematical complexity through the Bradley-Terry model and statistical theory, but provides minimal empirical validation with no backtesting or implementation details.
Research Flowchart
flowchart TD
A["Research Goal: Develop efficient methods to rank & select projects under uncertainty"] --> B{"Key Inputs: <br>Set of N Projects <br>Agent Evals & Pairwise Win Probabilities"}
B --> C["Methodology: <br>Aggregate Probabilities & Apply Quicksort with Bradley-Terry Model"]
C --> D{"Data Processing: <br>Computational Ranking & Portfolio Selection"}
D --> E["Validation: <br>Benchmark against existing aggregation methods"]
E --> F["Key Findings: <br>Higher performance & efficiency achieved <br>Sampling reduces pairwise comparisons"]
Evidence Card
What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card
Proposes
The paper proposes pairwise‑comparison aggregation rules based on Quicksort and the Bradley–Terry model to rank projects using agents’ win probabilities.
Source sentence
We propose comparison rules based on Quicksort and the Bradley--Terry model, which connects rankings to pairwise "win" probabilities. In our model, each agent determines win probabilities of a pair of projects based on his or her specific evaluation of the projects' long-term benefit.
Assumes 4
- The noise in perceived project values is independently and normally distributed. statistical
Source sentence
(7) Under the assumption that the noise in the perceived value is independently and normally distributed, the quantity ηiℓ−η jℓfollows a normal distribution with a mean of 0 and a standard deviation of q σ2 iℓ+ σ2 jℓ.
- Agents evaluate win probabilities wℓij as the probability that their perceived value for project i exceeds that for project j. model
Source sentence
Suppose agent ℓis evaluating projects i and j, with perceived values viℓand v jℓ, respectively, and corresponding noise terms ηiℓand η jℓ. As a starting point, we define wℓ ij B Pr vi > v j = Pr (viℓ−ηiℓ) > (vjℓ−ηjℓ) = Pr (ηiℓ−ηjℓ) < (viℓ−vjℓ) .
- Project intrinsic values are fixed ground‑truth numbers and agents have no direct knowledge of them. model
Source sentence
To model organizational decision-making, agents can be considered as having specific expertise, while projects are characterized by an intrinsic value (representing their long-term benefit and assumed to be ground truth) and a defining type [1].
- Monte‑Carlo simulations with additive Gaussian noise are used to generate synthetic project evaluations. other
Source sentence
The uncer-tainty in agent ℓ’s project evaluations is quantified by additive Gaussian noise with zero mean and standard deviation σiℓ= |ti −eℓ|, where the expertise level eℓof agent ℓis given by Eq. (6). All our results are based on Monte Carlo simulations.
Data
- Monte Carlo simulation of 30 projects with Gaussian noise synthetic unknown
Source sentence
We define the value of project i as vi = i (i ∈{1, . . . , 30}). The uncer-tainty in agent ℓ’s project evaluations is quantified by additive Gaussian noise with zero mean and standard deviation σiℓ= |ti −eℓ|, where the expertise level eℓof agent ℓis given by Eq. (6).
Costs
Not applicable
Finds 3
- Several of the proposed aggregation methods perform better than the two most effective existing aggregation methods. performance
Source sentence
Several of the methods we propose perform better than the two most effective aggregation methods currently available. In Section 6, we examine the performance of these aggregation methods to show that several of the methods we propose perform better than existing aggregation methods.
- When win probabilities are restricted to a finite set, the relative performance ranking of the methods remains unchanged. performance
Source sentence
The relative performance ranking of the methods remains unchanged.
- The Two‑Phase Bradley–Terry method achieves favorable performance with a relatively small number of pairwise comparisons, making it the most practical approach for real‑world use. performance
Source sentence
In contrast, the Two-Phase Bradley–Terry method achieves favorable performance with a relatively small number of pairwise comparisons, making it the most practical approach for real-world use. 7.
Checked by others
No other paper in the archive has been found to support or contradict these findings (compared with 18 related papers). Coverage is partial, so this is neither replication nor refutation.
Code
No repository by the authors found in the paper.
Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.