Paper: arXiv 2504.16093

Authors: Yurun Ge, Lucas Böttcher, Tom Chou, Maria R. D’Orsogna

Abstract

How to allocate limited resources to projects that will yield the greatest long-term benefits is a problem that often arises in decision-making under uncertainty. For example, organizations may need to evaluate and select innovation projects with risky returns. Similarly, when allocating resources to research projects, funding agencies are tasked with identifying the most promising proposals based on idiosyncratic criteria. Finally, in participatory budgeting, a local community may need to select a subset of public projects to fund. Regardless of context, agents must estimate the uncertain values of a potentially large number of projects. Developing parsimonious methods to compare these projects, and aggregating agent evaluations so that the overall benefit is maximized, are critical in assembling the best project portfolio. Unlike in standard sorting algorithms, evaluating projects on the basis of uncertain long-term benefits introduces additional complexities. We propose comparison rules based on Quicksort and the Bradley–Terry model, which connects rankings to pairwise “win” probabilities. In our model, each agent determines win probabilities of a pair of projects based on his or her specific evaluation of the projects’ long-term benefit. The win probabilities are then appropriately aggregated and used to rank projects. Several of the methods we propose perform better than the two most effective aggregation methods currently available. Additionally, our methods can be combined with sampling techniques to significantly reduce the number of pairwise comparisons. We also discuss how the Bradley–Terry portfolio selection approach can be implemented in practice.

Complexity vs Empirical Score

  • Math Complexity: 7.5/10
  • Empirical Rigor: 3.0/10
  • Quadrant: Lab Rats — high math complexity, limited empirical testing

Why this score: The paper introduces significant mathematical complexity through the Bradley-Terry model and statistical theory, but provides minimal empirical validation with no backtesting or implementation details.

Research Flowchart

  flowchart TD
  A["Research Goal: Develop efficient methods to rank & select projects under uncertainty"] --> B{"Key Inputs: <br>Set of N Projects <br>Agent Evals & Pairwise Win Probabilities"}
  B --> C["Methodology: <br>Aggregate Probabilities & Apply Quicksort with Bradley-Terry Model"]
  C --> D{"Data Processing: <br>Computational Ranking & Portfolio Selection"}
  D --> E["Validation: <br>Benchmark against existing aggregation methods"]
  E --> F["Key Findings: <br>Higher performance & efficiency achieved <br>Sampling reduces pairwise comparisons"]

Evidence Card

What the authors claim, on what data, and who has checked it. Read from the paper's own text by a language model: every quote is verbatim, nothing was re-run. How to read this card

Proposes

The paper proposes pairwise‑comparison aggregation rules based on Quicksort and the Bradley–Terry model to rank projects using agents’ win probabilities.

Source sentence
We propose comparison rules based on Quicksort and the Bradley--Terry model, which connects rankings to pairwise "win" probabilities. In our model, each agent determines win probabilities of a pair of projects based on his or her specific evaluation of the projects' long-term benefit.

Assumes 4

  • The noise in perceived project values is independently and normally distributed. statistical
    Source sentence
    (7) Under the assumption that the noise in the perceived value is independently and normally distributed, the quantity ηiℓ−η jℓfollows a normal distribution with a mean of 0 and a standard deviation of q σ2 iℓ+ σ2 jℓ.
  • Agents evaluate win probabilities wℓij as the probability that their perceived value for project i exceeds that for project j. model
    Source sentence
    Suppose agent ℓis evaluating projects i and j, with perceived values viℓand v jℓ, respectively, and corresponding noise terms ηiℓand η jℓ. As a starting point, we define wℓ ij B Pr  vi > v j  = Pr  (viℓ−ηiℓ) > (vjℓ−ηjℓ)  = Pr  (ηiℓ−ηjℓ) < (viℓ−vjℓ)  .
  • Project intrinsic values are fixed ground‑truth numbers and agents have no direct knowledge of them. model
    Source sentence
    To model organizational decision-making, agents can be considered as having specific expertise, while projects are characterized by an intrinsic value (representing their long-term benefit and assumed to be ground truth) and a defining type [1].
  • Monte‑Carlo simulations with additive Gaussian noise are used to generate synthetic project evaluations. other
    Source sentence
    The uncer-tainty in agent ℓ’s project evaluations is quantified by additive Gaussian noise with zero mean and standard deviation σiℓ= |ti −eℓ|, where the expertise level eℓof agent ℓis given by Eq. (6). All our results are based on Monte Carlo simulations.

Data

  • Monte Carlo simulation of 30 projects with Gaussian noise synthetic unknown
    30 projects (i = 1,…,30)
    Source sentence
    We define the value of project i as vi = i (i ∈{1, . . . , 30}). The uncer-tainty in agent ℓ’s project evaluations is quantified by additive Gaussian noise with zero mean and standard deviation σiℓ= |ti −eℓ|, where the expertise level eℓof agent ℓis given by Eq. (6).

Costs

Not applicable The paper reports no trading or execution result.

Finds 3

  • Several of the proposed aggregation methods perform better than the two most effective existing aggregation methods. performance
    Source sentence
    Several of the methods we propose perform better than the two most effective aggregation methods currently available. In Section 6, we examine the performance of these aggregation methods to show that several of the methods we propose perform better than existing aggregation methods.
  • When win probabilities are restricted to a finite set, the relative performance ranking of the methods remains unchanged. performance
    Source sentence
    The relative performance ranking of the methods remains unchanged.
  • The Two‑Phase Bradley–Terry method achieves favorable performance with a relatively small number of pairwise comparisons, making it the most practical approach for real‑world use. performance
    Source sentence
    In contrast, the Two-Phase Bradley–Terry method achieves favorable performance with a relatively small number of pairwise comparisons, making it the most practical approach for real-world use. 7.

Checked by others

No other paper in the archive has been found to support or contradict these findings (compared with 18 related papers). Coverage is partial, so this is neither replication nor refutation.

Code

No repository by the authors found in the paper.

Extracted 2026-11-14 · same card as JSON · A language model chose these items: check the source sentence before relying on one.