A comparison of aggregation methods of subjective probability distributions

Gu, Yuhong

Permalink

https://hdl.handle.net/2142/14756 Copy

Description

Title

A comparison of aggregation methods of subjective probability distributions

Author(s)

Gu, Yuhong

Issue Date

2010-01-06T17:50:02Z

Director of Research (if dissertation) or Advisor (if thesis)

• Budescu, David V.
• Abbas, Ali E.

Department of Study

Psychology

Discipline

Psychology

Degree Granting Institution

University of Illinois at Urbana-Champaign

Degree Name

M.A.

Degree Level

Thesis

Keyword(s)

• subjective probability distributions
• aggregation quality

Abstract

One of the goals of psychological research on subjective judgments is to develop procedures that can improve judgment aggregation quality. The need to aggregate various judgments arises since in many cases the decision maker is uncertain about the possible outcomes of his decisions solicits suggestions from multiple advisors. In this paper, we study the quality of aggregation of multiple subjective probability distributions of future temperatures, using data collected by Abbas, Budescu, Yu and Haggerty (2008), as a function of 4 factors – the elicitation method (Fixed Probability versus Fixed Variable), the aggregation method (combining directly points on the distribution or aggregating parameters of fitted distributions), the aggregation statistic (using the mean or the, more robust, median to represent the aggregated values), and group size (we used data from 32 judges and we compare results of 200 replications of sub-groups of increasing size: the 32 single judges (n=1), 16 pairs of judges (n=2), 8 groups of n=4 judges, 4 groups of n=8 judges, 2 groups of n=16 judges, and a summary of all n=32 judges). The quality of aggregation is measured primarily by the closeness of the estimated probability distribution to the reference distribution based on historical data. We observed that as group sizes increases, aggregation quality improves (closer fit to the historical values) and it matters less which judges are aggregated and how the judgments are aggregated. Aggregates based on FP assessment generate higher quality than aggregates based on FV assessment under most circumstances. When FP is adopted, point aggregation generates better results than parameter aggregation. If FV has to be adopted for practical reasons, using parameter aggregation with mean may produce higher quality results.

Graduation Semester

2009-12

Permalink

http://hdl.handle.net/2142/14756

Copyright and License Information

Copyright 2009 Yuhong Gu

Owning Collections