22 September 1999
Technical Mathematics, Inc.
Suite 4, Strawmarket Business Plaza
Lonlinc, SK 04685
Dear TMI:
Allow us first to introduce our company: Q.I. Temad, Inc., is a pioneer in socio-psychological research, liberally funded by thoughtful think-tanks with historical and (most importantly) backward-thinking roots. We have contacted you to resolve the mathematical quandary that has recently come to bedevil the convoluted and anomalous minds that are those of our staff.
Needless to say, the deep and unfathomable socio-psychological research in which we deliberately dabble requires the assessment of the various mental capacities of our different clients, who numbers are exceedingly large (at least when counting multiple personalities). These assessments are generally administered in the form of some sort of test or exam, replete with open-ended and difficult to interpret questions, and our staff must then score the resulting papers in an appropriate manner. It is to assist with the development of an efficient scoring system for ranking these according to some unspecified scale that we are contacting you.
In particular, it is reasonable to assume that there are a large number P (say, 100) of papers to be scored, and our staff numbers relatively few (say, S=8 people). Funding clearly determines the number of staff available, as well as the time they are able to spend scoring the papers.
Ideally, each staff-member would read all papers and rank-order them, but there are too many papers for this. Instead, there are a number of screening rounds in which each staff-member reads some number of papers and gives them scores. Then some selection scheme is used to reduce the number of papers under consideration: if the papers are rank-ordered, then the bottom 30% that each staff-member rank-orders could be rejected. Alternatively, if they do not rank-order the papers, but instead give them numerical scores (say, from 1 to 100), then all papers falling below some cutoff level could be rejected.
The new pool of papers is then passed back to the staff-members, and the process is repeated. A concern is that the total number of papers that each staff-member reads must be substantially less than P. The process is stopped when there are only W (``winning'') papers left. Typically, for P=100, we have W=3.
Your task is to determine a selection scheme, using a combination of rank-ordering, numerical scoring, and other methods, by which the final W papers will include only papers from the ``best'' 2W papers. (By ``best,'' we assume that there is an absolute rank-ordering to which all staff-members would agree.) For example, the top three papers found by your method will consist entirely of papers from among the ``best'' six papers. Among all such methods, the one that requires each staff-member to read the least number of papers is desired.
Note the possibility of systematic bias in a numerical scoring scheme. For example, for a specific collection of papers, one staff-member could average 70 points, while another could average 80 points. How would you scale your scheme to accommodate for changes in the given parameters (P, S, and W)?
It is with delicious anticipation that we await your solution to this perplexing problem. We expect a first-presentation of your essentially-completed results on the 1st of October, with final presentations to follow on the 6th or 8th of that month. Your final report will be ecstatically received on the 11th of October.
Yours most sincerely,
Simon F. Reud
Head Theorist, Q.I. Temad, Inc.
sfr:glr