Examples
Examples
Try the starting example
- Ratings: candidate ID, interviewer ID, numeric rating
- C1,A,4 C1,B,4 C2,A,2 C2,B,4 C2,C,3
- Minimum allowed rating
- 1
- Maximum allowed rating
- 5
- Observed rating range by candidate
- C1: 0; C2: 2
- Mean absolute difference across same-candidate rater pairs
- 1
- Exact agreement across same-candidate rater pairs (%)
- 25
- Number of comparable unordered rater pairs
- 4
- Candidates with fewer than two ratings
- None
Try another scenario
- Ratings: candidate ID, interviewer ID, numeric rating
- C1,A,4 C1,B,5 C2,A,2 C2,B,4 C2,C,3
- Minimum allowed rating
- 1
- Maximum allowed rating
- 5
- Observed rating range by candidate
- C1: 1; C2: 2
- Mean absolute difference across same-candidate rater pairs
- 1.25
- Exact agreement across same-candidate rater pairs (%)
- 0
- Number of comparable unordered rater pairs
- 4
- Candidates with fewer than two ratings
- None
Calculation method
Calculation method
Observed rating range by candidate
Maximum−minimum supplied score for each candidate; zero with a single rating.
Mean absolute difference across same-candidate rater pairs
List every unordered distinct-interviewer pair within a candidate, then average absolute score differences over all pairs.
Exact agreement across same-candidate rater pairs (%)
Equal-score pair count/all comparable pairs×100.
Number of comparable unordered rater pairs
Sum n×(n−1)/2 within each candidate.
Candidates with fewer than two ratings
List candidates excluded from pair comparisons because only one rating was supplied.
Symbols and units
- a
- Ratings: candidate ID, interviewer ID, numeric rating
- b
- Minimum allowed rating
- c
- Maximum allowed rating
Use the units shown. A list keeps the order in which you enter its values. The symbol ^ means “raised to the power of”.
Assumptions & limits
Assumptions & limits
- A descriptive agreement audit of supplied ratings. It is not chance-adjusted kappa, inter-rater reliability estimation, a candidate selection rule or proof of interviewer quality; no applicant ranking is produced.
Are these inputs private?
These calculations run in your browser. A shared link contains the values you choose to share. Only questions submitted to the separate AI finder are sent to OpenAI.
Further reading
ILO: supplied workforce records and full-time-equivalent measurement background ↗
Background reading; this does not imply endorsement or independent review of SumFizz.