Fixed & Free Marginal Kappa Calculator

⚙️ Setup

Number of Cases

Number of Categories

Number of Raters

How is your data organized?

Categories in Columns Each column = category count (e.g., 2 raters chose "Mild")

Raters in Columns Each column = rater's rating (e.g., "Mild", "Moderate", "Severe")

📝 Manual Entry

📋 Paste Data

📁 Upload File

📝 Data Entry

Enter the number of raters who assigned each case to each category. Each row must sum to the number of raters.

📋 Paste Data

Paste tab-separated or comma-separated data. Each row = one case. Each column = one category count.

First row contains column headers (skip first row)

📁 Upload Excel or CSV

Drag & drop your file here, or click to browse. Supports .xlsx, .xls, .csv. Make sure your data matches the selected format above.

First row contains column headers (skip first row)

📂

Click to upload or drag & drop

Excel (.xlsx, .xls) or CSV (.csv)

📚 About This Calculator

🎯 What is Kappa?

Kappa is a chance-adjusted measure of inter-rater agreement. Unlike simple percent agreement, it accounts for agreement expected by chance alone. Values range from -1 (complete disagreement) to +1 (perfect agreement).

🔓 Free-Marginal Kappa (Randolph, 2005)

Assumes raters are not forced to assign a certain number of cases to each category. Uses uniform chance agreement (1/k). Recommended when raters have free choice.

🔒 Fixed-Marginal Kappa (Fleiss, 1971)

Assumes raters are forced to assign a certain distribution of cases. Uses observed marginal distribution for chance agreement. Use when category quotas are fixed.

📊 Data Formats

Categories in columns: Each cell = count of raters for that category
Raters in columns: Each cell = the category label each rater assigned
Excel/CSV: Upload your spreadsheet directly

📐 Formulas Used

Observed Agreement (Pₒ): Pₒ = (1/N) Σᵢ [Σⱼ rᵢⱼ(rᵢⱼ-1)] / [r(r-1)] Fixed Chance Agreement (Pₑ): Pₑ = Σⱼ pⱼ² where pⱼ = (Σᵢ rᵢⱼ)/(Nr) Free Chance Agreement (Pₑ): Pₑ = 1/k Kappa: κ = (Pₒ − Pₑ) / (1 − Pₑ) 95% CI: κ ± 1.96 × SE(κ) (Gwet's variance formula, 2010)

📏 Interpretation Guide (Landis & Koch, 1977)

Kappa Value	Strength of Agreement
< 0.00	Poor (less than chance)
0.00 – 0.20	Slight
0.21 – 0.40	Fair
0.41 – 0.60	Moderate
0.61 – 0.80	Substantial
0.81 – 1.00	Almost Perfect

References: Fleiss (1971), Randolph (2005), Gwet (2010), Brennan & Prediger (1981), Landis & Koch (1977)