Dataset duplicate sample audit calculator

Calculate duplicate hash groups and sample IDs and cross-split duplicate hash groups with your own figures.

Your numbers

Use the units shown. Results update as you change valid inputs.

Use duplicate hash groups and sample IDs and cross-split duplicate hash groups to compare changes to your inputs. The method and limits below explain the scope.

Examples

Examples

Try the starting example

Records: unique sample ID, supplied canonical hash, split label
A,h1,train B,h1,test C,h2,train D,h2,train E,h3,test
Duplicate hash groups and sample IDs
h1: A|B; h2: C|D
Cross-split duplicate hash groups
h1: A|B
Distinct supplied hashes
3
Redundant records beyond one per supplied hash
2
Sample IDs in cross-split duplicate groups
A, B

Try another scenario

Records: unique sample ID, supplied canonical hash, split label
A,h1,train B,h1,train C,h2,train D,h2,train E,h3,test
Duplicate hash groups and sample IDs
h1: A|B; h2: C|D
Cross-split duplicate hash groups
None
Distinct supplied hashes
3
Redundant records beyond one per supplied hash
2
Sample IDs in cross-split duplicate groups
None
Calculation method

Calculation method

Duplicate hash groups and sample IDs

Group exact supplied hash strings with more than one sample.

Cross-split duplicate hash groups

Duplicate hashes present in distinct literal split labels.

Distinct supplied hashes

Count distinct exact hash strings.

Redundant records beyond one per supplied hash

Record count minus distinct hash count.

Sample IDs in cross-split duplicate groups

All samples in cross-split duplicate groups.

Symbols and units
a
Records: unique sample ID, supplied canonical hash, split label

Use the units shown. A list keeps the order in which you enter its values. The symbol ^ means “raised to the power of”.

Assumptions & limits

Assumptions & limits

  • Hashes must already be computed using one agreed canonicalization scheme. This checks literal equality; it does not fetch files, calculate hashes, infer semantic duplicates or delete samples.
Are these inputs private?

These calculations run in your browser. A shared link contains the values you choose to share. Only questions submitted to the separate AI finder are sent to OpenAI.

Further reading

OpenStax: arithmetic and geometry ↗

Background reading; this does not imply endorsement or independent review of SumFizz.

Share this page
Share this calculator

Leave unchecked to share the page without your entered values.