THE NUMBERS, EXPLAINED
How to use this calculator.
Offset fits a constant dimensional loss; scale fits measured = factor × nominal through the origin. These are calibration-sample fits, not guarantees for other materials, orientations or hole sizes.
Start with your inputs
Enter nominal test holes (mm), measured test holes (mm), fit model (offset or scale), desired finished hole (mm). Keep amounts in the units shown on each field. The example is a starting point, not a recommended value.
| Input | Example value |
|---|---|
| Nominal test holes (mm) | 10,20 |
| Measured test holes (mm) | 9.8,19.8 |
| Fit model (offset or scale) | offset |
| Desired finished hole (mm) | 10 |
A worked example
With the example inputs above, compensated model diameter (mm) is 10.2. Change the inputs to explore your own scenario.
| Result | Example answer |
|---|---|
| Compensated model diameter (mm) | 10.2 |
| Fitted relation | measured = nominal − 0.2 mm |
| Measured minus fitted residuals (mm) | 0, 0 |
How this is calculated
Compensated model diameter (mm)
Offset model: desired diameter plus mean(nominal−measured). Scale model: desired diameter divided by sum(nominal × measured)/sum(nominal²).
Fitted relation
Display the fitted offset or scale relation.
Measured minus fitted residuals (mm)
For each test hole, subtract fitted measured diameter from actual measured diameter.
Symbols and units
- a
- Nominal test holes (mm)
- b
- Measured test holes (mm)
- c
- Fit model (offset or scale)
- d
- Desired finished hole (mm)
Use the units shown. A list keeps the order in which you enter its values. The symbol ^ means “raised to the power of”.
Calculation steps and notation
sum(list) adds all entries in the list.
Assumptions and limits
Offset fits a constant dimensional loss; scale fits measured = factor × nominal through the origin. These are calibration-sample fits, not guarantees for other materials, orientations or hole sizes.
Calculations use the values you enter. Blank or invalid inputs show an error instead of silently substituting zero. Results are displayed to useful precision; calculations retain the underlying numeric precision.
Questions about this tool
What does the result tell me?
Compensated model diameter (mm); Fitted relation; Measured minus fitted residuals (mm). The result applies to the method and inputs described above. Check the units and assumptions before using it in a decision.
Why might another calculator differ?
Differences in units, rounding, timing or model definitions can produce different answers. Compare the input definitions and the method, rather than only the final number. For this tool: Offset fits a constant dimensional loss; scale fits measured = factor × nominal through the origin. These are calibration-sample fits, not guarantees for other materials, orientations or hole sizes.
Are my inputs stored or sent to AI?
Calculations on this page happen in your browser. The separate AI question finder sends its submitted conversation to OpenAI; these calculator fields are not sent. Favourites and recent tool names are stored locally. A shared link contains the inputs you choose to include.
Further reading
OpenStax: algebra and proportional reasoning ↗
Background reading; this does not imply endorsement or independent review of SumFizz.
Review status
Software example and browser checks are recorded in our quality report. No external expert review is claimed. Created with coding assistance; calculations run as ordinary browser code.
Read the quality report →