Control deadband quantization model calculator

Calculate output trace and error trace with your own figures.

Your numbers

Use the units shown. Results update as you change valid inputs.

Use output trace and error trace to compare changes to your inputs. The method and limits below explain the scope.

Examples

Examples

Try the starting example

Command samples
0,0.1,0.4,0.9,0.3,-0.2
Deadband half width
0.15
Quantization step
0.1
Output trace
[0,0,0.3,0.8,0.1,-0.1]
Error trace
[0,-0.1,-0.1,-0.1,-0.2,0.1]
Root mean square error
0.11547

Try another scenario

Command samples
[0, 0.3, 0.8, -0.8]
Deadband half width
0.15
Quantization step
0.1
Output trace
[0,0.1,0.7,-0.7]
Error trace
[0,-0.2,-0.1,0.1]
Root mean square error
0.122474
Calculation method

Calculation method

Output trace

Zero output inside deadband; outside subtract the deadband then round half away from zero.

Error trace

Quantized output minus each input.

Root mean square error

Square root of mean squared errors.

Symbols and units
a
Command samples
b
Deadband half width
c
Quantization step

Use the units shown. A list keeps the order in which you enter its values. The symbol ^ means “raised to the power of”.

Assumptions & limits

Assumptions & limits

  • Memoryless symmetric dead-zone followed by uniform quantization. Does not model hysteresis or a mechanical actuator. Quantities are deterministic calculations from supplied observations and assumptions; no external data is fetched. Supported limits: Finite inputs only; numeric fields bounded as labelled. Lists and JSON records are limited to 256 entries unless a stricter tool bound is stated. No automatic design approval, legal conclusion, treatment recommendation or safety certification. Command samples: minimum None, maximum 1000000000000.0. Deadband half width: minimum 0, maximum 1000000000000.0. Quantization step: minimum 1e-12, maximum 1000000000000.0.
Are these inputs private?

These calculations run in your browser. A shared link contains the values you choose to share. Only questions submitted to the separate AI finder are sent to OpenAI.

Further reading

Method reference ↗

Background reading; this does not imply endorsement or independent review of SumFizz.

Share this page
Share this calculator

Leave unchecked to share the page without your entered values.