Understanding why certain layers in large neural networks become precision bottlenecks requires a mathematical perspective on quantization error. When a network layer contains weights or activations with outlier values—those significantly larger or smaller than the majority—these outliers force the quantization scheme to expand its dynamic range. For a fixed bit width, this means the quantization step size must increase to accommodate the extremes, which reduces the resolution for the bulk of values.

Mathematically, if $x_{max}$ and $x_{min}$ are the largest and smallest values in a tensor, the quantization step $Δ$ for a uniform quantizer with n bits is:

Outlier values inflate $x_{max} - x_{min}$, making $Δ$ larger and causing greater quantization error for the majority of the data. Layers with such outlier weights or activations, therefore, dominate the overall quantization error and become precision bottlenecks.

Heavy-tailed distributions, such as those following a Pareto or Cauchy law, are common in deep models—especially in weights or post-activation statistics. These distributions have a significant probability of producing extreme values, which increases higher-order statistical moments like variance and kurtosis. In quantization, the mean squared error (MSE) is influenced by the second moment (variance) of the distribution. If a tensor's values are heavy-tailed, the variance is dominated by rare but large outliers, which, as explained above, force the quantizer to use a large step size. This results in poor precision for the majority of values clustered near the mean and amplifies the overall quantization error. In summary, heavy-tailed distributions challenge quantization by increasing the error due to their large statistical moments.

Normalization layers play a critical role in controlling quantization error in large models. By standardizing the distribution of activations, normalization can mitigate the precision bottlenecks created by outlier values and heavy-tailed distributions. When activations are normalized, the range of values is compressed, and the quantization step size can be reduced, improving precision for most values. However, normalization can also exacerbate bottlenecks in some cases: if normalization parameters are themselves quantized poorly, or if normalization is misapplied (for example, by amplifying small differences), the resulting distribution may still contain outliers or have increased variance. Thus, while normalization is generally stabilizing for quantized networks, its effectiveness depends on careful implementation and parameter quantization.