Proposition: Representation of Functions in an RKHS

Every function in a reproducing kernel Hilbert space (RKHS) associated with a kernel $K$ can be written as a (possibly infinite) linear combination of kernel sections. That is, for any function $f$ in the RKHS $H$, there exists a countable collection of points ${x_i}$ in the domain and coefficients ${\alpha_i}$ such that

where the sum converges in the norm of the Hilbert space.

Proof Sketch: Justification via RKHS Construction

To understand this representation, recall how an RKHS is constructed. The space is formed by taking all finite linear combinations of kernel sections, which are functions of the form $K(x_i, \cdot)$. These finite combinations are then completed with respect to the Hilbert space norm, allowing for limits of Cauchy sequences of such combinations. As a result, any function in the RKHS can be approximated arbitrarily well (in the RKHS norm) by finite sums of kernel sections. By the completeness of the Hilbert space, these limits exist within the space, and thus every function in the RKHS is a possibly infinite linear combination of kernel sections.

Discussion: Implications for Smoothness, Regularity, and Structure

This characterization has important consequences. Because kernel sections themselves are determined by the properties of the kernel $K$, the functions in the RKHS inherit smoothness and regularity from the kernel. For example, if the kernel is smooth, then every function in the RKHS is also smooth to the same degree. The structure of the RKHS is therefore tightly controlled by the choice of kernel: the space may consist of very smooth functions (as with the Gaussian kernel) or less regular functions (as with some other kernels). This representation also underpins practical methods in applied mathematics and machine learning, where solutions to many problems are sought as sums of kernel sections centered at data points.