Low-bit representations of oversampled signal expansions and neural networks
ABSTRACT
We discuss mathematical aspects of how to digitally represent information. Redundancy or oversampling is an important ingredient in many types of signal representations. For example, in the classical Shannon sampling theorem, redundancy provides design flexibility and robustness against noisy measurements. We shall discuss analog-todigital conversion for redundant signal representations. We show error bounds which quantify how well different quantization methods, such as consistent reconstruction and Sigma-Delta quantization, utilize redundancy. Lastly, we discuss the problem of training neural networks with low-bit weights; we consider an approach based on stochastic Markov gradient descent (SMGD) and prove that the method performs well both theoretically and numerically.
Registration URL:
https://cityu.zoom.us/meeting/register/tJcrd--vqz4sGND_bAdFEMq8Ssj-UIrFxOwE