BNN

This research aims at proving the conjecture that there is an efficient bitwise version of neural networks and we can learn such a network that spends a minimal amount of resources in small device, whilst still enjoying the stat-of-the-art performance. A growing need for embedded systems is to efficiently run an Artificial Intelligence (AI) engine on the chip, so that it can process and analyze various kinds of real-time sensor signals around the device to predict the context, behavior, and status of the user. However, the state-of-the-art AI depends largely on the complex models, e.g. Deep Neural Networks (DNN) with easily over millions of parameters, whose run-time operations are burdensome for an embedded system with often constrained resources such as power and memory. The proposal explores Bitwise Neural Networks (BNN) to achieve both performance and efficiency goals. BNN is a transformative research direction that can entirely replace the fixed- or floating-point variables and the operations on them with binary variables and bitwise arithmetics. For example, every participating BNN parameter is represented with a bipolar binary value, i.e. +1 and -1. Consequently, computations on them are much more efficient in hardware, e.g. multiplication between real numbers is replaced with a single bitwise operation, XNOR. To prevent the possible performance degradation due to the extremely condensed nature of BNNs, one of the goals in this research is to develop training strategies for BNNs to maximize their performance, whose upper bound is that of a comprehensive DNN. At the same time, another goal is to demonstrate their efficiency by implementing BNNs on simple FPGA boards. The performance of the proposed systems will be tested on various real-time tasks, such as keyword spotting, human activity monitoring, signal denoising, etc.