Cameras capture a substantial amount of image data, the processing of which is performed post-priori at powerful backend servers. While post-priori and non-real-time video analysis may be enough for certain groups of applications, it does not suffice for applications such as autonomous navigation in complex environments with drones, that require real-time video and image analysis under SWAP (Size Weight and Power) constraints. We hypothesize that future challenges in real-time imaging can efficiently be addressed by pushing computation into the image sensor. Focal plane sensor processors (FPSP) and smart sensors have tried to address limitations of conventional image processing systems and System-on-chip that incorporate hardware accelerators have been considered a viable solution in recent years to provide in-situ efficiency in image processing applications. While programmable devices and hardware accelerators offer the benefits of close-to-sensor processing such as performance and bandwidth reduction, they exhibit many drawbacks. Their architecture is limited to some standard low-level kernel acceleration that does not always reflect the computing pattern and requirements of the algorithms at hand. Because pixels are treated equally and processed at the same rate despite differences in input relevance for a given application, existing systems spend more time spinning on non-relevant data, which increases sensing and computation time, and power consumption.
This talk will discuss the design and implementation of a highly parallel, hierarchical, reconfigurable and 3D vertically-integrated sensing-computing architecture for real-time, and low-power video analysis. To increase performance, while reducing power, the proposed architecture leverages the concept of biological vision systems that reduces data redundancy by deploying more resources on an important parts of scene images. Visual attention is used by the brain to rapidly detect and deploy more resources to salient parts of a given scene, more precisely, it allows the brain to remove redundancy and transfer only useful information to high-level parts of the brain for further processing. The paradigm is implemented in the brain in a chain of fast feed forward signals that carry information to high-level part of the brain, while feedback signal provides configuration to the lower parts of the visual cortex. We explain the details of architecture that exploits saliency-based visual attention along with maximal parallelism for substantial power reduction and performance increase. The talk will provide details on each level of the architecture, particularly the programmable processing units and interconnect in the three planes, the inclusion of partial scene data in high-level learning, the design space exploration and the high-level synthesis.
Dr. Bobda is Professor of Computer Engineering at the University of Arkansas, in Fayetteville, AR. He received the License in mathematics from the University of Yaoundé, Cameroon in 1992, the diploma of computer science and the Ph.D. degree (Summa Cum Laude) in computer science from the University of Paderborn in Germany in 1999 and 2003 respectively. He was potsdoc at the University of Erlangen-Nuremberg from 2003 to 2005, assistant professor at the University of Kaiserslautern from 2005 to 2007 and professor at University of Potsdam from 2007 to 2010. In 2010 Dr. Bobda joined the University of Arkansas as associate professor and was promoted to full professor in 2016. Dr. Bobda research interests include Reconfigurable Computing, System-on-Chip and Computer Architecture with applications in Embedded Imaging, Cloud Computing and Cybersecurity. Dr. Bobda’s research has been sponsored by the NSF, the AFRL, the ONR and the German DFG and has led to more than 150 publications in leading journals and conferences, citation count over 2230 and more than 20,000 downloads.