Processeur d'images | EPFL Graph Search

Le processeur d'images est un composant important d'un appareil photographique numérique qui joue un rôle essentiel dans la création de l'. Le processeur d'images doit réaliser plusieurs tâches complexes. Les photodiodes présentes dans un capteur photographique sont monochromes par nature : elles peuvent seulement enregistrer différents niveaux de gris. Pour obtenir une image en couleurs, elles sont recouvertes par trois filtres colorés : rouge, vert et bleu (RGB) selon un arrangement appelé filtre de Bayer - d'après le nom de son inventeur. Comme chaque photodiode enregistre l'information de couleur d'un seul pixel de l'image, sans processeur d'images il y aurait un pixel vert près de chaque pixel rouge et bleu (en fait, sur la plupart des capteurs il y a deux diodes vertes pour chaque diode bleue ou rouge). Le processeur d'images est constitué d'une combinaison de processeurs (hardware) et d'algorithmes (software). Le processeur d'images rassemble les informations de luminance et

A 170μW Image Signal Processor Enabling Hierarchical Image Recognition for Intelligence at the Edge

Bowen Liu

We propose an ultra-low power (ULP) Image Signal Processor (ISP) that performs on-the-fly in-processing frame (de)compression and hierarchical event recognition to exploit the temporal and spatial sparsity in an image sequence to achieve a 16× imaging system energy gain. The ISP is fabricated in 40 nm CMOS and consumes only 170 μW at 5 fps for neural network-based intruder detection and 192× compressed image recording.

IEEE2020

(Geometry Aware) Deep Learning-based Omnidirectional Image Compression

Yamin Sepehri

Omnidirectional images are the spherical visual signals that provide a wide, 360◦, view of a scene from a specific position. Such images are becoming increasingly popular in fields like virtual reality and robotics. Compared to conventional 2D images, the storage and badwidth requirements of omnidirectional signals are much higher, due to the specific nature of them. Thus, there is a need for image compression schemes to reduce the dedicated storage space of omnidirectional images. Image compression algorithms can be broadly classified into two groups: lossless and lossy. Lossless schemes are able to reconstruct the exact original data but they cannot reduce the size beyond a specific criteria. Lossy methods are generally better solutions if they do not add a high visual distortion to the reconstructed image, as long as they provide a decent compression rate. If a planar, lossy image compression scheme is applied on omnidirectional images, some problems show up. It is possible to apply a planar compression scheme on a projected version of a 360◦image; however, in these projection schemes (such as equirectangular projection) the sampling rate is different in the poles and the center. Consequently, the filters of the planar compression schemes that do not consider this difference ends in suboptimal result and distortions in the reconstructed images. Recently, with the success of deep neural networks in many image processing tasks, researchers began to use them for the image compression as well. In this study, we propose a deep learning-based method for the compression of omnidirectional images by combining some state of the art approaches from the deep learning-based image compression schemes and some special convolutional layers that take into account the geometry of the omnidirectional image. In comparison to the available methods, it is the first method that can be applied directly on the equirectangularly projected version of omnidirectional images and considers the geometry in the scheme and the layers themselves. To propose this method, different geometry-aware convolutional layers have been tried. We exploited various methods of downsampling and upsampling, such as spherical pooling layers, strided or transposed convolutions, bilinear interpolation, and pixel shuffle. In the end, a method is proposed that benefits from specific spherical convolutional layers which contain sampling methods considering the geometry of omnidirectional images. The sampling positions differ in the different heights of the image based on the nature of the projected omnidirectional image. Additionally, as it benefits from an iterative training method that calculates the residual between the output and input and feeds it again to the network as input of the next iteration, it can provide different compression rates with just one pass of training. Finally, it benefits from a novel method of patching that is well-aligned with the spherical convolution layers and helps the method to run efficiently without a need for a high computational power. The model was compared with a similar architecture without spherical convolutions and spherical patching and showed some improvements. The architecture has been optimized and improved and it has the potential to compete with popular image compression schemes such as JPEG especially in terms of reconstructing the colors.

2020

(Geometry Aware) Deep Learning-based Omnidirectional Image Compression

Yamin Sepehri

2020