Compression artifact | EPFL Graph Search

A compression artifact (or artefact) is a noticeable distortion of media (including , audio, and video) caused by the application of lossy compression. Lossy data compression involves discarding some of the media's data so that it becomes small enough to be stored within the desired or transmitted (streamed) within the available bandwidth (known as the data rate or bit rate). If the compressor cannot store enough data in the compressed version, the result is a loss of quality, or introduction of artifacts. The compression algorithm may not be intelligent enough to discriminate between distortions of little subjective importance and those objectionable to the user. The most common digital compression artifacts are DCT blocks, caused by the discrete cosine transform (DCT) compression algorithm used in many digital media standards, such as JPEG, MP3, and MPEG video file formats. These compression artifacts appear when heavy compression is appl

DepthInSpace: Exploitation and Fusion of Multiple Video Frames for Structured-Light Depth Estimation

Camilla Carta, François Fleuret

We present DepthInSpace, a self-supervised deep-learning method for depth estimation using a structured-light camera. The design of this method is motivated by the commercial use case of embedded depth sensors in nowadays smartphones. We first propose to use estimated optical flow from ambient information of multiple video frames as a complementary guide for training a single-frame depth estimation network, helping to preserve edges and reduce over-smoothing issues. Utilizing optical flow, we also propose to fuse the data of multiple video frames to get a more accurate depth map. In particular, fused depth maps are more robust in occluded areas and incur less in flying pixels artifacts. We finally demonstrate that these more precise fused depth maps can be used as self-supervision for fine-tuning a single-frame depth estimation network to improve its performance. Our models' effectiveness is evaluated and compared with state-of-the-art models on both synthetic and our newly introduced real datasets.

IEEE2021

(Geometry Aware) Deep Learning-based Omnidirectional Image Compression

Yamin Sepehri

Omnidirectional images are the spherical visual signals that provide a wide, 360◦, view of a scene from a specific position. Such images are becoming increasingly popular in fields like virtual reality and robotics. Compared to conventional 2D images, the storage and badwidth requirements of omnidirectional signals are much higher, due to the specific nature of them. Thus, there is a need for image compression schemes to reduce the dedicated storage space of omnidirectional images. Image compression algorithms can be broadly classified into two groups: lossless and lossy. Lossless schemes are able to reconstruct the exact original data but they cannot reduce the size beyond a specific criteria. Lossy methods are generally better solutions if they do not add a high visual distortion to the reconstructed image, as long as they provide a decent compression rate. If a planar, lossy image compression scheme is applied on omnidirectional images, some problems show up. It is possible to apply a planar compression scheme on a projected version of a 360◦image; however, in these projection schemes (such as equirectangular projection) the sampling rate is different in the poles and the center. Consequently, the filters of the planar compression schemes that do not consider this difference ends in suboptimal result and distortions in the reconstructed images. Recently, with the success of deep neural networks in many image processing tasks, researchers began to use them for the image compression as well. In this study, we propose a deep learning-based method for the compression of omnidirectional images by combining some state of the art approaches from the deep learning-based image compression schemes and some special convolutional layers that take into account the geometry of the omnidirectional image. In comparison to the available methods, it is the first method that can be applied directly on the equirectangularly projected version of omnidirectional images and considers the geometry in the scheme and the layers themselves. To propose this method, different geometry-aware convolutional layers have been tried. We exploited various methods of downsampling and upsampling, such as spherical pooling layers, strided or transposed convolutions, bilinear interpolation, and pixel shuffle. In the end, a method is proposed that benefits from specific spherical convolutional layers which contain sampling methods considering the geometry of omnidirectional images. The sampling positions differ in the different heights of the image based on the nature of the projected omnidirectional image. Additionally, as it benefits from an iterative training method that calculates the residual between the output and input and feeds it again to the network as input of the next iteration, it can provide different compression rates with just one pass of training. Finally, it benefits from a novel method of patching that is well-aligned with the spherical convolution layers and helps the method to run efficiently without a need for a high computational power. The model was compared with a similar architecture without spherical convolutions and spherical patching and showed some improvements. The architecture has been optimized and improved and it has the potential to compete with popular image compression schemes such as JPEG especially in terms of reconstructing the colors.

2020

Central and Cortical Gray Mater Segmentation of Magnetic Resonance Images of the Fetal Brain

Anouk André, Meritxell Bach Cuadra, Gabriele Vincenzo Bonanno, Marie Schaer, Jean-Philippe Thiran

Motivation. The study of human brain development in its early stage is today possible thanks to in vivo fetal magnetic resonance imaging (MRI) techniques. A quantitative analysis of fetal cortical surface represents a new approach which can be used as a marker of the cerebral maturation (as gyration) and also for studying central nervous system pathologies [1]. However, this quantitative approach is a major challenge for several reasons. First, movement of the fetus inside the amniotic cavity requires very fast MRI sequences to minimize motion artifacts, resulting in a poor spatial resolution and/or lower SNR. Second, due to the ongoing myelination and cortical maturation, the appearance of the developing brain differs very much from the homogenous tissue types found in adults. Third, due to low resolution, fetal MR images considerably suffer of partial volume (PV) effect, sometimes in large areas. Today extensive efforts are made to deal with the reconstruction of high resolution 3D fetal volumes [2,3,4] to cope with intra-volume motion and low SNR. However, few studies exist related to the automated segmentation of MR fetal imaging. [5] and [6] work on the segmentation of specific areas of the fetal brain such as posterior fossa, brainstem or germinal matrix. First attempt for automated brain tissue segmentation has been presented in [7] and in our previous work [8]. Both methods apply the Expectation-Maximization Markov Random Field (EM-MRF) framework but contrary to [7] we do not need from any anatomical atlas prior. Data set & Methods. Prenatal MR imaging was performed with a 1-T system (GE Medical Systems, Milwaukee) using single shot fast spin echo (ssFSE) sequences (TR 7000 ms, TE 180 ms, FOV 40 x 40 cm, slice thickness 5.4mm, in plane spatial resolution 1.09mm). Each fetus has 6 axial volumes (around 15 slices per volume), each of them acquired in about 1 min. Each volume is shifted by 1 mm with respect to the previous one. Gestational age (GA) ranges from 29 to 32 weeks. Mother is under sedation. Each volume is manually segmented to extract fetal brain from surrounding maternal tissues. Then, in-homogeneity intensity correction is performed using [9] and linear intensity normalization is performed to have intensity values that range from 0 to 255. Note that due to intra-tissue variability of developing brain some intensity variability still remains. For each fetus, a high spatial resolution image of isotropic voxel size of 1.09 mm is created applying [2] and using B-splines for the scattered data interpolation [10] (see Fig. 1). Then, basal ganglia (BS) segmentation is performed on this super reconstructed volume. Active contour framework with a Level Set (LS) implementation is used. Our LS follows a slightly different formulation from well-known Chan-Vese [11] formulation. In our case, the LS evolves forcing the mean of the inside of the curve to be the mean intensity of basal ganglia. Moreover, we add local spatial prior through a probabilistic map created by fitting an ellipsoid onto the basal ganglia region. Some user interaction is needed to set the mean intensity of BG (green dots in Fig. 2) and the initial fitting points for the probabilistic prior map (blue points in Fig. 2). Once basal ganglia are removed from the image, brain tissue segmentation is performed as described in [8]. Results. The case study presented here has 29 weeks of GA. The high resolution reconstructed volume is presented in Fig. 1. The steps of BG segmentation are shown in Fig. 2. Overlap in comparison with manual segmentation is quantified by the Dice similarity index (DSI) equal to 0.829 (values above 0.7 are considered a very good agreement). Such BG segmentation has been applied on 3 other subjects ranging for 29 to 32 GA and the DSI has been of 0.856, 0.794 and 0.785. Our segmentation of the inner (red and blue contours) and outer cortical surface (green contour) is presented in Fig. 3. Finally, to refine the results we include our WM segmentation in the Freesurfer software [12] and some manual corrections to obtain Fig.4. Discussion. Precise cortical surface extraction of fetal brain is needed for quantitative studies of early human brain development. Our work combines the well known statistical classification framework with the active contour segmentation for central gray mater extraction. A main advantage of the presented procedure for fetal brain surface extraction is that we do not include any spatial prior coming from anatomical atlases. The results presented here are preliminary but promising. Our efforts are now in testing such approach on a wider range of gestational ages that we will include in the final version of this work and studying as well its generalization to different scanners and different type of MRI sequences. References. [1] Guibaud, Prenatal Diagnosis 29(4) (2009). [2] Rousseau, Acad. Rad. 13(9), 2006, [3] Jiang, IEEE TMI 2007. [4] Warfield IADB, MICCAI 2009. [5] Claude, IEEE Trans. Bio. Eng. 51(4) (2004). [6] Habas, MICCAI (Pt. 1) 2008. [7] Bertelsen, ISMRM 2009 [8] Bach Cuadra, IADB, MICCAI 2009. [9] Styner, IEEE TMI 19(39 (2000). [10] Lee, IEEE Trans. Visual. And Comp. Graph. 3(3), 1997, [11] Chan, IEEE Trans. Img. Proc, 10(2), 2001 [12] Freesurfer, http://surfer.nmr.mgh.harvard.edu.

2010