Orthographic projection | EPFL Graph Search

Orthographic projection (also orthogonal projection and analemma) is a means of representing three-dimensional objects in two dimensions. Orthographic projection is a form of parallel projection in which all the projection lines are orthogonal to the projection plane, resulting in every plane of the scene appearing in affine transformation on the viewing surface. The obverse of an orthographic projection is an oblique projection, which is a parallel projection in which the projection lines are not orthogonal to the projection plane. The term orthographic sometimes means a technique in multiview projection in which principal axes or the planes of the subject are also parallel with the projection plane to create the primary views. If the principal planes or axes of an object in an orthographic projection are not parallel with the projection plane, the depiction is called axonometric or an auxiliary views. (Axonometric projection is synonymous with parallel projecti

(Geometry Aware) Deep Learning-based Omnidirectional Image Compression

Yamin Sepehri

Omnidirectional images are the spherical visual signals that provide a wide, 360◦, view of a scene from a specific position. Such images are becoming increasingly popular in fields like virtual reality and robotics. Compared to conventional 2D images, the storage and badwidth requirements of omnidirectional signals are much higher, due to the specific nature of them. Thus, there is a need for image compression schemes to reduce the dedicated storage space of omnidirectional images. Image compression algorithms can be broadly classified into two groups: lossless and lossy. Lossless schemes are able to reconstruct the exact original data but they cannot reduce the size beyond a specific criteria. Lossy methods are generally better solutions if they do not add a high visual distortion to the reconstructed image, as long as they provide a decent compression rate. If a planar, lossy image compression scheme is applied on omnidirectional images, some problems show up. It is possible to apply a planar compression scheme on a projected version of a 360◦image; however, in these projection schemes (such as equirectangular projection) the sampling rate is different in the poles and the center. Consequently, the filters of the planar compression schemes that do not consider this difference ends in suboptimal result and distortions in the reconstructed images. Recently, with the success of deep neural networks in many image processing tasks, researchers began to use them for the image compression as well. In this study, we propose a deep learning-based method for the compression of omnidirectional images by combining some state of the art approaches from the deep learning-based image compression schemes and some special convolutional layers that take into account the geometry of the omnidirectional image. In comparison to the available methods, it is the first method that can be applied directly on the equirectangularly projected version of omnidirectional images and considers the geometry in the scheme and the layers themselves. To propose this method, different geometry-aware convolutional layers have been tried. We exploited various methods of downsampling and upsampling, such as spherical pooling layers, strided or transposed convolutions, bilinear interpolation, and pixel shuffle. In the end, a method is proposed that benefits from specific spherical convolutional layers which contain sampling methods considering the geometry of omnidirectional images. The sampling positions differ in the different heights of the image based on the nature of the projected omnidirectional image. Additionally, as it benefits from an iterative training method that calculates the residual between the output and input and feeds it again to the network as input of the next iteration, it can provide different compression rates with just one pass of training. Finally, it benefits from a novel method of patching that is well-aligned with the spherical convolution layers and helps the method to run efficiently without a need for a high computational power. The model was compared with a similar architecture without spherical convolutions and spherical patching and showed some improvements. The architecture has been optimized and improved and it has the potential to compete with popular image compression schemes such as JPEG especially in terms of reconstructing the colors.

2020

Optimized Coding Strategies for Interactive Multiview Video

Ana Karina De Abreu Goes

The natural next step in improving the realistic experience in multimedia services is interactive multiview video (IMV). IMV promises to enable the users to freely navigate through a scene by selecting their preferred viewpoints from any view position for which the corresponding view is generated. A smooth navigation could be achieved with camera views and views synthesized at the decoder. However, the large amount of data required for such navigation experience still represents a challenge for the current systems, which implies the need for new efficient coding strategies that permit to save on storage and transmission resources, while preserving interactivity in the navigation. In this thesis, we focus on the optimization of coding strategies for IMV systems. First, we address the issues related to the coding techniques for IMV in a multiview video plus depth (MVD) scenario, where texture and depth maps are available for view synthesis at the decoder. We propose a low complexity algorithm for the selection of the interview prediction structures (PSs) and associated texture and depth quantization parameters (QPs) for IMV under transmission and storage constraints. Simulation results show that our novel low complexity algorithm has near-optimal compression efficiency while preserving interactivity properties at the decoder. Then, considering the limited and heterogeneous capabilities of current networks and decoding devices, we propose a novel adaptive solution for IMV based on a layered multiview representation where camera views are organized into layered subsets to offer different levels of navigation quality depending on the different client constraints. We propose an optimal and a reduced computational complexity greedy algorithms that jointly select the different view subsets and their encoding rates. Simulation results show the good performance of our novel algorithms compared to a baseline algorithm, proving that an effective IMV adaptive solution should consider the scene content, the client capabilities and their preferences, in building adaptive systems for multiview navigation. Finally, we build on the solution proposed in our second problem and present a general solution to rate allocation problems in multiview video. In particular, we propose a new algorithm to find the optimal Lagrange multiplier in a Lagrangian-based rate allocation problem. We show the performance of our proposed algorithm in both multiview and monoview video scenarios and show that the proposed method is able to compete with complex state-of-the-art rate control techniques. In summary, this thesis addresses important issues for coding multiview video in the design of efficient IMV systems under resource constraints. Our algorithm to select the optimal PS and QPs in a MVD scenario can improve the quality of the rendered views and it can indeed provide new insights for a deeper understanding of specific IMV coding requirements. We show that our algorithm for a layered representation of multiview video provides an effective adaptive streaming solution for IMV systems with users with limited and heterogeneous capabilities. Finally, our proposed Lagrangian-based rate allocation algorithm with an optimized selection of the Lagrange multiplier represents a general contribution that can be used in multiple video scenarios.

EPFL2015

Splines and wavelets for image warping and projection

Stefan Horbelt

This thesis investigates advanced signal processing concepts and their application to geometric processing and transformations of images and volumes. In the first part, we discuss the class of transformations that project volume data onto a plane using parallel line integrals; it is called the X-ray transform. In computer tomography (CT) the problem is to reconstruct the volume from these projections. We consider a basic setup with parallel projection and a geometry model in which the CT scanner rotates around one main axis of the volume. In this case, the problem is separable and reduces to the reconstruction of parallel images (slices of the volume). Each image is reconstructible from a series of 1D projections taken at different angular positions. The standard reconstruction algorithm is the filtered back-projection (FBP). We propose an alternative discretization of the Radon transform and of its inverse that is based on least-squares approximation and the convolution of splines. It improves the quality of the transform significantly. Next we discuss volume rendering based on the X-ray transform. The volume is represented by a multiresolution wavelet decomposition. The wavelets are projected onto an adaptive multiresolution 2D grid. The multiresolution grid allows to speed up the rendering process especially at coarse scales. In the second part of the thesis, we discuss transformations that warp images. In computer graphics, this is called texture mapping. Simple warps, such as shear, rotation, or zoom, can be computed by least-squares sampling, e.g. again with convolutions of splines. For more general warps there is not an easy continuous solution, if an analytical one exists at all. After a review of existing texture mapping methods, we propose a novel recursive one, which minimizes information loss. For this purpose, the texture is reconstructed from the mapped image and compared to the original texture. This algorithm outperforms the existing methods in terms of quality, but its complexity can be very high. A multiresolution version of the algorithm allows to keep the storage requirements and computational complexity within acceptable range. Fast transmission of textures and 3D models over communication links requires low bitrate and progressive compression. We can achieve very low bitrate if we code textures only with the information necessary for a given view of the 3D scene. If the view changes, the missing information will be transmitted. This results in a progressive bitstream for an animated 3D scene. In contrast to recursive texture mapping, we do not back-project the texture, but come up with a heuristic that predicts the information loss. This concept is called view-dependent scalability and we show how to apply it on DCT-based (as part of MPEG-4) and wavelet-based coders. Last, we inspect the question of how to balance the bit budget of a jointly coded mesh and texture for the progressive and view-dependent transmission of a 3D model. By exhaustive search, we find the rate-distortion optimal path. By marginal analysis we find a close solution but at much lower costs (only two evaluated frames per step as compared to a full search).

EPFL2001