Publications related to Computer vision

Visual tracking has become a very important component in computer vision, but achieving a robust, reliable and real time tracking remains a real challenge.In order to improve the actual state-of-the-art, we choose to study and improve one of the most performing adaptive tracker by detection. We selected Struck [27] for this quality performance and his low computational cost that makes it real time. Inspired by the great successes of binary keypoint descriptors, we choose to apply binary description to a patch. We propose to use Multi-Block Local Binary Pattern (MB-LBP), based on its great success in face detection and description. In this work we present a technique for selecting the best features for tracking. In combination with the feature selection we propose a technique to take into account contextual information in order to increase the robustness of the tracker. We propose a solution to add scale adaptation to the algorithm, and suggest to transpose this technique to add rotation adaptation. Experimentally we validate these techniques showing that we outperform the state-of-art racking algorithms. To do that we use a benchmarking tool using 51 videos and compare our algorithm to 29 algorithms.

2014

In the early stages of visual information processing one of the most fundamental and complex task is segmentation. This process is used to divide images or image sequences into meaningful regions or objects. Such a representation is definitely needed to go from the first rough description of the scene (i.e. light intensity) up to the semantic level, where one is able to discriminate shapes and other high-level visual primitives. The biological visual system performs this extremely difficult task with such ease that is still a major challenge for computer vision to reach this level of speed and accuracy. It is by now widely believed that a method inspired by biological vision has the best chances of accomplishing this goal. Therefore, our main goal in this thesis is to step on biologically inspired schemes to propose new and efficient techniques to solve the image segmentation problem. The starting point and key concept in this research, is the multiresolution paradigm. Indeed, the fact that the Human Visual System (HVS) processes the information in a multiresolution fashion is now well established. Several techniques have been introduced to model this behaviour. Scale-space is one of them and we will thus focus on such a formulation of the segmentation problem. Such a representation is composed by a stack of successive versions of the original data set at coarser scales. It is assumed that, the bigger the scale, the less information referred to local characteristics of the input data will appear. We also impose that general information applying to large scales will last through scale. Taking that into account, it is reasonable to think that local and high resolution scale information can be related to general and low resolution information. This will enable us to extract image structures. We thus propose to link the multiresolution approach to a computational model based on the analysis of the entire range of significant information across scales. The inherent multiscale image structure of the non-linear scale-space guides a coarse to fine segmentation using different strategies. The first one might be termed extrinsic because the non-linear scale-space is analyzed by an external process of linking information through scales. This leads to a completely adaptive scheme that can then be coupled to a labeling procedure in order to define a genuine segmentation. The second one could be labeled intrinsic because it adapts to the image through its inner properties and incorporate this in the labeling step of the main algorithm. These two approaches are particular examples of multiscale image segmentation. We also apply the same scale-space concept to define a segmentation variational formulation that leads to a region merging segmentation. In order to illustrate the proposed segmentation schemes, we apply our algorithms to gray-scale, noisy, color and texture images. Finally, it should be mentioned that the main problem addressed in this thesis, general image segmentation, plays an important role in the image understanding process, especially those that involve natural scenes rich in color and texture. The potential applications of the algorithms are thus numerous: medical image analysis, interactive multimedia developments, object based coding and compression, surveillance applications, image content based browsing and retrieval, etc.

EPFL2006

Constraint consistency techniques for continuous domains

Jamila Sam

Constraint Satisfaction Problems (CSPs) are ubiquitous in computer science. Many problems, ranging from resource allocation and scheduling to fault diagnosis and design, involve constraint satisfaction as an essential component. A CSP is given by a set of variables and constraints on small subsets of these variables. It is solved by finding assignments of values to the variables such that all constraints are satisfied. In its most general form, a CSP is combinatorial and complex. In this thesis, we consider constraint satisfaction problems with variables in continuous, numerical domains. Contrary to most existing techniques, which focus on computing a single optimal solution, we address the problem of computing a compact representation of the space of all solutions that satisfy the constraints. This has the advantage that no optimization criterion has to be formulated beforehand, and that the space of possibilities can be explored systematically. In certain applications, such as diagnosis and design, these advantages are crucial. In consistency techniques, the solution space is represented by labels assigned to individual variables or combinations of variables. When the labeling is globally consistent, each label contains only those values or combinations of values which appear in at least one solution. This kind of labeling is a compact, sound and complete representation of the solution space, and can be combined with other reasoning methods. In practice, computing a globally consistent labeling is too complex. This is usually tackled in two ways. One way is to enforce consistencies locally, using propagation algorithms. This prunes the search space and hence reduces the subsequent search effort. The other way is to identify simplifying properties which guarantee that global consistency can be enforced tractably using local propagation algorithms. When constraints are represented by mathematical expressions, implementing local consistency algorithms is difficult because it requires tools for solving arbitrary systems of equations. In this thesis, we propose to approximate feasible solution regions by 2k-trees, thus providing a means of combining constraints logically rather than numerically. This representation, commonly used in computer vision and image processing, avoids using complex mathematical tools. We propose simple and stable algorithms for computing labels of arbitrary degrees of consistency using this representation. For binary constraints, it is known that simplifying convexity properties reduces the complexity of solving a CSP. These properties guarantee that local degrees of consistency are sufficient to ensure global consistency. We show how, in continuous domains, these results can be generalized to ternary and in fact arbitrary n-ary constraints. This leads to polynomial-time algorithms for computing globally consistent labels for a large class of constraint satisfaction problems with continuous variables. We describe and justify our representation of constraints and our consistency algorithms. We also give a complete analysis of the theoretical results we present. Finally, the developed techniques are illustrated using practical examples.

EPFL1995

Automation of the diagnosis process in the railway system : Detection of defects in concrete sleepers using vision-based machine learning models

Linah Charif

inspectors that walk over the track and check the defects on the rail surface, fasteners and sleepers. In the case of concrete sleepers, rail inspectors classify defects according to their size and occurrence over 20 sleepers. The manual inspection is error prone, it is time consuming and it can be sometimes risky. In an eort to overcome these issues, SBB has developed a project called AISI (Articial Intelligence Streckeninspection) in order to automatize the rail inspection by collecting and analysing images with diagnosis trains equipped with cameras. The objective of this thesis is integrated within the AISI Project, since its goal is the conception of a deep learning model to detect and classify concrete sleepers defects using computer vision and more specically, object detection. The defects studied in this thesis are cracks and akings (or spallings). In the literature, many researchers have tackled the subject of defect detection in concrete, especially in civil infrastructures. The most common methods a combination of image-processing and Machine Learning (image pre-processing and a classier) whereas the latest methods use deep learning (CNN, FCN and RPN, YOLO). The proposed methodology uses a Fast-RCNN model developed with Tensor ow Object Detection API. The evaluation of the model is given on a total of 996 defects and its precision and recall are given. The model shows average results as the number of false positives is high. It has been noted that the labels provided needed to be improved. The proposed approach has a great potential of development since a large amount of training data can be added thanks to the produced relabelling process. Also, the designed pipeline establishes a base that can be adjustable for various algorithms and congurations.

2021

Impulsive noise removal via a blind CNN enhanced by an iterative post-processing

Hatef Otroshi Shahreza, Seyedeh Sahar Sadrizadeh

In digital imaging, especially in the process of data acquisition and transmission, images are often affected by impulsive noise. Therefore, it is essential to remove impulsive noise from images before any further processing. Due to the remarkable performance of deep neural networks in different applications of image processing and computer vision, we present an end-to-end fully convolutional neural network to remove impulsive noise from images. To train our network, we generate a customized dataset with various noise densities in which the highly corrupted images are more frequent. Hence, our convolutional neural network is blind since the percentage of impulsive noise is not required as prior knowledge. Moreover, we define a multi-term loss function to train our network. In particular, we define a novel term to impose the sparsity nature of impulsive noise. Experimental results indicate that our deep learning approach significantly outperforms other state-of-the-art methods in terms of reconstruction quality and speed on a system equipped with GPU. Meanwhile, we introduce a fast iterative method, as a post-processing stage, to further improve the reconstruction quality of our neural network. The proposed post-processing algorithm improves the reconstruction quality in only a fraction of a second. (c) 2021 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license ( http://creativecommons.org/licenses/by/4.0/ )

ELSEVIER2022

Model-based computer vision for supervised landmarking of cephalograms

Thierry Zimmerman

2000

LSSANet: A Long Short Slice-Aware Network for Pulmonary Nodule Detection

Bo Du, Rui Xu, Jiancheng Yang

Convolutional neural networks (CNNs) have been demonstrated to be highly effective in the field of pulmonary nodule detection. However, existing CNN based pulmonary nodule detection methods lack the ability to capture long-range dependencies, which is vital for global information extraction. In computer vision tasks, non-local operations have been widely utilized, but the computational cost could be very high for 3D computed tomography (CT) images. To address this issue, we propose a long short slice-aware network (LSSANet) for the detection of pulmonary nodules. In particular, we develop a new non-local mechanism termed long short slice grouping (LSSG), which splits the compact non-local embeddings into a short-distance slice grouped one and a long-distance slice grouped counterpart. This not only reduces the computational burden, but also keeps long-range dependencies among any elements across slices and in the whole feature map. The proposed LSSG is easy-to-use and can be plugged into many pulmonary nodule detection networks. To verify the performance of LSSANet, we compare with several recently proposed and competitive detection approaches based on 2D/3D CNN. Promising evaluation results on the large-scale PN9 dataset demonstrate the effectiveness of our method. Code is at https:// github.com/Ruixxxx/LSSANet.

SPRINGER INTERNATIONAL PUBLISHING AG2022

Real-time, Automated Characterization of 3D Morphology and Mechanics of Developing Plant Cells

Dimitrios Felekis, Geraldo Mecja, Mahmut Selman Sakar

In this article, we introduce the real-time cellular force microscope (RT-CFM), a high-throughput microrobotic platform for mechanical stimulation and characterization of single cells. We developed computer vision algorithms that fully automate the positioning of target cells and localization of the sensor tip. The control and acquisition architecture dramatically increases the accuracy, speed, and reliability of force measurements. Pollen tubes provide an ideal model system for the study of plant mechanics at the single-cell level. To quantitatively obtain the physical properties of the plant cell wall, we generated topography and stiffness measurements from 3D scans of living, growing pollen tubes. We report techniques for real-time monitoring and analysis of intracellular calcium fluxes during mechanical intervention. Our platform is compatible with various imaging systems and enables a powerful screening technology to facilitate biomechanical and morphological characterization of developing cells.

SAGE Publications2015

Beyond Cameras: Real-Time High-Resolution 3D and Panoramic MultiView Vision Systems with Their Applications

Bilal Demir

The cameras are invented by imitating the human visual system to capture the scene. The camera technologies have been substantially advanced in recent years. 108 MP resolution with 100x hybrid zoom has become standard features for smartphone flagships. In spite of the recent developments, the cameras are still restricted in terms of FoV, depth perception, pixel size, lens system, and more. That is why multicamera systems have progressively become prevalent. The multicamera systems can be a remedy to overcome the limitations that single cameras hold. The FoV can be raised to

360^\circ

through image stitching. The depth can be computed by stereo matching methods. The lens flaws and imperfections can be handled thanks to computer vision and image processing algorithms. Significant computational power is required to process a vast amount of pixels coming from multiple cameras. GPU and FPGA are promising platforms to implement computer vision and video processing applications due to their sophisticated parallelization features. FPGA platforms present more prevailing features, especially for real-time and portable vision applications. FPGAs provide less latency as they hold the connection to image sensors from a low level. FPGA enables the implementation of a system architecture dedicated to the target application. The compact systems can be designed by designing custom PCBs using only necessary ports. Also, FPGAs consume lower power and cheaper option compared to GPUs. On the other hand, GPUs propose more versatility and easy design and upgrade time. In the light of these observations, software and hardware integrated real-time high-resolution multi-view 3D and panoramic systems are presented in the scope of this thesis. Firstly, the depth estimation system is presented in the first part of the thesis. The proposed depth estimation system runs in real-time performance for up to 2K depth map resolution. The system adopts the trinocular scheme to address the occlusion problem. The pixel correspondence challenge in textureless-regions, from which the conventional stereo matching-based depth estimation systems suffer, is tackled by projecting artificial patterns through the integrated pico-projector. The application-specific system architecture ensures the high-performance depth map streaming. Secondly, the drone detection and tracking system is presented. The proposed drone detection system is capable of simultaneously monitoring 360° environment and detecting the drone from a long-range in real-time performance. The distributed architecture design enables ultra-high-resolution image processing. data coming from the hardware part. The GPU design is opted due to its high level of parallelization capability. The proposed system is appropriate to be employed for surveillance applications such as drone detection, passive radar system, vast terrain, and border control applications. Thirdly, the 3D stereoscopic panorama construction system is presented. The proposed system generated 2 separate panoramas for the left and right eyes to achieve 3D perception. The system offers the novel camera arrangement and the 3D panorama generation algorithm. The cameras are positioned to minimize the intra-panorama parallax while raising the inter-panorama parallax to augment 3D perception. Finally, the real-time vision systems are discussed with their pros and cons, and future predictions are presented in the conclusion part of the thesis.

EPFL2020

Graph Representation Learning in Computational Pathology

Guillaume Jaume

Advances in scanning systems have enabled the digitization of pathology slides into Whole-Slide Images (WSIs), opening up opportunities to develop Computational Pathology (CompPath) methods for computer-aided cancer diagnosis and prognosis. CompPath has been primarily developed using models based on Convolutional Neural Networks (CNNs), building on the recent successes in Computer Vision. A series of promising approaches have been proposed for nuclei segmentation and classification, tumor detection, tumor grading, among others. However, CNN-based methods suffer from several limitations. First, it is challenging to model both fine-grained nuclei-level information and long-range inter-glandular dependencies. Second, there is a discrepancy between the pixel-based analysis of CNNs and the histological entity-centered analysis employed for pathological diagnosis, which in turn can hinder model transparency. Third, the inherent complexity of training networks on large histology images with limited annotations constrains its learning capabilities.Instead, we propose an analytical paradigm shift, where we view and analyze histology images as a set of biological entities interacting with each other. Specifically, an image is represented as an entity-graph where nodes depict biological entities and edges encode interactions between these entities. Entity-graphs are further processed by a Graph Neural Network (GNN) model to jointly encode the entity morphological attributes and topological distribution, towards tissue phenotyping. In this thesis, we study three research directions in CompPath, namely, scalability, interpretability and explainability, and weakly-supervised learning.First, histology images are orders of magnitude larger than natural images, where diagnostically relevant regions can represent only a fraction of the image. We propose a scalable hierarchical cell-to-tissue representation (HACT) and GNN model, HACT-Net, for learning on arbitrary large inputs. We show the capabilities of HACT-Net on our proposed BR ACS dataset, the largest cohort to date for breast tumor Regions-of-Interest subtyping.Second, computer-aided diagnostic tools must be transparent and their decision-making process justified. By shifting the analysis from pixel- to entity-based, we make the input space interpretable for pathologists that can better relate to the model input. We further propose entity-centric graph explainers, exemplified with cell-graph model explainability, along with novel metrics to evaluate explanations based on entity-level pathological concepts.Third, acquiring ground-truth data to train deep learning systems requires pathologists to provide specific annotations, which is time-consuming, expensive, and subject to inter- and intra-observer variability. We therefore propose WholeSIGHT, a method that reduces annotation requirements to WSI-level labels only, for joint classification and segmentation of WSIs. We show the capabilities of WholeSIGHT for Gleason pattern segmentation and grading on multi-source WSI prostate datasets. The generalization properties of WholeSIGHT are further evaluated on unseen cohorts, and compared to Bayesian variants to strengthen the estimation of the model uncertainty. Finally, we introduce HistoCartography, a novel python library designed to accelerate the development of graph analytics in CompPath.

EPFL2022