Touradj Ebrahimi

Related publications (542)

The JPEG AI Standard: Providing Efficient Human and Machine Visual Data Consumption

Touradj Ebrahimi

The Joint Photographic Experts Group (JPEG) AI learning-based image coding system is an ongoing joint standardization effort between International Organization for Standardization (ISO), International Electrotechnical Commission (IEC), and International Telecommunication Union - Telecommunication Sector (ITU-T) for the development of the first image coding standard based on machine learning (a subset of artificial intelligence), offering a single stream, compact compressed domain representation, targeting both human visualization and machine consumption. The main motivation for this upcoming standard is the excellent performance of tools based on deep neural networks, in image coding, computer vision, and image processing tasks. The JPEG AI aims to develop an image coding standard addressing the needs of a wide range of applications such as cloud storage, visual surveillance, autonomous vehicles and devices, image collection storage and management, live monitoring of visual data, and media distribution. This article presents and discusses the rationale behind the JPEG AI vision, notably how this new standardization initiative aims to shape the future of image coding, through relevant application-driven use cases. The JPEG AI requirements, the JPEG AI history, and current status are also presented, offering a glimpse of the development of the first learning-based image coding standard.

IEEE COMPUTER SOC2023

A novel assessment framework for learning-based deepfake detectors in realistic conditions

Touradj Ebrahimi, Yuhang Lu

Detecting manipulations in facial images and video has become an increasingly popular topic in media forensics community. At the same time, deep convolutional neural networks have achieved exceptional results on deepfake detection tasks. Despite the remarkable progress, the performance of such detectors is often evaluated on benchmarks under constrained and non-realistic situations. In fact, current assessment and ranking approaches employed in related benchmarks or competitions are unreliable. The impact of conventional distortions and processing operations found in image and video processing workflows, such as compression, noise, and enhancement, is not sufficiently evaluated. This paper proposes a more rigorous framework to assess the performance of learning-based deepfake detectors in more realistic situations. This framework can serve as a broad benchmarking approach for both general model performance assessment and the ranking of proponents in a competition. In addition, a stochastic degradation-based data augmentation strategy driven by realistic processing operations is designed, which significantly improves the generalization ability of two deepfake detectors.

2022

A Novel Framework for Assessment of Deep Face Recognition Systems in Realistic Conditions

Luca Barras, Touradj Ebrahimi, Yuhang Lu

Deep convolutional neural networks have shown remarkable results on face recognition (FR). Despite their significant progress, the performance of current face recognition techniques is often assessed in benchmarks under not always realistic conditions. The impact of outdoor environment, post-processing operations, and unexpected human behaviors are not sufficiently studied. This paper proposes a universal methodology that systematically measures the impact of various types of influencing factors on the performance of FR methods. Based on extensive experiments and analysis, the key influencing factors are identified, highlighting the need for suitable precautions on modern FR systems. The robustness of the state-of-the-art deep face recognition techniques is further benchmarked with our assessment framework. The best-performing CNN architecture and discriminative loss function are identified, in order to better guide the deployment of an FR system in real world.

2022

In this paper, we introduce a new solution and underlying architecture that allows remote participants to interact with hosts in a broadcast scenario. To achieve this, background extraction is first applied to video received from remote participants to extract their faces and bodies. Considering that the video from remote participants are usually of lower resolutions when compared to content produced by professional cameras in production, we propose to scale the extracted video with a super-resolution module. Finally, the processed video from remote participants are merged with studio video and streamed to audiences. Given the real-time and high-quality requirements, both background extraction and super-resolution modules are learning-based solutions and run on GPUs. The proposed solution has been deployed in the Advance Mixed Reality (AdMiRe) project. The objective and subjective assessment results show that the proposed solution works well in real world applications.

SPIE2022

Effective know-your-customer method for secure and trustworthy non-fungible tokens in media assets

Touradj Ebrahimi, Clément Sanh

Non-fungible tokens (NFTs) are becoming very popular in a large number of applications ranging from copyright protection to monetization of both physical and digital assets. It is however a fact that NFTs suffer from a large number of security issues that create a lack of trust in solutions based on them. In this paper, we provide an overview of some of the most critical security challenges in media assets in form of visual content and then propose a specific solution for one among them, namely, secure person identification used in the context of Know-Your-Customer (KYC) with emphasis on liveness detection. The solution includes an authentication procedure that matches a selfie photo to a photograph of an identity document (ID). The system runs through a series of steps. First, detection is applied to extract faces from the selfie and the ID. Then a face comparison is performed to assess if they belong to the same person. While these two procedures are standard in KYC, a liveness check is also included so as to increase the security. The latter ensures that the user undergoing identity verification is in front of the camera and not a fraudster attempting to impersonate another individual. The system instructs the user to perform gestures such as waving hands or tilting head in front of the camera. The algorithmic detection of these actions during the live feed will reveal whether or not the user is carrying out the instructed activities. Performance of the proposed solution is then assessed under varying conditions.

SPIE-INT SOC OPTICAL ENGINEERING2022

The use of point clouds to digitally represent three-dimensional objects with both geometry and color attributes is rapidly increasing in several applications. Since storage and transmission of uncompressed point cloud data are often impractical, several lossy compression algorithms have been proposed, each exhibiting specific types of visual distortion. This creates a challenging environment for objective quality metrics, which might be effective only on a restricted number of distortion types and contexts. To evaluate the performance of the most recent objective quality metrics in predicting distortions as perceived by humans, a benchmarking study is conducted using subjective scores from observers examining models distorted with a conventional as well as a learning-based compression method, while rendering them on both a traditional flat monitor and an eye-sensing light field display. The results are then analyzed and conclusions are drawn on the correlation between recent state-of-the-art objective quality metrics and human perception on two different types of visualization devices.

IEEE2022

Point cloud representation is a popular modality to code immersive 3D contents. Several solutions and standards have been recently proposed in order to efficiently compress the large volume of data that point clouds require, in order to make them feasible for real-life applications. Recent studies adopting learning-based methods for point cloud compression have demonstrated high compression efficiency specially when compared to the conventional compression standards. However, they are mostly evaluated either on geometry or color separately, and no learning-based joint codec with performance comparable to state-of-the-art methods have been proposed. In this paper, we propose an integrated learned coding architecture by joining a previously proposed geometry coding module based on three-dimensional convolutional layers with a color compression method relying on graph Fourier transform (GFT) using a learning-based mean and scale hyperprior to compress the obtained coefficients. Evaluation on a test set with dense point clouds shows that the proposed method outperforms GPCC and achieves competitive performance with V-PCC when evaluated with state-of-the-art objective quality metrics.

2022

Latent Space Slicing for Enhanced Entropy Modeling in Learning-Based Point Cloud Geometry Compression

Touradj Ebrahimi

The growing adoption of point clouds as an imaging modality has stimulated the search for efficient solutions for compression. Learning-based algorithms have been reporting increasingly better performance and are drawing the attention from the research community and standardisation groups such as JPEG and MPEG. Learned autoencoder architectures based on 3D convolutional layers are popular solutions and have demonstrated higher performance when adopting latent space entropy modeling based on learned hyperpriors. We propose an enhanced entropy model that takes into account both the hyperprior and previously encoded latent features to estimate the mean and scale of compressed features. The obtained results show a large increase in performance, with a BD PSNR gain of 5.75dB when compared to the Octree coding module in G-PCC for the D2 PSNR metric. We also perform an ablation study to quantify the impact of network parameters in the performance of the model, drawing useful insights for future research.

IEEE2022

On combining denoising with learning-based image decoding

Touradj Ebrahimi, Michela Testolina

Noise is an intrinsic part of any sensor and is present, in various degrees, in any content that has been captured in real life environments. In imaging applications, several pre- and post-processing solutions have been proposed to cope with noise in captured images. More recently, learning-based solutions have shown impressive results in image enhancement in general, and in image denoising in particular. In this paper, we review multiple novel solutions for image denoising in the compressed domain, by integrating denoising operations into the decoder of a learning-based compression method. The paper starts by explaining the advantages of such an approach from different points of view. We then describe the proposed solutions, including both blind and non-blind methods, comparing them to state of the art methods. Finally, conclusions are drawn from the obtained results, summarizing the advantages and drawbacks of each method.

SPIE2022

On the impact of spatial rendering on point cloud subjective visual quality assessment

Touradj Ebrahimi, Michela Testolina

2022