Adaptive and Robust Partition Learning for Person Retrieval With Policy Gradient

Person retrieval aims at effectively matching the pedestrian images over an extensive database given a specified identity. As extracting effective features is crucial in a high-performance retrieval system, recent significant progress was achieved by part-based models that have constructed robust local representations on top of vertically striped part features. However, this kind of models use predefined partitioning strategies, making the number and size of each partition identical even when input images vary a lot. This unchangeable setting usually leads to less flexibility and robustness in capturing visual variance. The primary reason for such a negative effect is that a fixed partitioning strategy is unable to deal with (a) the significant variance from pose, illumination and viewpoint which is common in a pedestrian image dataset, and (b) also the inference error and misalignment of human bodies introduced by the prepositive pedestrian detection module or human pose estimation module. In this paper, we tackle this problem via introducing the novel Adaptive Partition Network (APN). The APN utilizes deep reinforcement learning and applies an agent to generate optimal partitioning strategies dynamically for different input images. The agent inside the APN is optimized with the policy gradient algorithm and maximizes the reward of choosing the best partition setting. By leveraging the supervision cues from the objective partitioning strategies that are generated on a set of held-out training images, the agent is trained jointly with other parts of APN, which ensures the APN's robustness and generalization ability. Extensive experimental results on multiple datasets, including CUHK03, DukeMTMC and Market-1501, demonstrate the superiority of APN over the state-of-the-art models.

Representation Learning for Multi-relational Data

Eda Bayram

Recent years have witnessed a rise in real-world data captured with rich structural information that can be better depicted by multi-relational or heterogeneous graphs.However, research on relational representation learning has so far mostly focused on the problems arising in simple, homogeneous graphs. Integrating the structural priors provided by multi-relational data may further empower the generalization capacity of representation learning models, yet it still remains an open challenge.Although there is a strong line of works on relational machine learning on knowledge graphs, it is quite concentrated on the task ofcompleting missing edges, which is known as link prediction. In this thesis, we shift the focus away from the well-addressed node and graph classification problems on simple graphs or the link prediction problem on knowledge graphs, and prompt new research questions targeting the representation learning problems that are overlooked in multi-relational data.First, we focus on the problem of node regression on multi-relational graphs, noting that inference of continuous node features across a graph is rather under-studied in the current relational learning research.We propose a novel propagation method which aims to complete missing features at the nodes of a multi-relational and directed graph.Our multi-relational propagation algorithm is composed of iterative neighborhood aggregations which originate from a relational local generative model. Our findings show the benefit of exploiting the inductive bias led by the multi-relational structure of the data.Next, we consider the node attribute completion problem in knowledge graphs, which is relatively unexplored by the knowledge graph reasoning literature.We propose a novel multi-relational attribute propagation method where we harness not only the relational structure of the knowledge graph, but also the dependencies between various types of numerical node attributes relying on a heterogeneous feature space.Our algorithm is framed within a message-passing scheme where the propagation parameters are estimated in advance. We also propose an alternative semi-supervised learning framework where the parameters and the missing node attributes are inferred in an end-to-end fashion.Experimental results on well-known knowledge graph datasets relay the effectiveness of our message-passing approach, which specifies the computational graph by the heterogeneity of the data.Finally, we study graph learning in multi-relational data domain.Unlike the existing structure inference methods, we aim at exploiting and combining each source of relational information provided by the data domain to learn the underlying graph of a set of observations.For this purpose, we employ a multi-layer graph representation which encodes multiple types of relationships between data entities. Then, we propose a mask learning method to infer a specific combination of the layers which reveals the structure of observations.Experiments conducted both on simulated and real-world data suggest that incorporating multi-relational domain knowledge enhances structure inference by boosting its adaptability to a variety of input data conditions.

EPFL2021

Representation Learning for Multi-relational Data

Eda Bayram

EPFL2021