Exploiting environmental signals to enable policy correlation in large-scale decentralized systems

Can artificial agents benefit from human conventions? Human societies manage to successfully self-organize and resolve the tragedy of the commons in common-pool resources, in spite of the bleak prediction of non-cooperative game theory. On top of that, real-world problems are inherently large-scale and of low observability. One key concept that facilitates human coordination in such settings is the use of conventions. Inspired by human behavior, we investigate the learning dynamics and emergence of temporal conventions, focusing on common-pool resources. Extra emphasis was given in designing a realistic evaluation setting: (a) environment dynamics are modeled on real-world fisheries, (b) we assume decentralized learning, where agents can observe only their own history, and (c) we run large-scale simulations (up to 64 agents). Uncoupled policies and low observability make cooperation hard to achieve; as the number of agents grow, the probability of taking a correct gradient direction decreases exponentially. By introducing an arbitrary common signal (e.g., date, time, or any periodic set of numbers) as a means to couple the learning process, we show that temporal conventions can emerge and agents reach sustainable harvesting strategies. The introduction of the signal consistently improves the social welfare (by 258% on average, up to 3306%), the range of environmental parameters where sustainability can be achieved (by 46% on average, up to 300%), and the convergence speed in low abundance settings (by 13% on average, up to 53%).

Technology Strategy in Dynamic Environments: A Computational Analysis of the Automation of Routines, the Organization of AI, and the Evaluation of Technology Risk

In this dissertation, I develop theory and evidence to argue that new technologies are central to how firms organize to create and capture value. I use computational methods such as reinforcement learning and probabilistic topic modeling to investigate three topics: the automation of routines, the organization of artificial intelligence (AI), and the evaluation of technology risk. Overall, I argue that new technologies are not a panacea for the firm but require deliberate strategic planning to manage the potential downsides of myopic automation, AI interdependencies, and the disclosure of technology risks.In the first essay, I argue that while automation can increase productivity by reducing the costs of coordinating individuals, the automation of routines can also incur an indirect opportunity cost due to slow adaptation to environmental change. I develop a reinforcement learning simulation to model the impact of automation on the returns from the division of labor in dynamic environments and to show how automation incurs opportunity costs through lost learning and slow adaptation. Moreover, automation can be suboptimal when it brings about myopic behavior, i.e. high returns from the division of labor in the short term, but negative returns in the long term. Given the simulation results, I argue that firms need dynamic routines to simultaneously balance learning and automation. I open-source the simulation platform as OrgSim-RL on GitHub.In the second essay, I argue that a data-driven culture - what I define as a Data Clan - can help to coordinate complex interdependencies between AI components within a firm. I analyze in-depth semi-structured interview data with a hierarchical stochastic block model (hSBM) and hand-coding to find that managers focus primarily on building a strong culture and establishing high-quality data assets when allocating resources to AI initiatives. Given the results, I inductively develop implications for theory and argue that the emergence of a Data Clan can be a governance mechanism to reduce coordination frictions and build a competitive advantage in the age of AI.In the third essay, I argue that investors require a higher initial return to take on more technology risk disclosure during an IPO. I quantify the magnitude of disclosed risk and the risk disclosure topics based on a latent Dirichlet allocation (LDA) topic model of IPO prospectus text and find a return-for-risk association between text-based technology risk disclosure and underpricing. The study also finds evidence that owning granted patents is associated with a lower return-for-risk association, suggesting that intellectual property allows the disclosure of risk without losing the competitive advantage. I open-source the code for quantifying risk disclosure as RiskyData-LDA on GitHub.In summary, this dissertation develops theory and finds evidence across three essays to argue that leveraging new technologies requires deliberate strategic planning to manage potential downsides of new technologies, such as the opportunity costs of automation, coordination costs, and costs associated with raising capital. The results suggest three mitigating solutions: dynamic routines to balance learning and automation, a Data Clan to improve coordination, and disclosure through patents to reduce underpricing.

EPFL2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

One of the main goal of Artificial Intelligence is to develop models capable of providing valuable predictions in real-world environments. In particular, Machine Learning (ML) seeks to design such models by learning from examples coming from this same environment. However, the real world is most of the time not static, and the environment in which the model will be used can differ from the one in which it is trained. It is hence desirable to design models that are robust to changes of environments. This encapsulates a large family of topics in ML, such as adversarial robustness, meta-learning, domain adaptation and others, depending on the way the environment is perturbed.In this dissertation, we focus on methods for training models whose performance does not drastically degrade when applied to environments differing from the one the model has been trained in. Various types of environmental changes will be treated, differing in their structure or magnitude. Each setup defines a certain kind of robustness to certain environmental changes, and leads to a certain optimization problem to be solved. We consider 3 different setups, and propose algorithms for solving each associated problem using 3 different types of methods, namely, min-max optimization (Chapter 2), regularization (Chapter 3) and variable selection (Chapter 4).Leveraging the framework of distributionally robust optimization, which phrases the problem of robust training as a min-max optimization problem, we first aim to train robust models by directly solving the associated min-max problem. This is done by exploiting recent work on game theory as well as first-order sampling algorithms based on Langevin dynamics. Using this approach, we propose a method for training robust agents in the scope of Reinforcement Learning.We then treat the case of adversarial robustness, i.e., robustness to small arbitrary perturbation of the model's input. It is known that neural networks trained using classical optimization methods are particularly sensitive to this type of perturbations. The adversarial robustness of a model is tightly connected to its smoothness, which is quantified by its so-called Lipschitz constant. This constant measures how much the model's output changes upon any bounded input perturbation. We hence develop a method to estimate an upper bound on the Lipschitz constant of neural networks via polynomial optimization, which can serve as a robustness certificate against adversarial attacks. We then propose to penalize the Lipschitz constant during training by minimizing the 1-path-norm of the neural network, and we develop an algorithm for solving the resulting regularized problem by efficiently computing the proximal operator of the 1-path-norm term, which is non-smooth and non-convex.Finally, we consider a scenario where the environmental changes can be arbitrary large (as opposed to adversarial robustness), but need to preserve a certain causal structure. Recent works have demonstrated interesting connections between robustness and the use of causal variables. Assuming that certain mechanisms remain invariant under some change of the environment, it has been shown that knowing the underlying causal structure of the data at hand allows to train models that are invariant to such changes. Unfortunately, in many cases, the causal structure is unknown. We thus propose a causal discovery algorithm from observational data in the case of non-linear additive model.

EPFL2022

Introducing Productive Engagement for Social Robots Supporting Learning

Jauwairia Nasir

We have all been one such student or seen such students who can maintain the 'good student' image while playing a video game under the table or those loyal backbenchers, seemingly always distracted, who then ace their exams. These intricacies of human behaviors are just a few examples of what makes it non-trivial and challenging even for expert teachers to know how students' visible behaviors relate with learning. As research investigates ways in which robots and AI can support teachers and students, it is faced with the same challenge of inferring students' engagement; thus, making the investigation of this topic increasingly popular in educational HRI. The state of the art usually explores the relationship between the robot behaviors and the engagement state of the learner while assuming a linear relationship between engagement and learning. However, is it correct to assume that to maximize learning, one needs to maximize engagement? Furthermore, conventional supervised engagement models require human annotators to get labels. This not only is laborious but can also introduce subjectivity. Can we have machine-learning engagement models where annotations do not rely on human annotators? Additionally, with the increase in open-ended learning activities which by design employ the 'learning by failing' paradigm, in-task performance can not be the best measure for learning. Can we instead rely on multi-modal behaviors? In an effort to cater for these challenges, this thesis dives deep to identify and quantify the relationship between learning and engagement, which we term as Productive Engagement (PE). In order to develop, design, and evaluate our PE framework, (1) we first designed and developed an open-ended collaborative learning activity that served as a platform for evaluating different robot variants over time. With 98 children interacting with the baseline version from 2 international Swiss schools, we showed that in-task performance and learning are indeed not correlated. Thus, this showed the importance of not being limited to robot interventions that affect only superficial measures of students' learning. (2) Then, with learner's multi-modal behaviors, we showed that indeed there is a hidden link between learner's behaviors and learning that can be quantified, i.e., validating the proposed concept of Productive Engagement. (3) This quantifiable link surfaced three collaborative multi-modal learner profiles, by using a forward and backward clustering and classification technique, two of which are linked to higher learning. This technique gave a possibility to surface data driven labels for engagement; thus, evading the process of human annotations. We then identified similarities and differences between these learner profiles both at an aggregate and at the temporal level. (4) Based on (3), we constructed a PE score that can either be directly used as an assessment metric by a social robot in real-time or as data driven labels for building more sophisticated regression models. (5) With the learner profiles and the PE score, we designed and evaluated more advanced robot variants for the final studies with ~160 students from 7 international Swiss schools. With the design of different robot variants that employ knowledge about the learner's skills conducive to learning, rather than domain knowledge, in order to provide interventions; we provided a complementary perspective on the role of social robots in educational settings.

EPFL2022

Introducing Productive Engagement for Social Robots Supporting Learning

Jauwairia Nasir

EPFL2022

Technology Strategy in Dynamic Environments: A Computational Analysis of the Automation of Routines, the Organization of AI, and the Evaluation of Technology Risk

EPFL2022

Predicting in Uncertain Environments: Methods for Robust Machine Learning

Paul Thierry Yves Rolland

EPFL2022