Laboratoire de systemes intelligents

Laboratory

Arm-wrist haptic sleeve for drone teleoperation

Dario Floreano, Matteo Macchini, Vivek Ramachandran

Teleoperators rely on both visual and haptic feedback to perform drone teleoperation tasks, such as obstacle avoidance. Haptic feedback becomes essential when visual feedback is compromised, either due to visual occlusions or poor depth perception. However, haptic interfaces, are often bulky because they require heavy actuators to provide force feedback. The bulkiness reduces user mobility and makes these interfaces unsuitable for prolonged use. Here, we propose a wearable haptic sleeve that encompasses the wrist and elbow joints of a human arm. The two joint rotations control the motion of a drone in a simulated environment along a horizontal plane. The sleeve is composed of modular electroadhesive clutches that block the joint movement when the drone is in the vicinity of an obstacle. The clutches are lightweight (27 g), require low power (~1 mW) to operate, and can be mounted on the user without affecting the user's mobility. A motor learning subject study is conducted to navigate a drone through a hole in a wall where the depth perception of visual feedback is compromised. The results of the study show that subjects trained with the wearable haptic sleeve learnt the drone obstacle avoidance task and retained the necessary motor skills after haptic training, compared to subjects who received only visual feedback and were unable to learn the motor task.

2021

Tales from a Robotic World. How Intelligent Machines Will Shape Our Future

Dario Floreano, Nicola Nosengo

Stories from the future of intelligent machines—from rescue drones to robot spouses—and accounts of cutting-edge research that could make it all possible. Tech prognosticators promised us robots—autonomous humanoids that could carry out any number of tasks. Instead, we have robot vacuum cleaners. But, as Dario Floreano and Nicola Nosengo report, advances in robotics could bring those rosy predictions closer to reality. A new generation of robots, directly inspired by the intelligence and bodies of living organisms, will be able not only to process data but to interact physically with humans and the environment. In this book, Floreano, a roboticist, and Nosengo, a science writer, bring us tales from the future of intelligent machines—from rescue drones to robot spouses—along with accounts of the cutting-edge research that could make it all possible. These stories from the not-so-distant future show us robots that can be used for mitigating effects of climate change, providing healthcare, working with humans on the factory floor, and more. Floreano and Nosengo tell us how an application of swarm robotics could protect Venice from flooding, how drones could reduce traffic on the congested streets of mega-cities like Hong Kong, and how a “long-term relationship model” robot could supply sex, love, and companionship. After each fictional scenario, they explain the technologies that underlie it, describing advances in such areas as soft robotics, swarm robotics, aerial and mobile robotics, humanoid robots, wearable robots, and even biohybrid robots based on living cells. Robotics technology is no silver bullet for all the world's problems—but it can help us tackle some of the most pressing challenges we face.

MIT Press2022

Leveraging Spiking Deep Neural Networks to Understand the Neural Mechanisms Underlying Selective Attention

Davide Zambrano

Spatial attention enhances sensory processing of goal-relevant information and improves perceptual sensitivity. Yet, the specific neural mechanisms underlying the effects of spatial attention on performance are still contested. Here, we examine different attention mechanisms in spiking deep convolutional neural networks. We directly contrast effects of precision (internal noise suppression) and two different gain modulation mechanisms on performance on a visual search task with complex real-world images. Unlike standard artificial neurons, biological neurons have saturating activation functions, permitting implementation of attentional gain as gain on a neuron's input or on its outgoing connection. We show that modulating the connection is most effective in selectively enhancing information processing by redistributing spiking activity and by introducing additional task-relevant information, as shown by representational similarity analyses. Precision only produced minor attentional effects in performance. Our results, which mirror empirical findings, show that it is possible to adjudicate between attention mechanisms using more biologically realistic models and natural stimuli.

MIT PRESS2022

Low-noise high-dynamic-range single-photon avalanche diodes with integrated PQAR circuit in a standard 55 nm BCD

Claudio Bruschini, Edoardo Charbon, Francesco Gramuglia, Pouyan Keshavarzian, Ekin Kizilkan, Myung Jae Lee

Single-photon avalanche diode (SPAD) based sensors and systems enable a variety of applications in biomedical, automotive, consumer, and security domains. While several established standard technologies, which can facilitate the design of SPAD-based systems are already in existence, challenges remain for the development of deep sub-micron monolithic integration of circuits and SPADs. In this work, we present SPADs along with pixel circuits in a standard GF 55 nm BCDL process. Two different designs demonstrate the flexibility allowed by the technology for a variety of applications. Both shallow and deep junction SPADs present excellent noise performance of less than 1 cps/mu m(2) at 3 V excess bias. An integrated passive-quench active-recharge (PQAR) circuit is used in conjunction with the SPADs, which enables a dead time of less than 2 ns, easily allowing for high dynamic range applications that require > 100 Mcps such as quantum communication and information technologies. The deep and shallow junction SPADs demonstrate an afterpulsing probability of < 0.5 % and < 2 % at 3V excess bias, respectively. The dead time is adjustable through analog control of the active-recharge circuit, allowing for afterpulsing reduction to below 0.1 % while maintaining Mcps operation. The shallow junction, which has a breakdown voltage of about 18 V and a peak sensitivity at 430 nm is particularly interesting for applications requiring low supply voltage, whereas the deep SPAD, which demonstrates > 4 % photon detection probability (PDP) at 940 nm, can be implemented in LiDAR sensors that require enhanced sensitivity at near-infrared (NIR) wavelengths. The measured timing jitter of both SPADs is < 50 ps FWHM at 3 V excess bias and 780 nm.

SPIE-INT SOC OPTICAL ENGINEERING2022

NIR Fluorescence lifetime macroscopic imaging with a time-gated SPAD camera

Claudio Bruschini, Edoardo Charbon, Arin Can Ülkü

The performance of SwissSPAD2 (SS2), a large scale, widefield time-gated CMOS SPAD imager developed for fluorescence lifetime imaging, has recently been described in the context of visible range and fluorescence lifetime imaging microscopy (FLIM) of dyes with lifetimes in the 2.5 - 4 ns range. Here, we explore its capabilities in the NIR regime relevant for small animal imaging, where its sensitivity is lower and typical NIR fluorescent dye lifetimes are much shorter (1 ns or less). We carry out this study in a simple macroscopic imaging setup based on a compact NIR picosecond pulsed laser, an engineered diffuser-based illumination optics, and NIR optimized imaging lens suitable for well-plate or small animal imaging. Because laser repetition rates can vary between models, but the synchronization signal frequency accepted by SS2 is fixed to 20 MHz, we first checked that a simple frequency-division scheme enables data recording for different laser repetition rates. Next, we acquired data using different time gate widths, including gates with duration longer than the laser period, and analyzed the resulting data using both standard nonlinear least-square fit (NLSF) and phasor analysis. We show that the fixed synchronization rate and large gate widths characterizing SS2 (10 ns and over) are not an obstacle to accurately extracting lifetime in the 1 ns range and to distinguishing between close lifetimes. In summary, SS2 and similar very large gated SPAD imagers appear as a versatile alternative to other widefield time-resolved detectors for NIR fluorescence lifetime imaging, including preclinical molecular applications.

SPIE-INT SOC OPTICAL ENGINEERING2022

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions with Drones

Adriana Arza Valdes, David Atienza Alonso, Ricardo Andres Chavarriaga Lozano, Dario Floreano, Ping-Keng Jao

In search and rescue missions, drone operations are challenging and cognitively demanding. High levels of cognitive workload can affect rescuers’ performance, leading to failure with catastrophic outcomes. To face this problem, we propose a machine learning algorithm for real-time cognitive workload monitoring to understand if a search and rescue operator has to be replaced or if more resources are required. Our multimodal cognitive workload monitoring model combines the information of 25 features extracted from physiological signals, such as respiration, electrocardiogram, photoplethysmogram, and skin temperature, acquired in a noninvasive way. To reduce both subject and day inter-variability of the signals, we explore different feature normalization techniques, and introduce a novel weighted-learning method based on support vector machines suitable for subject-specific optimizations. On an unseen test set acquired from 34 volunteers, our proposed subject-specific model is able to distinguish between low and high cognitive workloads with an average accuracy of 87.3% and 91.2% while controlling a drone simulator using both a traditional controller and a new-generation controller, respectively.

2022

A Lightweight Device for Energy Harvesting from Power Lines with a Fixed-Wing UAV

Dario Floreano, William John Stewart

Modern UAVs (Unmanned Aerial Vehicles) suffer from restrictions in flight range and time due to battery capacity limitations. One way to overcome these limitations is to harvest energy from overhead power lines. A growing body of scientific literature has documented the development of energy harvesters on multirotor UAVs. However, to date, there have been no implementations on a fixed-wing UAV. In this paper we build upon a previously developed passive perching mechanism for fixed-wing UAVs by incorporating into it a split-core transformer for energy harvesting. We validate the perching mechanism as well as battery recharging in a realistic outdoor environment. We find that a minimum speed of 1.25m/s is required to successfully perch on the power line. This is below the stall speed of the UAV used in this project. Furthermore, experiments verified that no damage was done to the power line during perching, as the induced forces were equivalent to 1m/s wind gusts. Characterization of the energy harvesting shows that with a maximum of 3.3W of power from the harvester, we can fully charge a 2200mAh battery in 2.2 hours. Weighing only 420g, our device is the lightest energy harvester developed to date.

IEEE2022

In vitro and in vivo NIR fluorescence lifetime imaging with a time-gated SPAD camera

Claudio Bruschini, Edoardo Charbon, Arin Can Ülkü

Near-infrared (NIR) fluorescence lifetime imaging (FLI) provides a unique contrast mechanism to monitor biological parameters and molecular events in vivo. Single-photon avalanche diode (SPAD) cameras have been recently demonstrated in FLI microscopy (FLIM) applications, but their suitability for in vivo macroscopic FLI (MFLI) in deep tissues remains to be demonstrated. Herein, we report in vivo NIR MFLI measurement with SwissSPAD2, a large time-gated SPAD camera. We first benchmark its performance in well-controlled in vitro experiments, ranging from monitoring environmental effects on fluorescence lifetime, to quantifying Forster resonant energy transfer (FRET) between dyes. Next, we use it for in vivo studies of target-drug engagement in live and intact tumor xenografts using FRET. Information obtained with SwissSPAD2 was successfully compared to that obtained with a gated intensified charge-coupled device (ICCD) camera, using two different approaches. Our results demonstrate that SPAD cameras offer a powerful technology for in vivo preclinical applications in the NIR window. (C) 2022 Optica Publishing Group under the terms of the Optica Open Access Publishing Agreement

Optica Publishing Group2022

Sub-10 ps Minimum Ionizing Particle Detection With Geiger-Mode APDs

Claudio Bruschini, Edoardo Charbon, Carlo Alberto Fenoglio, Francesco Gramuglia, Emanuele Ripiccini, Ming-Lo Wu

Major advances in silicon pixel detectors, with outstanding timing performance, have recently attracted significant attention in the community. In this work we present and discuss the use of state-of-the-art Geiger-mode APDs, also known as single-photon avalanche diodes (SPADs), for the detection of minimum ionizing particles (MIPs) with best-in-class timing resolution. The SPADs were implemented in standard CMOS technology and integrated with on-chip quenching and recharge circuitry. Two devices in coincidence allowed to measure the time-of-flight of 180 GeV/c momentum pions with a coincidence time resolution of 22 ps FWHM (9.4 ps Gaussian sigma). Radiation hardness measurements, also presented here, highlight the suitability of this family of devices for a wide range of high energy physics (HEP) applications.

FRONTIERS MEDIA SA2022

Laboratoire de systemes intelligents

Arm-wrist haptic sleeve for drone teleoperation

Tales from a Robotic World. How Intelligent Machines Will Shape Our Future

Leveraging Spiking Deep Neural Networks to Understand the Neural Mechanisms Underlying Selective Attention

Low-noise high-dynamic-range single-photon avalanche diodes with integrated PQAR circuit in a standard 55 nm BCD

NIR Fluorescence lifetime macroscopic imaging with a time-gated SPAD camera

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions with Drones

A Lightweight Device for Energy Harvesting from Power Lines with a Fixed-Wing UAV

NeuroMechFly: an integrative simulation testbed for studying Drosophila behavioral control

In vitro and in vivo NIR fluorescence lifetime imaging with a time-gated SPAD camera

Sub-10 ps Minimum Ionizing Particle Detection With Geiger-Mode APDs

Arm-wrist haptic sleeve for drone teleoperation

Low-noise high-dynamic-range single-photon avalanche diodes with integrated PQAR circuit in a standard 55 nm BCD

Tales from a Robotic World. How Intelligent Machines Will Shape Our Future

Leveraging Spiking Deep Neural Networks to Understand the Neural Mechanisms Underlying Selective Attention

NIR Fluorescence lifetime macroscopic imaging with a time-gated SPAD camera

NeuroMechFly: an integrative simulation testbed for studying Drosophila behavioral control

Machine-Learning Based Monitoring of Cognitive Workload in Rescue Missions with Drones

A Lightweight Device for Energy Harvesting from Power Lines with a Fixed-Wing UAV

In vitro and in vivo NIR fluorescence lifetime imaging with a time-gated SPAD camera

Sub-10 ps Minimum Ionizing Particle Detection With Geiger-Mode APDs