Data processing | EPFL Graph Search

Data processing is the collection and manipulation of digital data to produce meaningful information.
Data processing is a form of information processing, which is the modification (processing) of information in any manner detectable by an observer. The term "Data Processing", or "DP" has also been used to refer to a department within an organization responsible for the operation of data processing programs. Data processing functions Data processing may involve various processes, including:

Validation – Ensuring that supplied data is correct and relevant.
Sorting – "arranging items in some sequence and/or in different sets."
Summarization(statistical) or (automatic) – reducing detailed data to its main points.
Aggregation – combining multiple pieces of data.
Analysis – the "collection, organization, analysis, interpretation and presentation of data."
Reporting – list detail or summary data or computed information.
Clas

Data Processing of tomoPIV measurements of a wind turbine in a wind tunnel

Florian Elias Landerer

This semester project had the focus on the processing of tomographic particle image velocimetry (tomoPIV) measurements. The measurements are based on a new approach which has been developed by LaVision and conducted previously in collaboration with the laboratory WIRE at the EPFL. This enabled for the first time the data processing of a volume containing the downwind wake of a wind turbine model in a wind tunnel. The present work shows the first steps of such a processing of data from these kind of measurements towards a three dimensional representation.

2016

Uncooperative Rendezvous in Space: Design of an Electronic Architecture for High Performance Avionic with Multi Sensor Input and Intensive Data Rate.

Michaël Yannick Juillard

Active Debris Removal missions consist of sending a satellite in space and removing one or more debris from their current orbit. A key challenge is to obtain information about the uncooperative target. By gathering the velocity, position, and rotation of the desired object, the satellite is able to plan its trajectory and define the sequence of approach. It requires the use of a variety of sensors with often a high data rate. For this task, a dedicated payload computer is envisioned with the responsibility of processing the information from the various rendezvous sensors. This component has the goal to provide meaningful information to the main satellite computer about the targeted debris. The focus of this work is on the data processing, the number of elements, and the electrical energy with constraints due to internal communication.First, an avionic testbench was built at the EPFL Space Center to assess early hardware and software architectures. The goal is to enable Hardware-In-the-Loop testing while developing the payload computer. A communication data bus has been implemented between the Platform and the Payload On-Board Computer. Emphasis was placed on the reliability of the high-level protocol and multiple concepts for improvement were analyzed.In a second phase, the testbench has been extended to support other data buses. The aim was to develop a reliable and efficient backup or fall-back data bus in case of failure in the main link. Four data buses have been tested with extensive analyses on their resilience to error and the efficiency of their data exchange.In parallel, extensive work has been performed to develop a simulation and optimization tool. Its goal is to support the design of payload avionic architectures for ADR missions by providing trade-offs and analyses. In the first iteration, a simulator has been created with instances of various high-level elements modeled.The second iteration introduced the additional dimension of optimization. It is trying to determine the best set of instruments and algorithms to use. With this capability, numerous analyses have been conducted on the influence of various parameters linked to the general optimizer behavior. It allows us to better understand the strength and limitations of the tool.The third step regarding the tool was to start its verification. The task has been to develop an Hardware-In-the-Loop architecture to compare its behavior with the results of the optimizer. The implementation of various mock-up algorithms enables the verification of their models in the tool. These analyses guarantee the feasibility of the outputted solution.The last part was dedicated to the creation and analysis of a realistic payload avionic architecture. The goal was to test the capability of the tool with hardware elements inspired by actual components. This work has shown the procedure to efficiently use the tool in the design phase of a mission.In conclusion, the work on the communication between the Payload and the Platform On-Board Computer has brought valuable lessons and experiences to the projects. They can now be used for the establishment of the high-level protocol into ClearSpace-1 flight software. In addition, the optimizer created allows tackling the design of complex payload avionic architecture where mass saving and processing resources are crucial. It is an essential point to develop a highly efficient payload computer for an Active Debris Removal satellite.

EPFL2022

Function Passing: A Model for Typed, Distributed Functional Programming

Jocelyn Boullier, Philipp Haller

The most successful systems for “big data” processing have all adopted functional APIs. We present a new programming model we call function passing designed to provide a more principled substrate on which to build data-centric distributed systems. A key idea is to build up a persistent functional data structure representing transformations on distributed immutable data by passing well-typed serializable functions over the wire and applying them to this distributed data. Thus, the function passing model can be thought of as a persistent functional data structure that is distributed, where transformations to data are stored in its nodes rather than the distributed data itself. The model simplifies failure recovery by design–data is recovered by replaying function applications atop immutable data loaded from stable storage. Deferred evaluation is also central to our model; by incorporating deferred evaluation into our design only at the point of initiating network communication, the function passing model remains easy to reason about while remaining efficient in time and memory. We formalize our programming model in the form of a small-step operational semantics which includes a semantics of functional fault recovery, and we provide an open-source implementation of our model in and for the Scala programming language, along with a case study of several example frameworks and end-user programs written atop of this model.

Assoc Computing Machinery2016

Uncooperative Rendezvous in Space: Design of an Electronic Architecture for High Performance Avionic with Multi Sensor Input and Intensive Data Rate.

Michaël Yannick Juillard

EPFL2022