Evaluation of a Web-based learning environment for Hands-on experimentation

The evaluation of learning environments depends highly on the evaluation instruments used and on the position of the evaluation in the design cycle. In this article, we present a use-case report of a summative evaluation of a Web-based learning environment for engineering education that allows small groups of students to conduct hands-on experimentation sessions with local and remote access to laboratory devices. This use-case illustrates an evaluation methodology based on a choice of usability engineering methods. In particular, we define specific metrics that take into account students’ interactions with some of the components of the environment. The metrics defined can easily be extended to any e-learning project involving production and sharing of artifacts in a shared workspace. The methodology is illustrated with the results of a preliminary evaluation with 30 students.

Evaluation of a Web-Based Training Environment for Hands-On Experimentation

Denis Gillet, Anh Vu Nguyen Ngoc, Stéphane Sire

The role of evaluation in software development has been well established. It is of first importance in usability engineering. Over the last years, many interactive systems have benefited from user evaluation, including e- learning applications. As with other interactive systems, designers of Web- based training environments can benefit from the results of an evaluation to detect the strengths and weaknesses of their application, and to prepare the next version. However, it is not obvious to evaluate educational software. It requires the transposition of the usability concepts of effectiveness, efficiency and satisfaction to a domain that is at the crossroads of multiple sub-domains: interactive systems, collaborative work and learning theories. In this article, we present a use-case of the evaluation of a flexible learning environment for hands on experiments. The first part of the article presents the flexible environment proposed to engineering students at the Swiss federal institute of Technology. It also defines the objectives of a user evaluation according to some of the design choices we made. In this evaluation, we have two types of objectives. First, we want to check if the flexibility permitted by a Web-based training environment is really put into practice by the students, if it has some effects on the place and the time at which they choose to work. Second, we want to check if a new component that we have introduced in our environment for sharing and commenting experimental results, the eJournal, really supports the flexible learning scenario and enhances collaborative learning. The second part of the article examines some of the most common evaluation methods used in interactive system design and in Computer Supported Cooperative Work. For each of these methods, such as checklists, questionnaires, log analysis and interviews, we describe how they could contribute to meet our evaluation objectives as defined in the first part. Finally, the third part of the article reports on our user evaluation of the flexible environment described in the first part. This reports describes first the evaluation methodology that we have selected. It describes the evaluation tools selected among the tools introduced in the second part, and the adaptations we made to fit our purposes. In particular, it describes some of the constraints imposed by experimenting with a panel of students pursuing their academic year at the same time. The reports then give some of the results of the evaluation we made with 30 students in automatic control using our prototype during the winter semester 2002/2003. The contribution of the article is to expose, through a use-case, the challenges of the evaluation of a Web-based training environment for sharing laboratory experiments. We think this is a necessary step towards the development of standard practices for evaluating e-learning applications.

2003

Development of a hyphenated analytical methodology for the fractionation and characterization of environmental colloids

Enrica Alasonati

Natural colloids are defined as macromolecules and nanoparticles with sizes ranging between 1 nm and 1 µm. Colloidal pool is complex "ill-defined" mixture of humic substances, exopolymeric substances (EPS), inorganic oxides, clays, etc. Aquatic colloids have been recognized as key mediators in the removal of trace elements, facilitating contaminant transport, organic carbon cycling and micronutrient bioavailability. The residence times of the colloidal pool (and associate contaminants) can be order of magnitude different from those of the particulate fraction, but unlike the "truly" dissolved fraction, can be subjected to different coagulation and sedimentation processes. Colloids may also influence metal bioavailability by affecting free metal ion concentrations, constraining diffusion and affecting the dissociation kinetics of biological processes. The important environmental role of natural colloids is largely influenced by their surface area and closely related colloidal size. Nonetheless, it is currently unclear to what extend colloidal size can influence their behaviour and contaminant (e.g. toxic metals) binding properties. In our view this is partially related to the difficulties in determination of the colloidal size distributions and their impact in metal behaviour, mainly due to the inherent heterogeneity of the environmental colloids and their low concentrations in the environment. Furthermore, in addition to average parameters, there is a lack of appropriate sample collection and processing procedures, as well as analytical techniques to measure simultaneously the size (or related parameter) distributions of the colloid associated metal. The objective of the present work was therefore to develop sensitive analytical procedures for the fractionation of natural colloids and to apply them to improve the current understanding on the fate of the colloids and associated toxic metals in the freshwaters. To achieve these goals, the capabilities of asymmetric flow-field flow fractionation (aFlFFF) coupled to several detectors was explored. aFlFFF is a versatile technique separating colloids according to their diffusion coefficient. The system was hyphenated with multi-angle laser light scattering (MALS), differential refractive index (DRI), UV detector and, when necessary, high resolution inductively coupled plasma mass spectrometry (ICP-MS). Correct interpretation of the experimental results and extrapolation of meaningful molecular parameters by using an analytical tool with such a level of complexity requires improvement of the knowledge of colloids behaviour in the aFlFFF channel and careful optimization of the separation conditions. Given the very complex and dynamic nature of environmental colloids, different experimental procedures was first optimized and validate under lab conditions by using "standardized" colloids (e.g. alginate and EPS isolate from lab bacterial cultures), then applied to characterize colloidal organic matter (COM) isolated from natural water systems. The influence of critical operating parameters, such as crossflow rate, carrier composition and concentration, and sample load, on the alginate retention was carefully evaluated. Combined information obtained simultaneously by DRI and MALS detectors, allowed to set the appropriate combination of optimal parameters for the characterization of alginate, as a model COM. The metal distributions of metal-alginate complexes were probed by aFlFFF - ICP-MS: average values and continuous distributions of molar masses, radius of gyration and hydrodynamic radius, critical for understanding their role as carriers of metal pollutants, were evaluated in presence of lead or cadmium and compared to those in metal-free solutions of alginate. Furthermore, the molecular characteristic of exopolymeric substances (EPS) secreted by bacterium Sinorhizobium meliloti wild type and exoY strains, which is deficient in the production of the succinoglycan, were studied. EPS were collected in the extracellular environment or desorbed from the cell surface using different extraction protocols. EPS fractions with different chemical characteristic and molar mass were found, depending on the strain and extraction protocol. Molecular parameters as the distributions and average values of molar mass and gyration radius, and the polydispersity index have been measured and compared with data obtained by the chemical characterization. Size exclusion chromatography coupled to UV detection was also employed to provide additional information on the small class of size. Moreover, the effect of the cadmium, lead and calcium addition to exopolymeric substances excreted in the extracellular environment was investigated, with the aim to study how does EPS composition, size and mass distributions, can affect the trace metal binding properties of EPS and how the metal addition affects the EPS conformation. Obtained results revealed that cadmium and calcium bound preferentially on the low-size species population and the amount of the bound metal increased with the increasing of the concentration of added metal. In contrast, lead bound on preferentially larger classes of EPS size. The aFlFFF separation parameters were also optimized for the fractionation and characterization of humic substances, as other COM component, and applied to probe the temporal and spatial variability in the size and mass characteristics of COM samples collected in the Amazon River basin. COM characterization demonstrated the existence of a clear decrease of size of the colloidal material when passing from small to large watercourses (spatial variation). No significant changes were found in size distributions of samples collected in different period of the year, during the low and high flow, respectively (temporal variation). The analytical methodology, adapted to determine not only mean characteristics, but the population distributions, developed in the present work, allowed to obtain new information about the colloidal organic matter size distribution, which play utmost role in colloids behaviour and fate in the environment, in particular as toxic metal carriers.

EPFL2009

Synthesis, modeling, and experimental validation of distributed robotic search

James Pugh

The work of this thesis centers around the research subject of distributed robotic search. Within the field of distributed robotic systems, a task of particular interest is attempting to locate one or more targets in a possibly unknown environment. While numerous studies have proposed and analyzed methods to accomplish this, there is currently a lack of a strong foundation of tools and techniques that can be used to facilitate the development and evaluation of different approaches to distributed robotic search. In this work, we aim to provide such a foundation through tools, methods, models, and analysis of experimentation. An often overlooked aspect of the distributed robotic research process is the development and analysis of tools and modules to be used with robotic systems. These may include plug-ins for realistic robotic simulators, software/hardware systems to track multiple mobile robots in real-time, extension boards for robotic platforms, and the robots themselves. Along with other tools developed and used for this work, we focus particularly on the development, characterization, and validation of a fast, accurate on-board system for relative positioning and communication between robots, a capability which is critical for effective distributed robotic search. Designing individual robot controllers to generate a specific group behavior is a difficult and often counter-intuitive process. A possible alternative to hand-crafting distributed search controllers is automatic synthesis using machine-learning techniques. We explore the effectiveness of using a noise-resistant version of the Particle Swarm Optimization algorithm to optimize the weights of an embedded Artificial Neural Network, allowing the robot to learn obstacle avoidance behavior (a common benchmark for robotic learning techniques); we find that this technique appears to offer superior performance as compared to the canonical approach of using Genetic Algorithms for this type of learning. A method for faster learning using distributed evaluation in a robot team is tested and is found to offer comparable performance using only a fraction of the original learning time. This technique can be used for fast, effective learning and adaptation in a distributed robotic system performing search. The process of designing and analyzing algorithms for distributed robotic systems can be greatly facilitated if models are available to describe the dynamics of the algorithm at an abstract level. Inspired by previous examples in the distributed robotic field, we work to design a model of robotic search that captures the system at different levels of abstraction, ranging from accurately recreating the details of individual robots to describing the entire system as an indivisible whole. To capture the entire search process, we model both the exploration phase, where robots cover an environment in an effort to detect traces of targets, and the localization phase, where robots use target emission sensing to navigate towards the target. The utility of our models is demonstrated by using them to develop an effective technique for the declaration phase of search, where robots decide that a target has been accurately localized and announce its position. In distributed robotics research, it is important that techniques developed with abstracted simulations and models are ultimately validated using real robotic platforms in order to verify their correctness. In that spirit, we run systematic sound search experiments using teams of up to ten real robots. These experiments utilize the tools developed throughout the research process, demonstrate the utility of our learning technique for fast search adaptation, and serve to validate our models of distributed robotic localization. In addition, they allow us to analyze and better understand the subtle dynamics of the search process, providing information which should be useful for any future work on distributed robotic search.

EPFL2008