Middleware | EPFL Graph Search

Middleware is a type of computer software that provides services to software applications beyond those available from the operating system. It can be described as "software glue". Middleware makes it easier for software developers to implement communication and input/output, so they can focus on the specific purpose of their application. It gained popularity in the 1980s as a solution to the problem of how to link newer applications to older legacy systems, although the term had been in use since 1968. In distributed applications Middleware (distributed applications) The term is most commonly used for software that enables communication and management of data in distributed applications. An IETF workshop in 2000 defined middleware as "those services found above the transport (i.e. over TCP/IP) layer set of services but below the application environment" (i.e. below application-level APIs). In this more specific sense middleware can be described as the dash ("-") in client-

Cyclone: Unified Stream and Batch Processing

Matus Harvan, Thomas Locher, Ana Claudia Sima

Due to the rising demand for large-scale data processing, there is a growing interest in both batch processing, where large volumes of data are processed offline, and stream processing, where large quantities of streaming data are processed online. The dichotomy between these vastly different computing paradigms has led to the development of substantially different methodologies and systems. As there is an increasing number of applications requiring stream and batch processing, there is a need to bridge this gap and offer support for both paradigms. We introduce a new direction for the unification of stream and batch processing, which, contrary to other proposed approaches, uses a stream processing platform as its foundation and supports batch processing on top. Our proof-of-concept implementation of such a middleware layer, called Cyclone, offers the widely popular MapReduce programming model and translates MapReduce jobs for execution on the underlying streaming platform. Cyclone not only achieves a tight integration of batch and stream processing, our evaluation further shows significant performance gains, in particular for sequential and iterative jobs, which naturally arise in many applications.

Ieee2016

Run-time monitoring and on-line testing of middleware based communication services

With the present day's exponential growth of the (tele-)communications market, the explosion of the number of mobile communication customers, and the tremendous growth of the number of IP hosts, ensuring the reliability of communication services is one of the most challenging tasks in today's software engineering. This Thesis addresses this problem by providing a new method for increasing confidence in the implementation of communication services. We developed an approach for monitoring and testing communications services that are built on top of a middleware. Specifically, we show how to monitor a communication service and how to use the monitored information to test at runtime whether the behavior of the service conforms to constraints. The constraints are extracted from the requirements, expressed using a formal method. The approach advocated in this Thesis consists in combining the power of formal methods, which have reached a high degree of maturity in the academic domain, with a testing methodology, that fit the industrial needs, in order to test the implementation of a communication service. The developer can thus focus on the testing rather then on how the testing is performed. To solve this problem (1) we designed a monitoring mechanism that is able to observe at run-time, the events occuring in the system under scrutiny, (2) designed a testing mechanism that checks at runtime if properties expressed using Temporal Logic, are violated by the execution of the application, (3) implemented a prototype tool, MOTEL (MOnitoring and TEsting tooL). With our approach the tester need only to specify the properties. The other steps involved in the testing processes are performed automatically. According to the content of the properties the instrumentation is added in the system to be tested. This instrumentation reports a consistent order of the events occurring to a central Observer. Within this Observer, the properties, which are independent of the implementation language, are translated into deterministic automata. At run-time these automata are used, according to the events reported by the instrumentation to the Observer, to check if the properties are violated. Thus, our monitoring and testing mechanism fits well in the development process because it does not add noticeable new steps for the developer. In this thesis we put together many concepts and solutions from various domains (distributed systems, middleware, object-oriented development, CORBA, communication services, formal method, Temporal Logic, automata theory, monitoring, testing). By enhancing, developing and combining, and integrating them together, we reached a high level of integration that provides a new, powerful and useful functionality to test the implementation of communication services. We believe that there is not a more valuable argument for the application and practical relevance of our approach in the industry, than the introduction and use of our method and tool in and by the industry. Our partner, Alcatel, is currently integrating MOTEL in the distributed platform PERCO.

EPFL2000

Ianos: An intelligent application oriented scheduling framework for an HPCN grid

Vincent Keller, Pierre Kuonen

We present the architecture and design of the IANOS scheduling framework. The goal of the new Grid scheduling system is to provide a general job submission framework allowing optimal positioning and scheduling of HPCN applications. The scheduling algorithms used to calculate best-suited resources are based on an objective cost function that exploits information on the parameterization of applications and resources. This standard-based, interoperable scheduling framework comprises four general web services and three modules. The middleware is complemented with one client and one admin console. The implementation is based on proposed Grid and Web services standards (WSRF, WS-Agreement, JSDL, and GLUE). It is agnostic to a specific Grid middleware. The beta version of IANOS has been tested and integrated with UNICORE. The validation of IANOS is in progress by running different types of HPCN applications on a large-scale Grid testbed.

Springer, 233 Spring Street, New York, Ny 10013, United States2008

Run-time monitoring and on-line testing of middleware based communication services

EPFL2000