Run-time monitoring and on-line testing of middleware based communication services

With the present day's exponential growth of the (tele-)communications market, the explosion of the number of mobile communication customers, and the tremendous growth of the number of IP hosts, ensuring the reliability of communication services is one of the most challenging tasks in today's software engineering. This Thesis addresses this problem by providing a new method for increasing confidence in the implementation of communication services. We developed an approach for monitoring and testing communications services that are built on top of a middleware. Specifically, we show how to monitor a communication service and how to use the monitored information to test at runtime whether the behavior of the service conforms to constraints. The constraints are extracted from the requirements, expressed using a formal method. The approach advocated in this Thesis consists in combining the power of formal methods, which have reached a high degree of maturity in the academic domain, with a testing methodology, that fit the industrial needs, in order to test the implementation of a communication service. The developer can thus focus on the testing rather then on how the testing is performed. To solve this problem (1) we designed a monitoring mechanism that is able to observe at run-time, the events occuring in the system under scrutiny, (2) designed a testing mechanism that checks at runtime if properties expressed using Temporal Logic, are violated by the execution of the application, (3) implemented a prototype tool, MOTEL (MOnitoring and TEsting tooL). With our approach the tester need only to specify the properties. The other steps involved in the testing processes are performed automatically. According to the content of the properties the instrumentation is added in the system to be tested. This instrumentation reports a consistent order of the events occurring to a central Observer. Within this Observer, the properties, which are independent of the implementation language, are translated into deterministic automata. At run-time these automata are used, according to the events reported by the instrumentation to the Observer, to check if the properties are violated. Thus, our monitoring and testing mechanism fits well in the development process because it does not add noticeable new steps for the developer. In this thesis we put together many concepts and solutions from various domains (distributed systems, middleware, object-oriented development, CORBA, communication services, formal method, Temporal Logic, automata theory, monitoring, testing). By enhancing, developing and combining, and integrating them together, we reached a high level of integration that provides a new, powerful and useful functionality to test the implementation of communication services. We believe that there is not a more valuable argument for the application and practical relevance of our approach in the industry, than the introduction and use of our method and tool in and by the industry. Our partner, Alcatel, is currently integrating MOTEL in the distributed platform PERCO.