Specification and Implementation of Dynamic Web Site Benchmarks

The absence of benchmarks for Web sites with dynamic content has been a major impediment to research in this area. We describe three benchmarks for evaluating the performance of Web sites with dynamic content. The benchmarks model three common types of dynamic content Web sites with widely varying application characteristics: an online bookstore, an auction site, and a bulletin board. For the online bookstore, we use the TPCW specification. For the auction site and the bulletin board, we provide our own specification, modeled after ebay.com and slahdot.org, respectively. For each benchmark we describe the design of the database and the interactions provided by the Web server. We have implemented these three benchmarks with a variety of methods for building dynamic-content applications, including PHP, Java servlets and EJB (Enterprise Java Beans). In all cases, we use commonly used open-source software. We also provide a client emulator that allows a dynamic content Web server to be driven with various workloads. Our implementations are available freely from our Web site for other researchers to use. These benchmarks can be used for research in dynamic Web and application server design. In this paper, we provide one example of such possible use, namely discovering the bottlenecks for applications in a particular server configuration. Other possible uses include studies of clustering and caching for dynamic content, comparison of different application implementation methods, and studying the effect of different workload characteristics on the performance of servers. With these benchmarks we hope to provide a common reference point for studies in these areas.

Performance Comparison of Middleware Architectures for Generating Dynamic Web Content

Emmanuel Cecchet, Sameh Mohamed Elnikety, Willy Zwaenepoel

On-line services are making increasing use of dynamically generated Web content. Serving dynamic content is more complex than serving static content. Besides a Web server, it typically involves a server-side application and a database to generate and store the dynamic content. A number of standard mechanisms have evolved to generate dynamic content. We evaluate three specific mechanisms in common use: PHP, Java servlets, and Enterprise Java Beans (EJB). These mechanisms represent three different architectures for generating dynamic content. PHP scripts are tied to the Web server and require writing explicit database queries. Java servlets execute in a different process from the Web server, allowing them to be located on a separate machine for better load balancing. The database queries are written explicitly, as in PHP, but in certain circumstances the Java synchronization primitives can be used to perform locking, reducing database lock contention and the amount of communication between servlets and the database. Enterprise Java Beans (EJB) provide several services and facilities. In particular, many of the database queries can be generated automatically. We measure the performance of these three architectures using two application benchmarks: an online bookstore and an auction site. These benchmarks represent common applications for dynamic content and stress different parts of a dynamic content Web server. The auction site stresses the server front-end, while the online bookstore stresses the server back-end. For all measurements, we use widely available open-source software (the Apache Web server, Tomcat servlet engine, JOnAS EJB server, and MySQL relational database). While Java servlets are less efficient than PHP, their ability to execute on a different machine from the Web server and their ability to perform synchronization leads to better performance when the front-end is the bottleneck or when there is database lock contention. EJB facilities and services come at the cost of lower performance than both PHP and Java servlets.

2003

Performance and Scalability of EJB Applications

Emmanuel Cecchet, Willy Zwaenepoel

We investigate the combined effect of application implementation method, container design, and efficiency of communication layers on the performance scalability of J2EE application servers by detailed measurement and profiling of an auction site server. We have implemented five versions of the auction site. The first version uses stateless session beans, making only minimal use of the services provided by the Enterprise JavaBeans (EJB) container. Two versions use entity beans, one with containermanaged persistence and the other with bean-managed persistence. The fourth version applies the session façade pattern, using session beans as a façade to access entity beans. The last version uses EJB 2.0 local interfaces with the session façade pattern. We evaluate these different implementations on two popular open-source EJB containers with orthogonal designs. JBoss uses dynamic proxies to generate the container classes at run time, making an extensive use of reflection. JOnAS precompiles classes during deployment, minimizing the use of reflection at run time. We also evaluate the communication optimizations provided by each of these EJB containers. The most important factor in determining performance is the application implementation method. EJB applications with session beans perform as well as a Java servlets-only implementation and an order-of-magnitude better than most of the implementations based on entity beans. The fine-granularity access exposed by the entity beans limits scalability. Use of session façade beans improves performance for entity beans, but only if local communication is very efficient or EJB 2.0 local interfaces are used. Otherwise, session façade beans degrade performance. For the implementation using session beans, communication cost forms the major component of the execution time on the EJB server. The design of the container has little effect on performance. With entity beans, the design of the container becomes important. In particular, the cost of reflection affects performance. For implementations using session façade beans, local communication cost is critically important. EJB 2.0 local interfaces improve the performance by avoiding the communication layers for local communications.

2002

Stepwise refinement of formal specifications based on logical formulae

One of the steps making it possible to increase the quality and the reliability of the software executing on distributed systems consists of the use of methods of software engineering that are known as formal. The majority of the formal methods currently existing correspond in fact more to formal specifications languages than to methods themselves. This is due to the fact that the two fundamental aspects which are: the logic of use of the language and the coverage of the software life cycle are not, for the majority, defined. The development by stepwise refinement is one of the means making it possible to define these two aspects. This thesis aims to the definition of the concepts of refinement and implementation of model-oriented formal specifications. It brings a methodological base making it possible to use such a specifications language during a development by stepwise refinements and during the implementation stage. This thesis defines, initially, a theoretical framework for the refinement and the implementation of formal specifications. The main idea consists in associating a contract with each specification. A contract explicitly represents the whole of the properties of the specification which it is necessary to preserve at the time of a refinement of this specification. To show that a concrete specification refines some abstract specification, it is then a matter of showing that the contract of the concrete specification is sufficient to ensure the properties corresponding to the contract of the abstract specification. The second part of this thesis consists in applying this theoretical framework in the context of the CO-OPN/2 language. CO-OPN/2 is an object-oriented formal specifications language founded on algebraic specifications and Petri nets. Thus, definitions of the concepts of contracts, refinement and implementation are proposed for this language. The contracts are expressed using the Hennessy-Milner temporal logic (HML). This logic is used in the theory of test provided with language CO-OPN/2. Thus, the verification of the contractual properties, as well as the verification of the stages of refinement are facilitated. Refinement and implementation are controlled semantically by the satisfaction of the contracts; syntactically, a renaming is authorised. We specifically study the implementation using the Java programming language. We show how to specify classes of the Java programming language using language CO-OPN/2, so that the last stage of the process of refinement leads to a specification entirely built using CO-OPN/2 components specifying Java classes. The stage of implementation in the Java language itself is thus facilitated. The third part of this thesis shows how it is possible to practically verify that a CO-OPN/2 specification satisfies its own contract, that a stage of refinement is correctly carried out, and finally that the stage of implementation is correctly performed. These verifications are carried out using the theory of the test provided with language CO-OPN/2. Finally, the last part of this thesis illustrates the cogency of this approach by applying it to a complete and detailed case study. A distributed Java application is developped according to the method introduced for the CO-OPN/2 language. Refinement is guided mainly by the satisfaction of functional requirements and by constraints of design integrating the concept of client/server architecture. Lastly, the stages chosen in the refinement process of this development make it possible to study aspects specific to distributed applications, and to propose generic schemas for the design of such applications.

EPFL1999