Combining Multiple Results of a Reverse Engineering Algorithm: Application to the DREAM Five Gene Network Challenge

The output of reverse engineering methods for biological networks is often not a single network prediction, but an ensemble of networks that are consistent with the experimentally measured data. In this paper, we consider the problem of combining the information contained within such an ensemble in order to (1) make more accurate network predictions and (2) estimate the reliability of these predictions. We review existing methods, discuss their limitations, and point out possible research directions towards more advanced methods for this purpose. The potential of considering ensembles of networks, rather than individual inferred networks, is demonstrated by showing how an ensemble voting method achieved winning performance on the Five Gene Network Challenge of the second DREAM conference (Dialogue on Reverse Engineering Assessment and Methods 2007, New York, NY).

Generating Realistic In Silico Gene Networks for Performance Assessment of Reverse Engineering Methods

Dario Floreano, Daniel Marbach, Claudio Mattiussi, Thomas Schaffter

Reverse engineering methods are typically first tested on simulated data from in silico networks, for systematic and efficient performance assessment, before an application to real biological networks. In this paper we present a method for generating biologically plausible in silico networks, which allow realistic performance assessment of network inference algorithms. Instead of using random graph models, which are known to only partly capture the structural properties of biological networks, we generate network structures by extracting modules from known biological interaction networks. Using the yeast transcriptional regulatory network as a test case, we show that extracted modules have a biologically plausible connectivity because they preserve functional and structural properties of the original network. Our method was selected to generate the "gold standard" networks for the gene network reverse engineering challenge of the third DREAM conference (Dialogue on Reverse Engineering Assessment and Methods, Cambridge, MA, 2008).

2009

Evolutionary reverse engineering of gene networks

Daniel Marbach

The expression of genes is controlled by regulatory networks, which perform fundamental information processing and control mechanisms in a cell. Unraveling and modelling these networks will be indispensable to gain a systems-level understanding of biological organisms and genetically related diseases. In this thesis, we present an evolutionary reverse engineering method, which allows to simultaneously infer both the wirings and nonlinear dynamical models of gene regulatory networks from gene expression data. The proposed method reconstructs gene networks by mimicking the natural evolutionary process that constructed them. This is achieved by modelling both the way in which gene networks are encoded in the biological genome, and the different types of mutations and recombinations that drive their evolution, using an artificial genome called Analog Genetic Encoding (AGE). Since AGE mimics the evolutionary forces and constraints that shape biological gene networks, the reconstruction is naturally guided towards biologically plausible solutions. Consequently, the search space is explored more efficiently, and the networks are recovered more reliably, than with alternative methods. We have confirmed the state-of-the-art performance of AGE both in vivo (on real gene networks) and in silico (on simulated networks). In particular, AGE achieved winning performance in the in vivo gene network inference inference challenge of the 2nd DREAM (Dialogue on Reverse Engineering Assessment and Methods) conference, which consisted in predicting the structure of a synthetic-biology gene network in Saccharomyces cerevisiae from time-series data. In vivo performance assessment of network-inference methods is problematic because it is in general not possible to systematically validate predictions, except for few well-characterized gene networks. Consequently, in silico benchmarks are essential to understand the performance of network-inference methods. We have developed tools to generate biologically plausible in silico gene networks, which allow realistic performance assessment of network-inference methods. In contrast to previous in silico benchmarks, we generate network structures by extracting modules from known gene networks of model organisms, instead of using random graphs. Furthermore, we simulate network dynamics using more realistic kinetic models, which include both mRNA and proteins. We have implemented this framework in an open-source Java tool called GeneNetWeaver (GNW). Using GNW we have generated benchmarks for community-wide challenges of the 3rd and 4th DREAM conference (the DREAM in silico network challenges). Here, we assess the performance of 29 network-inference methods, which have been applied independently by participating teams of the DREAM3 challenge. Performance profiling on individual network motifs reveals that current inference methods are affected, to various degrees, by three types of systematic prediction errors. We find that these errors are induced by inaccurate prior assumptions of prevalent gene-network models. The evolutionary reverse engineering approach, which would have ranked 3rd in this challenge, can be used with a wide range of nonlinear models. It could thus provide the necessary framework for the development of models that better approximate different types of gene regulation, thereby enabling ever more accurate reconstruction of gene networks.

EPFL2009

GeneNetWeaver 3.0: realistic benchmark generation and performance profiling of network inference methods

Dario Floreano, Daniel Marbach, Thomas Schaffter

Numerous methods have been developed for inference of gene regulatory networks from expression data, however, their strengths and weaknesses remain poorly understood. Accurate and systematic evaluation of these methods is hampered by the difficulty of constructing adequate benchmarks and the lack of tools for a differentiated analysis of network predictions on such benchmarks. Here, we present the new version (3.0) of GeneNetWeaver (GNW), an open-source tool for in silico benchmark generation and performance profiling of net-work inference methods. GNW can be launched directly from any web browser and it has an intuitive graphical user interface. Using GNW it is possible to generate biologically plausible in silico gene networks and simulated expression data, which can be used as benchmarks for network inference methods. Realistic network structures are generated by extracting modules from known biological interaction networks. These networks are then endowed with dynamics using a kinetic model of transcription and translation, where transcriptional regulation is modeled using a thermodynamic approach allowing for both independent and synergistic interactions. Finally, these models are used to produce synthetic gene expression data by simulating different biological experiments. Simulations can be done either deterministically or stochastically to model internal noise in the dynamics of the networks, and experimental noise can be added using a model of noise observed in microarrays. Another important feature of GNW is systematic evaluation of the predictions from different inference methods on in silico networks in the benchmark. For a set of network predictions from one or several inference methods, GNW automatically generates a comprehensive report in PDF format. These reports include standard metrics used to assess the accuracy of network inference methods such as precision-recall and receiver operating characteristic (ROC) curves. Furthermore, the reports include network motif analysis, where the performance of inference methods is profiled on local connectivity patterns. The network motif analysis often reveals systematic prediction errors, thereby indicating potential ways of network reconstruction improvements. We are using GNW to provide an annual network inference challenge for the DREAM project. In the past three editions, a total of 91 teams submitted about 900 network predictions to evaluate the performance of their methods on GNW-generated benchmarks.