Andreas Grau, Klaus Herrmann and Kurt Rothermel: Scalable Network Emulation - The NET Approach, Journal of Communications - Special Issue on Advances in Communications and Networking II, Academy Publisher, Vol. 7(1), pp. 3-16, January 2012.
[+] Abstract Full text at Academy PublisherNetwork emulation is an efficient method for evaluating distributed applications and communication protocols by combining the benefits of real world experiments and network simulation. The process of network emulation involves the execution of connected virtual nodes running the software under test in a controlled environment. Our Network Emulation Testbed (NET) achieves high scalability by combining efficient node virtualization and adaptive virtual time. In this paper, we provide an overview of our system. First, we introduce our efficient emulation architecture. Second, we present our approaches (NETplace and NETbalance) to minimize the runtime of the network experiments. The idea of NETplace is to minimize the load of the testbed by calculating an initial placement of virtual nodes onto the testbed nodes. During the runtime of the experiment NETbalance adapts this placement to changed resource requirements of the software under test. Finally, we introduce NETcaptain, a graphical user interface to setup, control and visualize network experiments.
Andreas Grau, Klaus Herrmann and Kurt Rothermel: NETbalance: Reducing the Runtime of Network Emulation using Live Migration, In Proc. of the 20th Int'l Conf. on Computer Communication and Networks (ICCCN'11), IEEE Communications Society, pp. 1-6, Maui, HI, USA, August 2011.
[+] Abstract Full text at IEEENetwork emulation is an efficient method for evaluating distributed applications and communication protocols by combining the benefits of real world experiments and network simulation. The process of network emulation involves the execution of connected instances of the software under test (called virtual nodes) in a controlled environment. In previous work, we introduced an approach to minimize the runtime of network emulation experiments based on prior known average resource requirements of virtual nodes. In this paper, we introduce NETbalance, a novel approach to runtime reduction for experiments with unknown or varying resource requirements. NETbalance migrates virtual nodes during an experiment to distribute the load evenly across the physical nodes, avoiding overloaded nodes and exploiting the idle resources on underloaded nodes for speeding up the experiment execution. We make the following contributions: First, we present an emulation architecture for efficiently supporting live migration of virtual nodes. Second, we propose a cost model for determining the runtime reduction achieved through the migration. Third, we introduce an algorithm for calculating placements that minimize the experiment runtime. Our evaluations of the NETbalance prototype show, that it is able to reduce the experiment runtime by up to 70%.
Andreas Grau, Klaus Herrmann and Kurt Rothermel: NETplace: Efficient Runtime Minimization of Network Emulation Experiments, In Proc. of the Int'l Symposium on Performance Evaluation of Computer and Telecommunication Systems (SPECTS'10), [Best Paper Award], IEEE Communications Society, pp. 265-272, Ottawa, ON, Canada, July 2010.
[+] Abstract Full text at IEEENetwork emulation is an efficient method for evaluating distributed applications and communication protocols by combining the benefits of real world experiments and network simulation. The process of network emulation involves execution of thousands of connected virtual nodes running the software under test in a controlled environment. Along with the quality of the experiment results, the runtime of network experiments strongly influences the convenience of users and operators of emulation testbeds. The goal of this paper is, therefore, to minimize the experiment runtime of network emulations. In order to achieve this goal, we make the following contributions in this paper: First, we present a highly scalable emulation architecture to efficiently support network emulation testbeds with multicore CPUs. Second, we propose a detailed and generic cost model for the communication costs of emulation testbeds. Third, we present an efficient placement strategy (NETplace) to assign virtual nodes to physical nodes of the testbed while minimizing the runtime of network experiments. Therefore, we combine graph partitioning and greedy approaches. Our evaluations show that our placement strategy outperforms existing methods by reducing the experiment runtime up to 64%.
Andreas Grau, Klaus Herrmann and Kurt Rothermel: Efficient and Scalable Network Emulation using Adaptive Virtual Time, In Proc. of the 18th Int'l Conf. on Computer Communications and Networks (ICCCN'09), IEEE Communications Society, pp. 1-6, San Francisco, CA, USA, August 2009.
[+] Abstract Full text at IEEEPerformance analysis and functionality testing are major parts of developing distributed software systems. Since the number of communicating software instances heavily influences the behavior of distributed applications and communication protocols, evaluation scenarios have to consider a large number of nodes. Network emulation provides an infrastructure for running these experiments using real prototype implementations in a controllable and realistic environment. Large-scale experiments, however, have a high resource consumption which often exceeds available physical testbed resources. Time dilation allows for reducing the resource demands of a scenario at the expense of the experiment's runtime. However, current approaches only consider a constant time dilation factor, which wastes a lot of resources in case of scenarios with varying load. We propose a framework for adaptive time virtualization that significantly reduces the runtime of experiments by improving resource utilization in network emulation testbeds. In this framework, resource demands are monitored and the time dilation factor is dynamically adapted to the required level. Our evaluation shows that adaptive virtual time in combination with our lightweight node virtualization architecture allows us to increase the possible scenario sizes by more than an order of magnitude and, at the same time, ensure unbiased emulation results. This represents an important contribution to making network emulation systems highly scalable.
Daniel Minder, Andreas Grau, Pedro Jose Marron: Group Formation for Self-Adaptation in Pervasive Systems, In Proc. of the First Int'l Conf. on Autonomic Computing and Communication Systems (Autonomics'07) [invited paper], ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), pp. 16:1-16:10, Rome, Italy, October 2007.
[+] Abstract Full text at ACMAdaptation in Pervasive Computing normally focuses on services or on application behaviour, but the consideration of lower level algorithms in this process can lead to significant performance increase. To perform adaptation of algorithms the concept of context normally used in Pervasive Computing has to be extended. Based on the same context, group formation models are established to group devices with similar relevant context to optimise the adaptation process. In this paper, such group formation models are developed and evaluated. We also draw general conclusions for adaptation.
Steffen Maier, Andreas Grau, Harald Weinschrott, Kurt Rothermel: Scalable Network Emulation: A Comparison of Virtual Routing and Virtual Machines, In Proc. of the IEEE Symposium on Computers and Communications (ISCC'07), IEEE Computer Society, pp. 395-402, Aveiro, Portugal, July 2007.
[+] Abstract Full text at IEEEPerformance analysis is a necessary step during the development of distributed applications and communication protocols. Network emulation testbeds provide synthetic, configurable environments for comparative performance measurements of real implementations. However, realistic scenarios require more communicating nodes than usual testbeds are able to provide. In order to enable scalable network emulation, various concepts for the visualization of nodes have been proposed. The overhead of visualization strongly impacts the total size of a scenario, that can be emulated on a given testbed. However, the overhead of different visualization approaches in the context of network emulation has not been compared directly so far. In this paper, we present a comparison of different virtual machine implementations (Xen, User Mode Linux) and our own virtual routing approach (NET). We discuss qualitative evaluation criteria and present a quantitative evaluation showing the efficiency of each approach in a traditional wired infrastructure-based and in a wireless ad hoc network emulation scenario. Our results give insights on which visualization approach is best suited for which kind of network emulation.
Andreas Grau, Klaus Herrmann and Kurt Rothermel: NET - An Integrated Emulation Environment for Mobile Network Research, Workshop on Simulation and Prototyping Environments for Mobile/Wireless Research, 36th Meeting of VDE/ITG-Fachgruppe 5.2.4, Aachen, Germany, July 2011.
[+] Abstract Slides at University of StuttgartSoftware development of applications running on mobile nodes with wireless network access is challenging. Beside the algorithmic challenges, e.g. handling of high error rates, fast changing topologies, and limited energy; testing these applications is complicated and requires sophisticated tool support. Running real world tests on physical devices is often infeasible due to reproducibility, scalability, security, and cost issues.
In the NET project, we are tackling these problems using distributed network emulation. Therefore, we partition the resources of a PC-cluster into multiple virtual nodes running the Software under Test. These virtual nodes can be connected by wireless or wired networks. The properties of the links between the virtual nodes, such as bandwidth, delay, or loss, is accurately emulated by our tool NETshaper. Using trace-files, generated by arbitrary mobility models, our scenario controller NETcaptain can determine the properties of the wireless links using models varying from simple free-space propagation to complex ray-tracing.
We integrated NETcaptain into the development platform Eclipse, which enables the software developer to execute applications directly on emulated mobile nodes with a few klicks. A scripting engine to automate experiment runs and the visualization of the emulated network allows for evaluation of large scenarios.
In order to ensure accurate emulation results, we have developed the concept of adaptive time virtualization, which allows for slowing down the experiment execution speed in case of upcoming overload. To ensure minimal experiment execution times, we have developed the concepts to calculate an optimal placement of virtual nodes onto the physical nodes of the PC-cluster (NETplace) and to adjust the placement dynamically during the experiment run (NETbalance).
Andreas Grau, Klaus Herrmann and Kurt Rothermel: Exploiting Emulation Testbeds for Security Experiments, Workshop on Experimental platforms for Internet resilience, security and stability research [invited paper], JRC European Commission, Brussels, Belgium, June 2009.
[+] Abstract Full text at University of StuttgartSecurity, resilience and stability test are an important step to protect the information and communication-based digital society. Experimental platforms allow researchers to evaluate todays and tomorrows internet architectures, protocols and applications. However, selecting the most adequate platform is difficult. In this paper we first present a common used approach based on real world testbeds. We identify a couple of problem coming around with this approach and show how emulation-based testbeds can avoids them. Finally we present an hybrid approach to combine the benefits of both systems to provide a scalable, repeatable, controllable, flexible and secure experimental platform.
Andreas Grau, Steffen Maier, Klaus Herrmann and Kurt Rothermel: Time Jails: A Hybrid Approach to Scalable Network Emulation, In: Proc. of the 22nd ACM/IEEE/SCS Workshop on Principles of Advanced and Distributed Simulation (PADS'08), ACM SIGSIM, IEEE IEEE-TCSIM, SCS, pp. 7-14, Rome, Italy, June 2008.
[+] Abstract Full text at IEEEIt is essential to evaluate the performance of newly developed distributed software and network protocols. Network emulation enables reproducible evaluation of unmodified real implementations. Software built for distributed systems, such as a large scale peer-to-peer system, requires evaluation scenarios with thousands of communicating nodes. Two approaches for scaling network emulation to such scenario sizes have been proposed in the literature: node virtualization and time visualization. Node visualization allows maximizing the utilization of standard hardware used for emulation experiments. Time visualization enables trading experiment duration for visually increased resources of the hardware. It stands to reason that a combination of those two approaches may increase scalability even further. However, in existing combinations, either node visualization implies relatively high overhead or time virtualization requires modifications of the test subject implementation. In this paper, we present a novel hybrid approach called time visualized emulation environment (TVEE). It integrates node visualization with low overhead and time virtualization, which is transparent to the execution of test subjects. We introduce visual time based on epochs to enable better dynamic hardware utilization during long lasting experiments. Additionally, a mechanism similar to soft timers ensures an accurate reproduction of network properties in the time visualized emulation. Our evaluations show the accuracy and scalability of time visualized network emulation. Comparing TCP throughput, TVEE outperforms other approaches using an event based visual time by an order of magnitude.
Andreas Grau: Runtime Minimization of Scalable Network Emulation, PhD Thesis, University of Stuttgart, 2012.
[+] Abstract Full text at University of StuttgartNetwork emulation constitutes an approved methodology to evaluate the performance of distributed applications and communication protocols. The approach of network emulation models computer networks by connecting instances of the Software under Test (SuT), representing routers and hosts, using a distributed emulation tool. The emulation tool allows for specifying the parameters of these connections, like bandwidth, delay and loss rate. Therefore, network emulation combines the benefits of network simulation, like controllability and repeatability of network experiments, with the benefits of real world testbeds, like accuracy and realism of running the unmodified implementations of the SuT.
Recently, researchers have spent much effort to increase the scalability of network emulation to allow for the evaluation of distributed systems in large network topologies with thousands of network nodes. Basically two concepts are introduced to reach that goal: node and time virtualization. Node virtualization allows for partitioning the physical nodes of an emulation testbed to run multiple instances of the SuT on each physical node. However, the available resources of the physical nodes limit the number of virtual nodes that can be executed by the physical nodes without overloading the hardware. This overload can be avoided by applying the concept of time virtualization. Executing the network experiment with a virtual time, that runs a factor slower than real time, allows for increasing the resources like CPU and network capacity of the testbed by the same factor. However, the runtime of network experiments is also increased by that factor.
The goal of this work is to reduce the runtime of network experiments and, thus, increase the satisfaction of testbed users and testbed operators. Therefore, this thesis makes the following contributions. First, we present an efficient emulation architecture for testbeds with multi-core processors that provides node and time virtualization and minimizes CPU and memory consumption as well as the communication overhead. Second, we introduce the new concept of adaptive virtual time, that allows for dynamically adjusting the speed of the experiment to the resource requirements during the experiment runtime. Using this approach, the experiment can run with an increased speed during periods of low resource requirements. Third, we provide an accurate testbed model to capture the resource requirements of an experiment. Fourth, based on this model, we introduce an approach to calculate an initial placement of virtual nodes onto the physical nodes that minimizes the experiment runtime based on the experiment specification. Finally, in order to react on varying resource requirements during the experiment, we provide an approach to adapt the placement during the running experiment based on transparent migration of virtual nodes to further reduce the experiment runtime.
The developed concepts are implemented and integrated in our Network Emulation Testbed (NET). Detailed evaluations of our prototype show the efficiency and effectiveness of our concepts to minimize the runtime of experiments based on network emulation.
Andreas Grau: Group Formation for Adaptation Purposes in Wireless Sensor Networks, Diploma Thesis No. 2559, University of Stuttgart, May 2007.
[+] Abstract Full text at University of StuttgartIn the past years, several middleware systems to adapt algorithms in wireless sensor networks have been developed. One representative of such systems, named TinyCubus, focuses on tuning the used algorithms at the runtime of a network. The adaptation analyzes the state of the network and selects best performing parameters for each node. Due to the size of sensor networks, an approach using individual parameters per node does not scale.
A possible solution groups nodes with equal properties to limit the effort to find an optimal network configuration. Based on these groups, it is possible to configure each group with an optimal setting to improve the network performance. During this thesis node properties are identified which can be used for group formation and, based on these, group formation models are developed. The developed models, grouping nodes based on properties like location or density, are using distributed and central coordinated algorithms. During the evaluation, typical sensor network topologies, with regular and randomly placed nodes, are used. The simulation of these scenarios showed that
the performance of networks with group based or individually configured nodes is almost the same.
Andreas Grau, Harald Weinschrott and Christopher Schwarzer: Evaluating the Scalability of Virtual Machines for Use in Computer Network Emulation, Student Report Software Engineering No. 60, University of Stuttgart, October 2006..
[+] Abstract Full text at University of StuttgartEvaluation of distributed applications and network protocols is a difficult task, which can hardly be done by means of real world experiments. Also simulation cannot always be a good approach, because the test subject has to be adapted. Emulation as a third approach, which tries to combine the benefits of both real world experiments and simulation. With increasing hardware performance, emulation based on virtual machines, becomes feasible. Therefore, this case study evaluates the suitability of virtual machine implementations with different designs for use in computer network emulation. A comparison is drawn to an emulation approach based on virtual routing. It is shown that virtual machines are suitable for network emulation, however there are huge performance differences between the different implementations, and huge hardware requirements are needed compared to other computer network emulation approaches.
![[Best Paper Award]](pub.spects10bestpaper.jpg){kind=link}