Testing Scientific Software: A Systematic Literature Review.

CONTEXT: Scientific software plays an important role in critical decision making, for example making weather predictions based on climate models, and computation of evidence for research publications. Recently, scientists have had to retract publications due to errors caused by software faults. Systematic testing can identify such faults in code. OBJECTIVE: This study aims to identify specific challenges, proposed solutions, and unsolved problems faced when testing scientific software. METHOD: We conducted a systematic literature survey to identify and analyze relevant literature. We identified 62 studies that provided relevant information about testing scientific software. RESULTS: We found that challenges faced when testing scientific software fall into two main categories: (1) testing challenges that occur due to characteristics of scientific software such as oracle problems and (2) testing challenges that occur due to cultural differences between scientists and the software engineering community such as viewing the code and the model that it implements as inseparable entities. In addition, we identified methods to potentially overcome these challenges and their limitations. Finally we describe unsolved challenges and how software engineering researchers and practitioners can help to overcome them. CONCLUSIONS: Scientific software presents special challenges for testing. Specifically, cultural differences between scientist developers and software engineers, along with the characteristics of the scientific software make testing more difficult. Existing techniques such as code clone detection can help to improve the testing process. Software engineers should consider special challenges posed by scientific software such as oracle problems when developing testing techniques.

Cell phone-based system (Chaak) for surveillance of immatures of dengue virus mosquito vectors.

Capture of surveillance data on mobile devices and rapid transfer of such data from these devices into an electronic database or data management and decision support systems promote timely data analyses and public health response during disease outbreaks. Mobile data capture is used increasingly for malaria surveillance and holds great promise for surveillance of other neglected tropical diseases. We focused on mosquito-borne dengue, with the primary aims of: 1) developing and field-testing a cell phone-based system (called Chaak) for capture of data relating to the surveillance of the mosquito immature stages, and 2) assessing, in the dengue endemic setting of Mérida, Mexico, the cost-effectiveness of this new technology versus paper-based data collection. Chaak includes a desktop component, where a manager selects premises to be surveyed for mosquito immatures, and a cell phone component, where the surveyor receives the assigned tasks and captures the data. Data collected on the cell phone can be transferred to a central database through different modes of transmission, including near-real time where data are transferred immediately (e.g., over the Internet) or by first storing data on the cell phone for future transmission. Spatial data are handled in a novel, semantically driven, geographic information system. Compared with a pen-and-paper-based method, use of Chaak improved the accuracy and increased the speed of data transcription into an electronic database. The cost-effectiveness of using the Chaak system will depend largely on the up-front cost of purchasing cell phones and the recurring cost of data transfer over a cellular network.

Tool support for software lookup table optimization.

A number of scientific applications are performance-limited by expressions that repeatedly call costly elementary functions. Lookup table (LUT) optimization accelerates the evaluation of such functions by reusing previously computed results. LUT methods can speed up applications that tolerate an approximation of function results, thereby achieving a high level of fuzzy reuse. One problem with LUT optimization is the difficulty of controlling the tradeoff between performance and accuracy. The current practice of manual LUT optimization adds programming effort by requiring extensive experimentation to make this tradeoff, and such hand tuning can obfuscate algorithms. In this paper we describe a methodology and tool implementation to improve the application of software LUT optimization. Our Mesa tool implements source-to-source transformations for C or C++ code to automate the tedious and error-prone aspects of LUT generation such as domain profiling, error analysis, and code generation. We evaluate Mesa with five scientific applications. Our results show a performance improvement of 3.0 × and 6.9 × for two molecular biology algorithms, 1.4 × for a molecular dynamics program, 2.1 × to 2.8 × for a neural network application, and 4.6 × for a hydrology calculation. We find that Mesa enables LUT optimization with more control over accuracy and less effort than manual approaches.