International Journal of Electrical and Computer Engineering (IJECE) Vol.
No.
August 2017, pp.
ISSN: 2088-8708.
DOI: 10.
11591/ijece.
A Review on Web Application Testing and its Current Research Directions Rajya Lakshmi1.
Suguna Mallika2 Department of Computer Science and Engineering.
JNTUK-UCEN.
India Department of Computer Science and Engineering.
CVR College of Engineering.
India
Article Info
ABSTRACT
Article history:
Testing is an important part of every software development process on which companies devote considerable time and effort.
The burgeoning web applications and their proliferating economic significance in the society made the area of web application testing an area of acute importance.
The web applications generally tend to take faster and quicker release cycles making their testing very challenging.
The main issues in testing are cost efficiency and bug detection efficiency.
Coverage-based testing is the process of ensuring exercise of specific program elements.
Coverage measurement helps determine the AithoroughnessAn of testing achieved.
An avalanche of tools, techniques, frameworks came into existence to ascertain the quality of web applications.
A comparative study of some of the prominent tools, techniques and models for web application testing is presented.
This work highlights the current research directions of some of the web application testing techniques.
Received Jun 6, 2017 Revised Apr 8, 2017 Accepted Apr 22, 2017 Keyword:
Automated testing Coverage testing Testing techniques Testing tools Web application testing Copyright A 2017 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Suguna Mallika.
Department of Computer Science and Engineering.
CVR College of Engineering.
Vastunagar.
Mangalpally.
Ibramhimpatnam.
District.
India.
Email: suguna.
kishore@gmail.
INTRODUCTION
With the advent of internet revolution and the colossal rise in the development of web applications as well as their corresponding usage, it is becoming mandatory for quality testing of web applications.
Web Application Testing is gaining importance given the major stake of economic relevance in the contemporary The cost of fixing a bug is directly proportional to the time of its discovery.
The longer the time it takes to unearth a bug, the costlier it becomes to fix it as the software would have been distributed or under use by the customers.
The mammoth customer base and a global distribution of the customers in the case of web application testing (WAT), software bugs detected late proved pricy for the applications owners in the past and will continue to be in the future .
The versatility of web applications is a predominant feature which is making the testing of web applications a tough job .
However, high quality testing would always contribute in better customer retention and loyalty for web applications, thereby directly contributing to a thriving and sustaining business.
The principal feature of web applications that differentiates it from traditional testing on desktop is that web applications are completely heterogeneous in nature at various levels .
, .
Features of web applications which discriminate them from customary desktop applications are highlighted in Figure 1.
The enormous customer base, a heterogeneous execution environment, heterogeneous languages used for component development, heterogeneous operating systems, faster maintenance rate, multi-tier architecture, transactional concurrency, dynamic state changes .
ike for e.
, pressing of back button on Journal homepage: http://iaesjournal.
com/online/index.
php/IJECE IJECE
ISSN: 2088-8708
browser et.
) are the principal features which discriminate web applications from standalone applications .
Such complexities inherent with web applications make the testing of web applications a challenging job thus establishing a clear need for more sophistication in the WAT.
To quote an example.
PayPal had to pay huge compensation for its customers for a small service outage resulting from a faulty upgradation of its website .
Figure 1.
Features of Web Applications A focus on all the major web application testing challenges is presented in Section 2, different architectures proposed for web application testing earlier are consolidated in section 3, various models available for web application testing are presented in section 4, several methods, techniques for web application testing are presented in Section 5.
Some of the top testing tools available for automated testing of web applications and a comparative study of the applications are presented in Section 6.
WEB APPLICATION TESTING CHALLENGES
With a unique set of characteristics for web applications, the challenges involved in testing them are also multi fold .
To unearth a failure, it is necessary to test the web application in a combination of input with state.
With its close linkage to the environment in which it is running, web application testing poses critical challenges .
However the running environment has a predominant effect on the nonfunctional requirements like availability, performance, compatibility, stability, accessibility, usability, security etc.
Specifically, the heterogeneity involved in the various languages used, execution environments, technologies and operating system, make the testing of web applications a critical issue to handle .
efficient test suite should comprise of a those set of test cases which perform coverage testing of all possible combination of parameters .
Some of the major testing challenges with web applications are presented in Figure 2.
A Review on Web Application Testing and its Current Research Directions (D.
Rajya Lakshm.
A ISSN: 2088-8708 Figure 2.
Web Application Specific Faults WEB BASED APPLICATION ARCHITECTURES Paydar and Kahani proposed a framework which works on the principle that there should be a formal format for test specification which is understood by the executor module behaving like a web browser and performing HTTP based interactions.
An automated test execution framework which is extendible has been developed.
The framework is distributed supporting functional, security, load, stress and performance testing .
Kung presented a framework on BDI architecture i.
the Beliefs, the Desires, and the Intentions Architecture.
According to this, beliefs, desires and intentions are associated with the agents of the All kinds of testing including the performance, functional, coverage, state and structural etc.
supported by this architecture .
Li et.
al, proposed a model driven testing framework for testing web applications called MDWATP, which takes as input the model of the web application and generates test cases automatically.
This framework is also useful to perform regression testing of the web applications .
Yang et.
, proposed an architecture which makes use of six subsystems namely Source Document Analysis Subsystem (SDAS).
Test Management Subsystem (TMS).
Test Development Subsystem (TDS).
Test Execution Subsystem (TES).
Test Failure Analysis Subsystem (TFAS).
Test Measurement Subsystem (TMES) for performing the complete testing process including the activities extraction of control flow from source code, repository access interfaces, test execution, test validation, test failure analysis, and measurement of test coverage .
MODEL BASED TESTING OF WEB APPLICATIONS
A model based mutation testing where mutations are introduced into an a priori tested model of the web application using either a state chart diagram or an event sequence graph.
Any seepage in the faults unearthed during the regular model based testing are exposed in the MBMT approach .
Mining workflow models has been proposed by Schur.
Roth, and Zeller where a set of DOM trees are created in abundance keeping in view the incomplete set of execution traces.
The additional DOM trees are created by keeping in view the application behavior .
A User Representation Model Graph (URMG) is constructed based on Customer Behavior Model Graph (CBMG) which is in turn constructed from a customerAos access logs and automated tests are run on the web applications .
An object oriented testing model for testing web applications has been proposed wherein an Object Relationship Diagram(ORD) is devised embedding all the web application components including the IJECE Vol.
No.
August 2017 : 2132 Ae 2141
IJECE
ISSN: 2088-8708
relationships Navigation.
Request.
Response and Redirect using which the testers comprehend the structures and dependencies among the various web application components.
The Page Navigation Diagram (PND) and the Object State Diagram (OSD) are used for performing navigation testing and structural testing of web applications .
A navigation model of the transition of pages is constructed based on the hyperlinks present in the web pages.
Dynamic requests of users are modeled separately .
Realistic Usage Model (RUM) has been proposed which is used by a Simple Work Load Model (SLM).
SLM relies on simulating the number of users by observing the server logs and studying the user requests .
WEB APPLICATION TESTING METHODOLOGY
Different types of testing techniques like coverage testing, structural testing, statistical testing, combinatorial interaction testing, penetration testing.
Search based software engineering testing.
Unique Input/Output method using Genetic Algorithms .
Web application Slicing .
, .
Hierarchical testing .
Bypass testing.
Cross Browser compatibility testing .
Leveraging User session Data Testing have been presented by various researchers in the context of web application testing .
Structural Testing -Data flow analysis on web applications is performed and model for testing the application is built dynamically.
Statistical Testing Ae Input Sequence is generated to test the interactions with web applications based on the profile use of the web application.
Mutation Testing Ae The technique of introducing faulty code .
alled mutant.
into the source code deliberately at predetermined points and testing the software to uncover any unknown errors.
It is one of the effective coverage criterion techniques for testing of web applications .
Combinatorial Interaction Testing - Using a combination of different techniques by first designing a unique input space matrix for the web application Penetration Testing - Automated tests which are run simulating the active attacks to expose the susceptibilities of the web applications.
Search Based Software Engineering Testing - Exploration of solutions within a state space and calculating a fitness function to the solution iteratively until we arrive at a most optimal solution.
The technique is employed for branch coverage of web applications.
Using UIO and Genetic Algorithms - Path selection is done based on a unique input/output (UIO) algorithm and automatic test case generation using Genetic Algorithms which results in the best test GUI Interaction Testing - GUI widgets events sequences are performed and the web application tested for correctness by observing the state of the GUI widgets.
Web Application Slicing - Reduced web application which behaves completely as the original one with respect to some criterion and performing the testing.
Cross Browser Compatibility testing - Subjecting web applications to deployment across different browsers for adherence to expected results.
Hierarchical Strategy - High level operational profile is developed enumerating frequency of use of operations and a high level function group to thoroughly test such an operation or related components is Bypass Testing - Bypass client side checking by providing invalid inputs to web application to check correctness and security of the web application.
Leveraging User Session Data - Test cases are generated by applying strategies to collected user interactions in the form of URLAos and name-value pairs.
Browser Fuzzing By Scheduled Mutation - Browsers are validated by using the static and dynamic ways, the former based on the input format while the latter randomly executing instructions giving one input at a Invariant Based Technique - Testing the web application by crawling the web pages, and formally designing a state flow graph with all the possible user interaction sequences resulting in the possible user interface states.
Model Based Testing Technique - Web application is reduced to a state transition graph and navigation through links is tested to ascertain correct behavior of the web application.
A complete code coverage for any application assures thorough testing and higher probability of catching defects but the tester has to leverage upon the cost involved in complete code coverage verses the number of defects unearthed.
A plethora of code coverage tools are available both open source and licensed for testing the web applications.
A brief summary of all the various methods or techniques is presented in Table 1.
A Review on Web Application Testing and its Current Research Directions (D.
Rajya Lakshm.
A ISSN: 2088-8708 Table 1.
Summary of Different Testing Methods/ Techniques available for Web Application Testing Testing Technique Central Idea Behind the Technique References Structural Testing Statistical Testing Mutation Testing Combinatorial Interaction Testing Penetration Testing Search Based Software Engineering Testing Using UIO and Genetic Algorithms GUI Interaction Testing Web Application Slicing Cross Browser Compatibility Testing Hierarchical Strategy Bypass Testing Leveraging User Session Data Browser Fuzzing By Scheduled Mutation Invariant Based Technique Model Based Testing Technique Data flow analysis Interactions based on a profile use of a web application Fault based testing Combination of different techniques.
Active attack simulation Branch Coverage of web applications Path Selection using GA based algorithm State based testing of GUI widgets Testing on a slice of web application Testing on different browsers Development of an operational profile Security testing by avoiding client side validations URL testing Browser testing in static and dynamic ways State based testing of interactions State based testing based on link navigation .
, .
, .
, .
A detailed study of each of the above mentioned techniques and their future research directions are consolidated in Table 2.
The research directions highlighted in the table are directions in which the presenters of the respective work are heading towards.
Table 2.
Advantages and Disadvantages of Various Testing Techniques Testing Technique Future Research Directions Structural Testing Statistical Testing Mutation Testing Combinatorial Interaction Testing Penetration Testing Search Based Software Engineering Testing Using UIO and Genetic Algorithms GUI Interaction Testing Web Application Slicing Cross Browser Compatibility Testing Hierarchical Strategy Bypass Testing Leveraging User Session Data Browser Fuzzing Scheduled Mutation Invariant Based Technique Model Based Testing Technique Moving towards development of new automated testing tools for increased Automation to generate profiles of user accesses from the log file.
Optimization of test suites and a need for introduction of new mutation Expansion towards real time systems and try to automate the process of CIT model development.
Automation work for development of various tools The technique needs to expand to cater to the languages supporting dynamic types and automation to simulate client side responses.
Scope for automation of input values which are manually provided by the Varied test lengths to be augmented and more automation to come up with partition making automated.
Automation to build completely automated regression testing using the slicing method along with an improvisation in the selection of test cases.
Development of a larger catalog of known DOM level differences between various browsers and automation for detecting differences between various browsers not listed at DOM level.
Validation of approach by deploying it in the industry.
Development of some automated framework to develop automated bypass Combination of traditional testing techniques and user-session data to be pursued further.
Techniques for filtering user sessions and clustering algorithms can further be explored for taking the initial set of user sessions.
Cost Effectiveness against traditional testing stands to be estimated and Development of a reproducing mechanism for recording crash input as it is difficult to record persistent information for browser fuzzing.
New Seeds and methods to achieve more crashes.
Work needs to progress on how to capture user session data and expanding it to larger applications.
Need for the development of multiple tools and techniques to enhance Reference No .
, .
, .
, .
, .
TOOLS FOR WEB APPLICATION TESTING
A review of some of the automated testing tools and the type of testing supported by the tools led the survey to some interesting facts that there are not many tools available for testing the non-functional IJECE Vol.
No.
August 2017 : 2132 Ae 2141
IJECE
ISSN: 2088-8708
requirements of the web applications.
The tools that have been studied as part of this survey are listed in Table 3.
A comparative study of the tools is presented in Table 3.
Table 3.
Comparison of Various Testing Tools for Web Application Testing No.
Tool
Name
WATIR
Type of Testing Supported Functional Testing Selenium Functional Testing HP-QTP Functional Testing Fitnesse Acceptance Testing IE.
Chrome.
Safari.
Firefox
N/A
testComp Functional Testing.
Unit Testing IE.
Chrome.
Safari.
Firefox Load Runner Load Testing Chrome.
Safari.
IE.
Firefox
Test Ng
N/A
TOSCA
Integration Testing.
Functional Testing.
End-End Testing.
Unit Testing Functional Testing SilkTest Functional Testing IE.
Firefox WinRun Functional Testing Any Browser ApacheJ Meter NeoLoa Performance Testing.
Load Testing Load Testing LoadUI Load Testing Any Browser WebLoa Load Testing IE.
Firefox.
Chrome
WAPT
Rational Performa Tester Testing Anywher Qengine Load Testing.
Stress Testing Performance Testing IE.
Firefox.
Chrome and others Any Browser Functional Testing IE.
Firefox.
Chrome Functional Testing IE.
Mozilla.
Firefox
Functional Testing Functional Testing MUTAN
DIS
ATUSA
Browser Support Language Supported All Open Source/Licensed References Open source .
Java.
NET.
Ruby.
Perl.
PHP VB Script Open source .
, .
Licensed .
Java.
Python.
C#.
VBScript.
Jscript.
Python.
Delphi Script.
C Script.
Script Java.
NET.
JavaScript.
HTML Java Open source .
Licensed .
Licensed .
Open source .
Delphi, .
NET
WPF,
Java Swing/SWT/A WT.
NET,
Java.
Swing.
SWT.
DOM Any web based Licensed .
Licensed .
Licensed .
Any Browser web service Open source .
IE.
Chrome ASP, .
Net.
J2EE.
PHP Licensed .
Any web based HTTP/HTTPS (SSL.
TLS).
WebSocket.
PUSH.
AJAX.
SOAP.
HTML5.
WebDAV and Java Script Licensed .
Licensed .
Licensed .
Any Script.
XSS.
SOAP Licensed .
Licensed Any browser Any Web Based Application VBScript.
Jscript.
Python.
Delphi Script.
C Script.
C# Script Java Script Any browser Ajax based any Open source IE.
Chrome.
Safari.
Firefox IE.
Chrome.
Safari.
Firefox IE.
Firefox.
Chrome Firefox.
Licensed.
ut EndSal.
announced Open source .
A Review on Web Application Testing and its Current Research Directions (D.
Rajya Lakshm.
A No.
Tool
Name
Type of Supported Crawljax
Navigation Testing Any browser
JSART
Regression Testing Any Browser webMate Regression Testing IE.
Firefox.
Chrome and others reAjax Functional Testing Mozilla.
Firefox Functional Testing WebVizo Web Portal In Containe r Testing Veriweb Tool WebScar Acunetix Fortify ISSN: 2088-8708 Testing Open Source/Licensed References Open source .
Open source .
Licensed .
Open source .
Any browser Language Supported script crawling Ajax based any script crawling Java Script based any web VBScript.
Jscript.
Python.
Delphi Script.
C Script.
C# Script Ajax Any Language Open source .
Integration Testing Any browser Any Script Open source Navigation Testing Any browser JavaScript Open source .
Security Testing IE.
Firefox.
Chrome and others Any browser Any Script.
XSS.
SOAP Any Script.
XSS.
SOAP C#,.
NET.
Java.
ASP
Open source .
Licensed .
Licensed .
Layout Security Testing.
Penetration Testing Security Testing Browser Support
Any web browser
RESULTS AND ANALYSIS
There is a need for the development and devise of a new metric to indicate the health of a web A plotting .
onsidering only the tools under stud.
of the no.
of tools vs the type of testing supported by each tool is presented in Figure 3.
It is evident that there are not many tools testing the nonfunctional attributes of web applications like reliability, trustworthiness, and fault tolerance are not readily There is still however a dearth for open source tools using mutation analysis techniques which can perform automated test case execution based on mutation analysis while optimizing the test suite.
Figure 3.
No of Tools Vs Type of Testing The number of tools vs whether open source/ licensed has been plotted in Figure 4 which depicts that the topmost tools available for web application testing are mostly commercial in nature and there is a need to come up with a full-fledged open source tool for web application testing covering the various aspects of web application testing.
IJECE Vol.
No.
August 2017 : 2132 Ae 2141
IJECE
ISSN: 2088-8708
Figure 4.
No.
of Open Source Vs Licensed Tools CONCLUSIONS Tools for assessing the non-functional requirements in general are less where non-functional requirements actually play a key role in customer retention and popularity of the web application.
Specifically non-functional requirements like usability, compatibility are absolutely in oblivion when it comes to testing.
It is observed that with the web making a paradigm shift to the web services like SOAP.
REST, and WSDL et.
Compatibility testing of web applications is quite challenging and an area of growing concern where not much progress is seen in the earlier works.
There is a dire need for a testing strategy for testing such non-functional requirements.
Proposal of a metric to test the adequacy of test suite for exhaustive testing of a web application is the need of the hour.
A metric to indicate the overall health of a web application needs to be proposed.
There is a need for development of a framework to support automated testing of a web application for the proposed strategy.
With the entire web moving towards cloud based services, at the next level it is important to explore the testing of web applications on cloud.
A panoramic view of testing web applications shows a need for a testing tool that can be deployed online in order to observe the dynamic behavior of web applications, as it is always difficult to catch such bugs as the web applications are often tested on standalone The potential of testing techniques like mutation testing has not been completely tapped with testing web applications.
There is a lot of scope to expand the horizons of mutation testing technique with web applications and unleashing its real potential.
There is scope for the proposal and validation of some additional mutation operators pertaining to challenges like session management, cookie management where different languages like jsp, python can be explored to present more operators for mutating the web Scope for optimization of regression test suites using appropriate techniques still exists.
ACKNOWLEDGEMENTS
The authors would like to thank the pioneers of research in testing of web applications who proposed several novel techniques and inspiring them to perform a survey and further pursue a direction for effective testing of web applications.
REFERENCES