{"id":682,"date":"2020-03-08T09:13:29","date_gmt":"2020-03-08T09:13:29","guid":{"rendered":"https:\/\/megamart2-ecsel.eu\/?p=682"},"modified":"2020-03-08T09:17:56","modified_gmt":"2020-03-08T09:17:56","slug":"exploratory-performance-testing-using-reinforcement-learning","status":"publish","type":"post","link":"https:\/\/megamart2-ecsel.eu\/exploratory-performance-testing-using-reinforcement-learning\/","title":{"rendered":"Exploratory Performance Testing Using Reinforcement Learning"},"content":{"rendered":"

Due to the widespread availability of high-speed Internet, the level of expectation of the users with respect to the performance of web-based software systems has changed dramatically. For instance, about 40% of the customers will abandon a web application if the response time is greater than 3 seconds. Furthermore, Amazon, a leading online retailer, reports that 100 milliseconds (ms) additional delay in response time could cost them 1% drop in sales. Therefore, ensuring the reliability and efficiency of a software system is imperative for such companies and the overall success of the software projects.<\/strong><\/p>\n

In general, two-thirds of the performance bottlenecks are triggered only on specific input values<\/strong>. However, finding those input values<\/strong> for performance test cases that can identify performance bottlenecks in a large-scale complex system within a reasonable amount of time is a notoriously difficult and expensive task<\/strong>. The reason is that there can be numerous combinations of test input values and executing one such combination against the SUT (system-under-test) can take a considerable amount of time. Thus, it is almost impossible for test engineers to test exhaustively every possible combination. The problem becomes even more challenging when the SUT is a black-box<\/strong>. This means that we cannot inspect the internal system dynamics. The only way to interact with the SUT and monitor different key performance indicators (KPIs) of the SUT is through public interfaces and external observations during the execution of the SUT, respectively.<\/p>\n

In order to address the above issues, we formulate the test data generation problem for performance test cases as a reinforcement learning problem<\/strong>.\u00a0Reinforcement learning\u00a0(RL) is a class of machine learning techniques where an\u00a0agent\u00a0learns to interact with an unknown environment in order to accomplish a given goal through a trial and error process<\/p>\n

To that extent, we present an approach for\u00a0Performance EXploration using\u00a0RL\u00a0(PerfXRL) to explore a large space of combinations of the input parameter values in order to find performance bottlenecks (denoted as\u00a0good combinations<\/em>) in a black-box system without any prior domain knowledge. Our approach uses Dueling Deep Q-Network (DDQN), which is an RL technique. In addition, we develop tool support for our approach in Python and we empirically evaluate our approach and show that PerfXRL is able to detect 72% more performance bottlenecks than random testing.<\/p>\n

\"Performance<\/p>\n

More details can be found in Ahmad, Tanwir\u00a0 and Ashraf, Adnan and Truscan, Dragos and Porres, Ivan Exploratory Performance Testing Using Reinforcement Learning.\u00a0 In Euromicro Conference on Software Engineering and Advanced Applications (SEAA 2019), Kallithea, Chalkidiki, Greece. August 28 \u2013 30, 2019<\/em> (author version<\/a>)<\/p>\n","protected":false},"excerpt":{"rendered":"

Two-thirds of the performance bottlenecks are triggered only on specific input values. Finding those values is a very challenging task. We use reinforcement learning to help you on this.<\/p>\n","protected":false},"author":7,"featured_media":684,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_et_pb_use_builder":"","_et_pb_old_content":"","_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0},"categories":[23],"tags":[44,134,110],"_links":{"self":[{"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/posts\/682"}],"collection":[{"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/comments?post=682"}],"version-history":[{"count":0,"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/posts\/682\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/media\/684"}],"wp:attachment":[{"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/media?parent=682"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/categories?post=682"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/megamart2-ecsel.eu\/wp-json\/wp\/v2\/tags?post=682"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}