testomat.io управление авто тестамиИнтересная статья описывающая основные проблемы с jmeter и способы их решения
ЗЫ Многие вещи подтвержденны мной лично на практике, но к сожалению до прочтения этой статьи :)
JMeter Performance and Tuning Tips
Posted by Philippe Mouawad Dec 26, 2012
Recently, we came across a great, informative article on JMeter and wanted to share it with our readers. So, many thanks to Philippe Mouawad for graciously allowing us to repost.
JMeter Performance and Tuning Tips
Context
You want to make a Load Test that will use hundreds to thousands of Threads and you've read on so many blogs that :
- JMeter consumes a lot of memory...
- JMeter generates OutOfMemory...
- JMeter consumes a lot of CPU...
- JMeter can only cope with 100 threads and not much more...
- JMeter is fine for playing but once it becomes serious use another tool :-)
- etc.,etc,....
This has become a kind of "Urban Legend" partly due:
- to issues that have been fixed for a while now
- and partly in my opinion to some default configuration parameters that lead to these issues
- misuse of elements that are to be used during scripting and never during Load Test
Hopefully, You're in the right place, Everything you read is WRONG. :-)
In this short article, we will see how to get the most out of JMeter.
JMeter Performance Basic concepts
Understand important concepts about JMeter:
- The more it computes to create or process your samples, the less time it spends sampling, so when doing custom coding, ensure it is efficient
- Listeners receive Sample Results and do some processing with it, this takes resources (memory, CPU) so during Load Testing, a very simple rule is to REMOVE ALL LISTENERS!, you will ask How can I use my results then?, the answer is, you will do it after the run.
So NEVER EVER have one of these listeners in your Test Plan during a Load Test:- View Results in Table => OutOfMemory guarantee
- View Results in Tree => OutOfMemory guarantee
- Graph Results => Performance issues
- Assertion Results => OutOfMemory guarantee
- Comparison Assertion Visualizer
- Distribution Graph (alpha) => Performance issues
- Graph Results => Performance issues
Performance Checklist
Here are our tips to get the most out of JMeter:
Always use the last version of JMeter
JMeter performances and memory usage have been highly improved in last 3 versions and upcoming 2.9 version will bring new set of improvements regarding memory and CPU:
- http://wiki.apache.org/jmeter/JMeterPerformance
- https://issues.apache.org/bugzilla/show_bug.cgi?id=54268 (fixed in 2.9)
Use NON-GUI for Load Test
JMeter has 2 modes:
- GUI mode is for creating the test plan, checking it, debugging it BUT NOT FOR MASSIVE LOAD TEST. AWT Event Thread will disrupt your Load Test.
- NON-GUI mode is for massive load testing, it is as simple as:
<JMETER_HOME>/bin/jmeter -t <Path to Test Plan> -n -l <path to results>/results.csv
Configure JMeter Java options to meet your requirements
- Default JMeter java configuration comes with 512 Mo and very little GC tuning.
- First ensure you set -Xmx option value to a reasonable value regarding your test requirements.
- Then change MaxNewSize option in jmeter file to respect the original ratio between MaxNewSize and -Xmx.
- Finally try tuning GC options only if you master this domain.
Use CSV as output for SaveService
XML is verbose, it takes resources to be written (CPU and memory) and for analysis, CSV is great so forget about XML. Furthermore, for massive load tests there are many result data you don't need.
So, in user.properties, add:
jmeter.save.saveservice.output_format=csv
jmeter.save.saveservice.data_type=false
jmeter.save.saveservice.label=true
jmeter.save.saveservice.response_code=true
jmeter.save.saveservice.response_data.on_error=false
jmeter.save.saveservice.response_message=false
jmeter.save.saveservice.successful=true
jmeter.save.saveservice.thread_name=true
jmeter.save.saveservice.time=true
jmeter.save.saveservice.subresults=false
jmeter.save.saveservice.assertions=false
jmeter.save.saveservice.latency=true
jmeter.save.saveservice.bytes=true
jmeter.save.saveservice.hostname=true
jmeter.save.saveservice.thread_counts=true
jmeter.save.saveservice.sample_count=true
jmeter.save.saveservice.response_message=false
jmeter.save.saveservice.assertion_results_failure_message=false
jmeter.save.saveservice.timestamp_format=HH:mm:ss
jmeter.save.saveservice.default_delimiter=;
jmeter.save.saveservice.print_field_names=true
Use Post-Processor and Assertion efficiently
Post-Processor and Assertions have a cost.
Ensure you use them when required and use the ones that consume the less memory and CPU:
Use Regular Expression Extractor
Use Regular Expression Extractor for extracting data BUT never ever check Body (unescaped), choose among:
- Body
- Headers
- URL
- Response Code
- Response Message
Use efficient Regular expressions and extract as less data as possible
Avoid XPath Extractor when possible
XPath builds a DOM tree so it consumes CPU and memory, prefer Regular Expression extractor
Use Response Assertion or Size assertion
These 2 implementations fit 99% of requirements.
Avoid costly ones as:
- XML Assertion
- XML Schema Assertion
- XPath Assertion
Use JSR 223 + Groovy for Scripting
You have a lot of options to do scripting with JMeter:
- Beanshell
- BSF and all supported languages Javascript, Scala , Groovy, Java ...
- JSR223 and all supported languages Javascript, Scala , Groovy, Java ...
Although you can be lazy and choose the language you know, FORGET ABOUT IT.
Use the most efficient option, which is JSR223 + Groovy + Caching (supported since JMeter 2.8 in external script and in next upcoming JMeter 2.9 also supported with embedded scripts).
Using Groovy is as simple as adding groovy-VERSION-all.jar in <JMETER_HOME>/lib folder.
But of course ensure your script is necessary and efficiently written, DON'T OVERSCRIPT
Generate Reports AFTER the Run
Always generate Graphs and Reports AFTER the Load Test and NEVER EVER DURING IT
You can use JMeter reports:
- Aggregate Graph
- Response Time Graph
- JMeter Plugins graphs
Distributed (Remote) Testing
Once you reach the limits of one machine, you can switch to distributed or remote testing.
But JMeter defaults are not fine for efficient remote testing, so in user.properties, add: mode=StrippedBatch
This will:
- remove some data from the SampleResults as the response body, but do you need response body during a High Load Test, NO, DEFINITELY NO !
- will send Sample Results as Batches and not for every sample reducing CPU, IO and network roundtrips
About the Author
Philippe Mouawad works as an Architect and technical expert for Ubik-Ingenierie where he leads the development of Commercial Apache JMeter based Plugins for RIA and Ajax technologies (GWT, Flex). Philippe is also a committer on the Apache JMeter project and member of the PMC since October 2011.
*This article is reprinted with permission from Philippe Mouawad and Ubik-Ingenierie. (Original posting December 24, 2012 http://www.ubik-ingenierie.com)