Important things to remember before starting any large performance activity

by perfstories

There is a lot of articles, presentations about Software Performance Engineering Methodology. Just google it using, say, “Software Performance Engineering” and have fun.

So I won’t repeat all the things the search engines can find for us ;). Let me just outline several points, which are the most important from my point of view.

Make sure performance is really important for you application

This point will allow you to set clear and valuable targets for your work or performance requirements. Мake sure your bosses understand that performance is important.

I think that performance is always important except for the case when you can quickly and clearly answer why it isn’t. And to answer this question you have to know what your performance requirements are, where you are in terms of performance now and make sure it’s enough for your business goals. If it’s your case it means performance is important for you, as you know performance requirements, performance metrics and performance status (more likely you even measured something) ;). So now you can tell everyone that performance isn’t important for you (as you took care of it in advance).
However, if your application changes it’s impossible to measure nothing and know what performance status is. Yeah, you can assume what your performance is but you don’t KNOW it.

Define exactly what performance is

Define it for yourself and make sure your bosses understand it the same way.

For example, if you are going to work on performance of your JVM define a set of benchmarks and the scores on these benchmarks that you would like to achieve. Say, you may want SPECjvm2008 to be 25% faster on your JVM than on Sun Oracle JVM on the same hardware. Specify what hardware exactly you will torment.

If you have some J2EE application define (on a very high level so far) which parts are the most important for business. Say, this is some portal. And you would like to keep the users of this portal satisfied (excellent response time) and, say, 99% of time, portal can serve all incoming users. Then you can detail these requirements further.

Let me give you one more example of performance requirements. One of my bosses said: “My boss, Mr E, should sleep well.” Confused? Actually the situation was simple.

Mr E. was responsible for our application, in particular, for its behavior in production. He was to some extent an IT guy. Why didn’t he sleep well sometimes? Because his boss, Mr S., who wasn’t an IT guy at all, called Mr E. in the middle of the night always with the same questions: “What the heck is your application doing again? Why is it so slow? Why cannot our customers even login on our main portal?” or “What’s going on again? Why do I need to wait for 20 seconds to change a tariff? Who are you working for? Our competitors?”.

Then I started to answer some questions to formulate performance requirements:

  • What are the most business important parts of our application? (Say, if 0,04% of the users wait for 30 secs to login to the portal and fix a typo in their profile, Mr S. will not call Mr E. As business will not lose money…);
  • What should be appropriate resource utilization on our servers in production? (There should be enough resources to keep our application working. If cpu utilization and memory consumption be 100%, application will perform slower, sometimes even crash, so Mr S. will wake up and call Mr E. Do we need this? No, so let’s keep utilization much lower.);
  • What should be response times? (What would be comfortable for me as a user? What would be comfortable for Mr S. as a user?);
  • Etc.

You cannot improve what you don’t measure

When I worked on creation of my performance team the most of bosses in my company said something like this: “Performance of our application is a pain in the neck!”. It was very fashionable to talk this way as the commercial launch failed because of performance.

So I heard just a laugh when I asked: “How much and where exactly does it pain?”. But nobody could answer.

Please don’t talk this way. Much better if you say: “Response time on our payment scenario is 15 secs instead of 1 sec as in the requirements!”.

As you can see, to change the situation this way we need 2 things:

  1. Requirements;
  2. Measurements.

There are a lot of questions to be answered, like this:

  • When and where should we do measurements if production and test environments differ a lot (say, in terms of hardware)?;
  • How long should the measurements take?;
  • Which tools should we use?;
  • Should we measure each of the components separately?;
  • What should be measured exactly?.

Let’s talk about these questions but in other posts. As you remember now we talk about themain points.

For example, I asked a guy, Mr.F., who said more often than others that performance was a pain in the neck for him, what exactly he meant. He answered that the problem was in using a third party component for authentication/authorization and he expected the load on corresponding scenarios to increase several times in several months. As he was a very high level architect he had this pain, but as he was a very high level architect he didn’t think measurements could help. It was just more comfortable for him to live with this pain. So I clarified for myself what scenarios were exactly, wrote a very simple test and made measurements, which proved this third party application (OpenSSO, actually, or OpenAM now) could keep all expected load and even much more. So the problem didn’t even exist. The question was closed.
The main point here is: use numbers when you talk about performance.

Make sure you understand how to improve things

If you are familiar with SPE methodology, you will tell me: “This is obvious. We will start with performance analysis to understand what the problem is.”

And even if you are not familiar with SPE methodology but will think about this for some time this is also obvious. (First you need to find out what the main problem in your application is rather than guess it. Only after that it makes sense to think about any changes/optimizations.).

Unfortunately, not all people are familiar with SPE methodology or ready to think for some time. So prepare to explain your approach and why it will save money of your company.

I wonder how people frequently “improve” application without even thinking about what the problem is. At least it was popular in my company. You know, one of my colleagues, Mr. R., regularly proposed rewriting all the code (all the working distributed application) in C instead of Java, as “C is faster than Java” before any performance analysis (even before any business analysis of this rewriting).

Go ahead if you remember all of these points 😉

You can be so lucky that your boss understands these points, so you just need to implement this performance process in your company/team.

If you are not so lucky (like I was) arm yourself with patience and try to educate and convince your bosses.