TestingServlets

The real issue however, is DoesSunTestServlets? We think not!

Have any of you looked at HttpUnit lately? It's come a long way since I looked it over in 2001/2:
among other things, a built in servlet container. --MilesD

I've been using it lately, and agree it's got a lot better. I wrote little load testing framework for JPMC using it. It was really great for show why pages took 15 round trips to build! (masses of javascript and inner frames, oh my god). -- OliBye

There appear to be several challenges:- 1. How to avoid the overhead of TCP, to make the tests faster. 1. How to build a facade to persude servlets they're inside a real servlet engine. 1. How to check the output. 1. How to avoid look and feel changes, breaking lots of tests.

We're currently on this evolutionary path:-

• Stage 1
• • In true XP style avoid challenge 1,2 & 3 by saying: 4 is hard let's DoItWhenYouNeedIt? and simply made an 8 line method to send off an HTTP GET request. (see getServerResponse in https://xpdeveloper.com/cgi-bin/cvsweb.cgi/xtc/TestServlets.java?rev=1.1.1.1&content-type=text/x-cvsweb-markup) which returns a string, which we can make assertions on.
  • The test in testDisplaying is really only the following 3 lines:-

• Stage 2 (reached in another project, XtC should get to this stage very soon)
• • When using the equals method to check the output you're very sensitive to changes in look and feel.
  • The next simplest thing we found todo was to use the String.indexOf(String) method. This returns -1 if the invoked String doesn't contain the String argument.
  • This method worked until we started having multiple variables in the resulting HTML.
  • The point is XP says you should do this until you need something more complicated. To the extent that you can sometimes AdjustAStorySoItIsEasierToTest?, this is LateralThinkingNotCheating.

• Stage 3 (reached in another project).
• • Using Stage 1 & 2 for a while you should notice very soon that you are building strings and comparing them over and overagain. If you are AggressivelyRefactoring? you need to sort this out. Hence Stage 3.
  • I know ServletOutputStream?.print() works, if Sun had delivered me unit tests I could prove this (or I could write some tests). Therefore:-
  • • If I pass a test Object array to MessageFormat.format(), and I test my format string outputs what I expect using the 'Stage 1 "equals"' approach, then elsewhere in my tests I only need to check the correct Object array is returned.
    • In this way we can slightly decouple the HTML from the datamodel (objects returned in the Object array).
    • You can now use the equals method to make sure the datamodel object you're rendering as HTML is the real one you expect, by writing an implementation for equals, rather than using a String compare.

• Stage 4 (Next stage under consideration)
• • Since doGet doPost and the rest of the methods are just methods in a class, why not call them directly from the test (This was what we wanted to do all along but it's not the simplest thing (remember you've got to beat the 8 lines of Stage 1 (not that lines of code is a measure of simplicity)) - Oli neglects to mention that while its only 8 lines of code there is complexity in that often servlet engines don't behave quite as expected and you can spend a lot of time figuring out why something isn't working only to discover there was some memory leak in NT and a port didn't get closed properly (e.g. reboot your computer). These complexities (and you get the same issues with databases) are what drive the need to be able to Mock up lighter weight versions of systems (where you can control the behavior - e.g. introduce error conditions that your code should handle ) -- TimM

Frank, if you looked at the code for getServerResponse above you'll see the following

URL servlet = new URL( request ); URLConnection sc = servlet.openConnection();

BufferedReader in = new BufferedReader(new InputStreamReader(sc.getInputStream()));

String inputLine;

while ((inputLine = in.readLine()) != null) { result = inputLine; } in.close(); return result;

This actually sends of an HTTP request and gets the response back (it actually only gets the last line, which was corrected in a later version.) Does this help you -- OliBye

I feel uncomfortable with what Frank is proposing, the whole idea behind unit testing to mock up pieces so that you can test code in isoloation. Ideally a MockServletEngine would give you a good place to test your servlet code. I think Frank is getting more into FunctionalTesting?, and at this level, you are happy that your servlets are calculating correct results you are verifying performance or UI layout. These are hard to test, and certainly we haven't tried anything in this are. As we don't have a MockServletEngine, we have leveraged the fact that you can run the jsdk in your dev. environment locally (its not ideal but it does work). -- TimM

Thanks Oli, but that's not really my problem. Transferring the data from the development machine to the target machine currently uses a manual ftp process, and sometimes manual changes need to be made to the target machine's server configuration. Although Java is pretty powerful, it's not so good at scripting external software, particularly on a remote machine. I guess I could make/get/buy some ftp stuff for Java to handle the transfer, but I'm really wary of encoding some sort of remote config file edits into my unit test harness. All of this would be very specific to a particular machine name, server software and destination configuration. I'm just drooling for the packaged web-application support in JSDK 2.2, but it's not here yet. --FrankCarver

I've also got the makings of one of these, based on code I wrote to implement a servlet server when servlets were first introduced. The trouble I find is that any code you write is not quite close enough to the actual code in the server (unless you use your own server, in which case you can make it "test friendly" anyway). You also need to work quite hard to track changes and grey areas in the servlet API. I gave up on my server after JSDK 1.0, and test harnesses can have a tendency to stay similarly static. Please prove me wrong! --FrankCarver Servlets are a bit of a challenge to test, however with a bit of perseverance you can test anything. In our RealCode examples we are starting to show how we have tested Servlets in the past, and we have more ideas to introduce.

The real issue however, is DoesSunTestServlets? We think not!

There appear to be several challenges:- 1. How to avoid the overhead of TCP, to make the tests faster. 1. How to build a facade to persuade servlets they're inside a real servlet engine. 1. How to check the output. 1. How to avoid look and feel changes, breaking lots of tests.

We're currently on this evolutionary path:-

• Stage 1
• • In true XP style avoid challenge 1,2 & 3 by saying: 4 is hard let's DoItWhenYouNeedIt? and simply made an 8 line method to send off an HTTP GET request. (see getServerResponse in https://xpdeveloper.com/cgi-bin/cvsweb.cgi/xtc/TestServlets.java?rev=1.1.1.1&content-type=text/x-cvsweb-markup) which returns a string, which we can make assertions on.
  • The test in testDisplaying is really only the following 3 lines:-

• Stage 2 (reached in another project, XtC should get to this stage very soon)
• • When using the equals method to check the output you're very sensitive to changes in look and feel.
  • The next simplest thing we found todo was to use the String.indexOf(String) method. This returns -1 if the invoked String doesn't contain the String argument.
  • This method worked until we started having multiple variables in the resulting HTML.
  • The point is XP says you should do this until you need something more complicated. To the extent that you can sometimes AdjustAStorySoItIsEasierToTest?, this is LateralThinkingNotCheating.

• Stage 3 (reached in another project).
• • Using Stage 1 & 2 for a while you should notice very soon that you are building strings and comparing them over and overagain. If you are AggressivelyRefactoring? you need to sort this out. Hence Stage 3.
  • I know ServletOutputStream?.print() works, if Sun had delivered me unit tests I could prove this (or I could write some tests). Therefore:-
  • • If I pass a test Object array to MessageFormat.format(), and I test my format string outputs what I expect using the 'Stage 1 "equals"' approach, then elsewhere in my tests I only need to check the correct Object array is returned.
    • In this way we can slightly decouple the HTML from the datamodel (objects returned in the Object array).
    • You can now use the equals method to make sure the datamodel object you're rendering as HTML is the real one you expect, by writing an implementation for equals, rather than using a String compare.

• Stage 4 (Next stage under consideration)
• • Since doGet doPost and the rest of the methods are just methods in a class, why not call them directly from the test (This was what we wanted to do all along but it's not the simplest thing (remember you've got to beat the 8 lines of Stage 1 (not that lines of code is a measure of simplicity)) - Oli neglects to mention that while its only 8 lines of code there is complexity in that often servlet engines don't behave quite as expected and you can spend a lot of time figuring out why something isn't working only to discover there was some memory leak in NT and a port didn't get closed properly (e.g. reboot your computer). These complexities (and you get the same issues with databases) are what drive the need to be able to Mock up lighter weight versions of systems (where you can control the behavior - e.g. introduce error conditions that your code should handle ) -- TimM

Frank, if you looked at the code for getServerResponse above you'll see the following

URL servlet = new URL( request ); URLConnection sc = servlet.openConnection();

BufferedReader in = new BufferedReader(new InputStreamReader(sc.getInputStream()));

String inputLine;

while ((inputLine = in.readLine()) != null) { result = inputLine; } in.close(); return result;

This actually sends of an HTTP request and gets the response back (it actually only gets the last line, which was corrected in a later version.) Does this help you -- OliBye

I feel uncomfortable with what Frank is proposing, the whole idea behind unit testing to mock up pieces so that you can test code in isoloation. Ideally a MockServletEngine would give you a good place to test your servlet code. I think Frank is getting more into FunctionalTesting?, and at this level, you are happy that your servlets are calculating correct results you are verifying performance or UI layout. These are hard to test, and certainly we haven't tried anything in this are. As we don't have a MockServletEngine, we have leveraged the fact that you can run the jsdk in your dev. environment locally (its not ideal but it does work). -- TimM

Thanks Oli, but that's not really my problem. Transferring the data from the development machine to the target machine currently uses a manual ftp process, and sometimes manual changes need to be made to the target machine's server configuration. Although Java is pretty powerful, it's not so good at scripting external software, particularly on a remote machine. I guess I could make/get/buy some ftp stuff for Java to handle the transfer, but I'm really wary of encoding some sort of remote config file edits into my unit test harness. All of this would be very specific to a particular machine name, server software and destination configuration. I'm just drooling for the packaged web-application support in JSDK 2.2, but it's not here yet. --FrankCarver

Obviously a lot has changed since I wrote that. Servlets 2.2 has come and gone and some excellent tools are now available. I now solve this sort of problem using Ant, which integrates very well with JUnit, and provides loads of tools for stuff like building jar and war files, ftp transfers, remote commands and so on. A typical project has a set of "local" unit tests done on classes before building into an application, then a deployment to a test server and a collection of "remote" unit tests to test the deployment process and any late-notice issues (compiling JSP files on demand, server class path and file permission problems and so on). Only when both sets of tests pass is the project deployed to the "real" server for integration testing, and at the same time, the project source is "jar"ed up, timestamped and placed in a directory on a separate machine as a backup. Some projects also have an optional set of JUnit tests which take too long to be part of the main release process (up to 12 hours in one case!), but are run from time to time as a confidence check. --FrankCarver

I've also got the makings of one of these, based on code I wrote to implement a servlet server when servlets were first introduced. The trouble I find is that any code you write is not quite close enough to the actual code in the server (unless you use your own server, in which case you can make it "test friendly" anyway). You also need to work quite hard to track changes and grey areas in the servlet API. I gave up on my server after JSDK 1.0, and test harnesses can have a tendency to stay similarly static. Please prove me wrong! --FrankCarver

I have written some MockXXX? java classes that are fantastic for out of container testing. We found that mostly the servlets are two tier containers. eg a PageContext? has a JspPageWriter? inside it. MockObjects seem to save the day here.

I will link to the souce code examples soon. They are really dumb. (simple!)

Q: how do you post source code on Wiki?

BenHogan

³TestingServlet ³³host³³date³August 8, 2002³agent³Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)³TestingServlets