2009-09-03

Transparently supporting HTTP HEAD requests in Java and Spring MVC

The http 1.1 specification (RFC 2616) defines a number of methods: OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE and CONNECT.

Of these, the most familiar are GET and POST.
Web browsers rely on these two methods to send and receive data from web servers. (In compliance with the W3C Html 4 recommendation)

GET is meant for retrieving content from a web server. The requests should be idempotent. No critical state should change. Successive requests should return the same content. (This makes them ideal candidates for caching!)

POST on the other hand is typically used for operations that manipulate content on the server, such as adding, editing or removing content.



So, what about the other methods, you might ask?

Well it turns out browsers are not the only clients talking to our web servers. The web of 2009 has two more essential infrastructure components that our servers frequently have to deal with: proxy servers and web crawlers.
And it turns out these two types of clients are very fond of a third http method: HEAD.

HEAD is identical to GET except that only the http headers are returned. The body is discarded. This is primarily used for checking the validity of URLs. The load on the server will most likely remain the same as the content-length header must be returned (and thus potentially calculated based on the generated response body). Only the bandwidth is saved.



How do Java servlets deal with this?

Not too bad it turns out ! Deep inside the HttpServlet class (part of the Servlet API 2.5), we find the following code:

protected void doHead(HttpServletRequest req, HttpServletResponse resp)
 throws ServletException, IOException {
    NoBodyResponse response = new NoBodyResponse(resp);
 
    doGet(req, response);
    response.setContentLength();
}

Here is what the NoBodyResponse wrapper does (from the source code documentation):

A response that includes no body, for use in (dumb) "HEAD" support.
This just swallows that body, counting the bytes in order to set the content length appropriately. All other methods delegate directly to the HttpServletResponse object used to construct this one.

So this means the standard way of the servlet api to deal with with HEAD requests consists of:

1. Wrapping the response using the NoBodyResponse in order to suppress the body, but preserve the headers.
2. Execute the GET functionnality of the application (with the wrapped response object)
3. Set the content-length header of the response
4. Return the response headers to the client (without the body)

Sounds exactly like what we need, so where is the problem?


Comes in Spring MVC...

A typical Spring MVC 2.5 controller looks like this (from the Spring 2.5.6 reference documentation):

@Controller
@RequestMapping("/editPet.do")
@SessionAttributes("pet")
public class EditPetForm {
    private final Clinic clinic;

    @Autowired
    public EditPetForm(Clinic clinic) {
        this.clinic = clinic;
    }

    @ModelAttribute("types")
    public Collection<pettype> populatePetTypes() {
        return this.clinic.getPetTypes();
    }

    @RequestMapping(method = RequestMethod.GET)
    public String setupForm(@RequestParam("petId") int petId, ModelMap model) {
        Pet pet = this.clinic.loadPet(petId);
        model.addAttribute("pet", pet);
        return "petForm";
    }

    @RequestMapping(method = RequestMethod.POST)
    public String processSubmit(
            @ModelAttribute("pet") Pet pet, BindingResult result, SessionStatus status) {

        new PetValidator().validate(pet, result);
        if (result.hasErrors()) {
            return "petForm";
        }
        else {
            this.clinic.storePet(pet);
            status.setComplete();
            return "redirect:owner.do?ownerId=" + pet.getOwner().getId();
        }
    }
}

The editPetForm controller will respond to the /editPet.do URL.

GET /editPet.do executes the setupForm method.
POST /editPet.do executes the processSubmit method.


But what about HEAD /editPet.do?

It generates an error! Spring MVC cannot find a method annotated with

@RequestMapping(method = RequestMethod.HEAD)

and throws an exception.

There are two solutions to this problem.

Solution #1
Add the missing RequestMapping annotation to all controllers handling GET requests. This solves the problem, but not without significant drawbacks:

• It is verbose (this annotation must be added to all relevant controllers)
• It is tedious (every controller must be reviewed)
• It is error-prone (the burden lies on the developer not to forget this)

But luckily there is an alternative...

Solution #2
Add a servlet filter (in web.xml) in front of the Spring MVC servlet to lie about the http method and present all HEAD requests as GET.

Here is the code:

//Imports and documentation have been omitted...

public class HttpHeadFilter implements Filter {
    public void init(FilterConfig filterConfig) throws ServletException {
    }

    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) request;

        if (isHttpHead(httpServletRequest)) {
            chain.doFilter(new ForceGetRequestWrapper(httpServletRequest), response);
        } else {
            chain.doFilter(request, response);
        }
    }

    public void destroy() {
    }

    private boolean isHttpHead(HttpServletRequest request) {
        return "HEAD".equals(request.getMethod());
    }

    private class ForceGetRequestWrapper extends HttpServletRequestWrapper {
        public ForceGetRequestWrapper(HttpServletRequest request) {
            super(request);
        }

        public String getMethod() {
            return "GET";
        }
    }
}

With this filter, our previous example would work as follows:

The HEAD request will be seen as a GET by Spring MVC and therefore HEAD /editPet.do executes the setupForm method without the need for the extra annotation!


But what about the response?

The Spring MVC DispatcherServlet overrides the doService method from HttpServlet. This means that the nice NoBodyResponse logic is overridden, and thus never called.


There are once again two solutions to this problem:

Solution #1
Rely on your web container to suppress the response body for HEAD requests. Some containers, like Apache Tomcat, provide this functionality out of the box. Relying on container-specific behavior will increase your dependency on this particular server. This may or may not be a problem.

Solution #2
Integrate the NoBodyResponse wrapper with the HttpHeadFilter.

This is how the final solution looks like:

//Imports and documentation have been omitted...

public class HttpHeadFilter implements Filter {
    public void init(FilterConfig filterConfig) throws ServletException {
    }

    public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
        HttpServletRequest httpServletRequest = (HttpServletRequest) request;

        if (isHttpHead(httpServletRequest)) {
            HttpServletResponse httpServletResponse = (HttpServletResponse) response;
            NoBodyResponseWrapper noBodyResponseWrapper = new NoBodyResponseWrapper(httpServletResponse);

            chain.doFilter(new ForceGetRequestWrapper(httpServletRequest), noBodyResponseWrapper);
            noBodyResponseWrapper.setContentLength();
        } else {
            chain.doFilter(request, response);
        }
    }

    public void destroy() {
    }

    private boolean isHttpHead(HttpServletRequest request) {
        return "HEAD".equals(request.getMethod());
    }

    private class ForceGetRequestWrapper extends HttpServletRequestWrapper {
        public ForceGetRequestWrapper(HttpServletRequest request) {
            super(request);
        }

        public String getMethod() {
            return "GET";
        }
    }

    private class NoBodyResponseWrapper extends HttpServletResponseWrapper {
        private final NoBodyOutputStream noBodyOutputStream = new NoBodyOutputStream();
        private PrintWriter writer;

        public NoBodyResponseWrapper(HttpServletResponse response) {
            super(response);
        }

        public ServletOutputStream getOutputStream() throws IOException {
            return noBodyOutputStream;
        }

        public PrintWriter getWriter() throws UnsupportedEncodingException {
            if (writer == null) {
                writer = new PrintWriter(new OutputStreamWriter(noBodyOutputStream, getCharacterEncoding()));
            }

            return writer;
        }

        void setContentLength() {
            super.setContentLength(noBodyOutputStream.getContentLength());
        }
    }

    private class NoBodyOutputStream extends ServletOutputStream {
        private int contentLength = 0;

        int getContentLength() {
            return contentLength;
        }

        public void write(int b) {
            contentLength++;
        }

        public void write(byte buf[], int offset, int len) throws IOException {
            contentLength += len;
        }
    }
}

Conclusion

We now have a drop-in solution, compatible with any web framework and any container. It allows us to transparently support http HEAD requests in our applications and finally treat web crawlers and proxy servers as first class citizens.

The source code for HttpHeadFilter is available here. Feel free to use it as you wish.

Feedback is always welcome.

 

---

About Axel Fontaine

Axel Fontaine is the founder and CEO of Boxfuse the easiest way to deploy JVM and Node.js applications to AWS.

Axel is also the creator and project lead of Flyway, the open-source tool that makes database migration easy.

He is a Continuous Delivery and Immutable Infrastructure expert, a Java Champion, a JavaOne Rockstar and a regular speaker at many large international conferences including JavaOne, Devoxx, Jfokus, JavaZone, QCon, JAX, ...

You can follow him on Twitter at @axelfontaine