Search engine-friendly links with mod_rewrite

Introduction

One of the most frequent questions posted in the Apache Server forum is “How can I change my dynamic URLs to static URLs using mod_rewrite?” So this post is intended to answer that question and to clear up a very common misconception.

Mod_rewrite cannot “change” the URLs on your pages

First, the misconception: Mod_rewrite cannot be used to change the URL that the visitor sees in his/her browser address bar unless an external redirect is invoked. But an external redirect would ‘expose’ the underlying dynamic URL to search engines and would therefore completely defeat the purpose here. This application calls for an internal server rewrite, not an external client redirect.

It’s also important to realize that mod_rewrite works on requested URLs after the HTTP request is received by the server, and before any scripts are executed or any content is served. That is, mod_rewrite changes the server filepath and script variables associated with a requested URL, but has no effect whatsoever on the content of ‘pages’ output by the server.

How to change dynamic to static URLs

With that in mind, here’s the procedure to implement search engine-friendly static URLs on a dynamic site:

Better performance in production servers is possible with proper configuration of JVM parameters, particularily those related to memory usage and garbage collection.

The allocation of memory for the JVM is specified using -X options when starting Resin (the exact options may depend upon the JVM that you are using, the examples here are for the Sun JVM).

unix> bin/httpd.sh -Xmn100M -Xms500M -Xmx500M
win> bin/httpd.exe -Xmn100M -Xms500M -Xmx500M
install win service> bin/httpd.exe -Xmn100M -Xms500M -Xmx500M -install

It is good practice with server-side Java applications like Resin to set the minimum -Xms and maximum -Xmx heap sizes to the same value.

For efficient garbage collection, the -Xmn value should be lower than the -Xmx value.

Heap size does not determine the amount of memory your process uses

If you monitor your java process with an OS tool like top or taskmanager, you may see the amount of memory you use exceed the amount you have specified for -Xmx. -Xmx limits the java heap size, java will allocate memory for other things, including a stack for each thread. It is not unusual for the total memory consumption of the VM to exceed the value of -Xmx.

(thanks to Rob Lockstone for his comments)

There are essentially two GC threads running. One is a very lightweight thread which does “little” collections primarily on the Eden (a.k.a. Young) generation of the heap. The other is the Full GC thread which traverses the entire heap when there is not enough memory left to allocate space for objects which get promoted from the Eden to the older generation(s).

If there is a memory leak or inadequate heap allocated, eventually the older generation will start to run out of room causing the Full GC thread to run (nearly) continuously. Since this process “stops the world”, Resin won’t be able to respond to requests and they’ll start to back up.

The amount allocated for the Eden generation is the value specified with -Xmn. The amount allocated for the older generation is the value of -Xmx minus the -Xmn. Generally, you don’t want the Eden to be too big or it will take too long for the GC to look through it for space that can be reclaimed.

See also:

• Troubleshooting Technique: Garbage Collector
• Sun documentation on garbage collection

Each thread in the VM get’s a stack. The stack size will limit the number of threads that you can have, too big of a stack size and you will run out of memory as each thread is allocated more memory than it needs.

The Resin startup scripts (httpd.exe on Windows, wrapper.pl on Unix) will set the stack size to 2048k, unless it is specified explicity. 2048k is an appropriate value for most situations.

-Xss determines the size of the stack: -Xss1024k. If the stack space is too small, eventually you will see an exception class java.lang.StackOverflowError .

Some people have reported that it is necessary to change stack size settings at the OS level for Linux. A call to ulimit may be necessary, and is usually done with a command in /etc/profile:

ulimit -s 2048

JDK 5 includes a number of tools that are useful for monitoring the JVM. Documentation for these tools is available from the Sun website. For JDK’s prior to 5, Sun provides the jvmstat tools .

The most useful tool is jconsole. Details on using jconsole are provided in the Administration section of the Resin documentation.

win> ./httpd.exe -Dcom.sun.management.jmxremote
unix> bin/httpd.sh -Dcom.sun.management.jmxremote

 ... in another shell window ... 

win> jconsole.exe
unix> jconsole

Choose Resin's JVM from the "Local" list.

jps and jstack are also useful, providing a quick command line method for obtaining stack traces of all current threads. Details on obtaining and interpreting stack traces is in the Troubleshooting section of the Resin documentation.

Source: http://www.caucho.com/

Server caching can speed dynamic pages to near-static speeds. Many pages created by database queries only change every 15 minutes or so, e.g. CNN or Slashdot. Resin can cache the results and serve them like static pages. Resin’s caching will work for any servlet, including JSP and XTP pages. It depends only on the headers the servlet returns in the response.

By default, pages are not cached. To cache, a page must set a HTTP caching header.

Resin’s caching operates like a proxy cache. It’s controlled by the same HTTP headers as any proxy cache. Every user shares the same cached page.

Setting the max-age header will cache the results for a specified time. For heavily loaded pages, even setting short expires times can significantly improve performance.

Note, pages using sessions should not be cached, although more sophisticated headers like “Cache-Control: private” can specify caching only for the session’s browser.

The following example sets expiration for 15 seconds. So the counter should update slowly.

<%@ page session="false" %>
<%! int counter; %>
<%
response.addHeader("Cache-Control", "max-age=15");
%>
Count: <%= counter++ %>

max-age is useful for database generated pages which are continuously, but slowly updated. To cache based on something with a known modified date, like a file, you can use If-Modified.

When designing and testing your cached page, it’s important to see how Resin is caching the page. To turn on logging for caching, you’ll add the following to your resin.conf:

<log name="com.caucho.server.cache"
     path="log/cache.log"
     level="fine"/>

The output will look something like the following:

[10:18:11.369] caching: /images/caucho-white.jpg etag="AAAAPbkEyoA" length=6190
[10:18:11.377] caching: /images/logo.gif etag="AAAAOQ9zLeQ" length=571
[10:18:11.393] caching: /css/default.css etag="AAAANzMooDY" length=1665
[10:18:11.524] caching: /images/pixel.gif etag="AAAANpcE4pY" length=61

...

[2003/09/12 10:18:49.303] using cache: /css/default.css
[2003/09/12 10:18:49.346] using cache: /images/pixel.gif
[2003/09/12 10:18:49.348] using cache: /images/caucho-white.jpg
[2003/09/12 10:18:49.362] using cache: /images/logo.gif

An application can also set the Expires header to enable caching, when the expiration date is a specific time instead of an interval. For heavily loaded pages, even setting short expires times can significantly improve performance. Sessions should be disabled for caching.

The following example sets expiration for 15 seconds. So the counter should update slowly.

<%@ page session="false" %>
<%! int counter; %>
<%
long now = System.currentTimeMillis();
response.setDateHeader("Expires", now + 15000);
%>
Count: <%= counter++ %>

Expires is useful for database generated pages which are continuously, but slowly updated. To cache based on something with a known modified date, like a file, you can use If-Modified.

The If-Modified headers let you cache based on an underlying change date. For example, the page may only change when an underlying source page changes. Resin lets you easily use If-Modified by overriding methods in HttpServlet or in a JSP page.

The following page only changes when the underlying ‘test.xml’ page changes.

<%@ page session="false" %>
<%!
int counter;

public long getLastModified(HttpServletRequest req)
{
  String path = req.getRealPath("test.xml");
  return new File(path).lastModified();
}
%>
Count: <%= counter++ %>

If-Modified pages are useful in combination with the cache-mapping configuration.

Caching servlets is exactly like caching JSP pages (or XTP or static files.) Resin’s caching mechanism works like a proxy cache: it don’t care how the page is generated; as long as the proper caching headers are set, the page will be cached.

package test;

import javax.servlet.*;
import javax.servlet.http.*;
import java.io.*;

public class TestServlet extends HttpServlet {
  int counter;

  public long getLastModified(HttpServletRequest req)
  {
    String path = req.getRealPath("test.xml");
    return new File(path).lastModified();
  }

  public void doGet(HttpServletRequest req,
                    HttpServletResponse res)
    throws IOException, ServletException
  {
    PrintWriter out = res.getWriter();

    out.println("Count: " + counter++);
  }
}

Because Resin follows the HTTP caching spec, setting caching on a page which relies on sessions will make the page cacheable. You can test this by hitting the following page with separate browsers.

<% response.addHeader("Cache-Control", "max-age=3600"); %>

session id <%= session.getId() %>

Normally, this behavior is not what you want. Instead, you may want the browser to cache the page, but not let other browsers see the same page. To do that, you’ll set the “Cache-Control: private” header. You’ll need to use addHeader, not setHeader, so the browser will get both “Cache-Control” directives.

<%
  response.addHeader("Cache-Control", "max-age=3600");
  response.addHeader("Cache-Control", "private");
%>

session id <%= session.getId() %>

Resin can cache subpages even when the top page can’t be cached. Sites allowing user personalization will often design pages with jsp:include subpages. Some subpages are user-specific and can’t be cached. Others are common to everybody and can be cached.

Resin treats subpages as independent requests, so they can be cached independent of the top-level page. Try the following, use the first expires counter example as the included page. Create a top-level page that looks like:

<% if (! session.isNew()) { %>
Welcome back!
<% } %>

<jsp:include page="expires.jsp"/>

In some cases, you’ll want to have separate cached pages for the same URL depending on the capabilities of the browser. Using gzip compression is the most important example. Browsers which can understand gzip-compressed files receive the compressed page while simple browsers will see the uncompressed page. Using the “Vary” header, Resin can cache different versions of that page.

<%
  response.addHeader("Cache-Control", "max-age=3600");
  response.addHeader("Vary", "Accept-Encoding");
%>

Accept-Encoding: <%= request.getHeader("Accept-Encoding") %>

In many cases, logged in users get specialized pages, but anonymous users all see the same page. In this case, you can still take advantage of Resin’s caching, but you’ll need to do a little work in your design.

First, you’ll need to create an include() subpage that contains the common page. The top page can’t be cached because it depends on whether a user is logged in or not.

You must use include() because forward() is cached just like the top page. The top page isn’t cacheable because of the user login, so the forwarded page isn’t cacheable either.

Here’s what a sample subpage might look like:

<%@ page session=false %>
<%! int counter; %>
<%
long now = System.currentTimeMillis();
response.setDateHeader("Expires", now + 15000);

String user = request.getParameter("user");
%>
User: <%= user %> <%= counter++ %>

The top page slightly trickier because it needs to pass the user to the subpage. You need to pass a unique id. If you pass a boolean logged-in parameter, all logged in users will see the same page.

<%@ page session=true %>
<%
String user = getSomeSortOfUniqueUserId();
if (user == null)
  user = "Anonymous";
%>

...

Of course, the top-level page could also be a servlet:

...

String user = getSomeSortOfUniqueUserId(request);
if (user == null)
  user = "Anonymous";

RequestDispatcher disp;
disp = request.getRequestDispatcher("/cachedpage.jsp?user=" + user);

disp.include(request, response);

Resin includes an anonymous user caching feature. If a user is not logged in, she will get a cached page. If she’s logged in, she’ll get her own page. This feature will not work if anonymous users are assigned cookies for tracking purposes.

To make anonymous caching work, you must set the Cache-Control: x-anonymous header. If you omit the x-anonymous header, Resin will use the Expires to cache the same page for every user.

<%@ page session="false" %>
<%! int counter; %>
<%
response.addHeader("Cache-Control", "max-age=15");
response.addHeader("Cache-Control", "x-anonymous");

String user = request.getParameter("user");
%>
User: <%= user %> <%= counter++ %>

The top page must still set the Expires or If-Modified header, but Resin will take care of deciding if the page is cacheable or not. If the request has any cookies, Resin will not cache it or use the cached page. If it has no cookies, Resin will use the cached page.

When using x-anonymous, user tracking cookies will make the page uncacheable even if the page is the same for all users. Resin chooses to cache or not based on the existence of any cookies in the request, whether they’re used or not.

cache-mapping assigns a max-age and Expires to an If-Modified cacheable page. It does not affect max-age or Expires cached pages. The FileServlet takes advantage of cache-mapping because it’s an If-Modified servlet.

Often, you want a long Expires time for a page to a browser. For example, any gif will not change for 24 hours. That keeps browsers from asking for the same gif every five seconds; that’s especially important for tiny formatting gifs. However, as soon as that page or gif changes, you want the change immediately available to any new browser or to a browser using reload.

Here’s how you would set the Expires to 24 hours for a gif, based on the default FileServlet.

<web-app id='/'>
  <cache-mapping url-pattern='*.gif'
                 expires='24h'/>
</web-app>

The cache-mapping automatically generates the Expires header. It only works for cacheable pages setting If-Modified or ETag. It will not affect pages explicily setting Expires or non-cacheable pages. So it’s safe to create a cache-mapping for *.jsp even if only some are cacheable.

Source: http://www.caucho.com/