So, I looked into the logs and found:
14-Jan-2006 19:01:18 org.apache.pluto.portalImpl.services.log.LoggerImpl error
SEVERE: PortletInvokerImpl.render() - Error while dispatching portlet.
javax.xml.transform.TransformerFactoryConfigurationError: Provider org.apache.xalan.processor.TransformerFactoryImpl not found
at javax.xml.transform.TransformerFactory.newInstance(Unknown Source)
at com.plumtree.portlet.portlets.RssPortlet.init(RssPortlet.java:98)
at javax.portlet.GenericPortlet.init(GenericPortlet.java:100)
A little googling showed this was another instance of the JDK 1.5 XSL parser change. To fix, you just need to download and install Xalan into TOMCAT-HOME/common/endorsed and everything works fine.