Could this work:
<xsl:value-of select="replace(base-uri(), $folderURI, '') "/>(did not see this at the comments), since if the problem is the absolute path at the xref, replacing the path with empty value should solve this problem. Or I just understand something terrible wrong. :) This is my tests end result:
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE topic PUBLIC "-//OASIS//DTD DITA Topic//EN""topic.dtd"><topic id="to_new_topics"><title>New topics</title><body><ul><li><xref href="en/file3.dita"/></li><li><xref href="file1.dita"/></li><li><xref href="file2.dita"/></li></ul></body></topic>Tested with: XSL, newtopic.xml (to avoid problems and renamed this .xml, not .dita) and all file*.dita files are in the same folder or subfolder.