dirk,
Hmmm, are you using JACK that AV Linux came with or an updated package? What sound hardware ALSA, Firewire, USB? Are you certain your user has realtime JACK Permissions? This is very likely not an Ardour problem but if you want to ensure you have the build dependencies as root run 'apt-get build-dep ardour' (without quotes) further to that you can build a package by using the further commands here:
http://www.cyberciti.biz/ref/apt-dpkg-ref.html or by using the traditional Ardour scons build.
You may find the Ardour forum a more specific source of help for this particular issue
As for XFCE it was included as a courtesy in AV Linux 2.0r2 it will not be included in the upcoming 2.0r3 due to my limited time in being able to troubleshoot 2 very different D.E's. Of course users can install whatever they like on top of LXDE but I only use (and support) LXDE personally.