I don't know what is causing your x-runs, but I can offer a few suggestions:
1. Try a different USB port? It is possible that your USB port and soundcard are sharing an irq - this would cause problems like you are having. (Are you using Sabayon / Gentoo on a USB key?)
Is a good place to start to see if this is a problem.
2. You can try a different kernel - a desktop kernel for AVLinux is here:
(2.6.32 image and headers) http://www.bandshed.net/kernels/
If a generic kernel performs better than the default realtime kernel, something is probably getting messed up with your rtprio scheduling. Or, it may be possible that your hardware just will work better with a generic kernel.
3. Run jack in synchronous mode - add -S as the first switch in the jack command line, or if you are using Jack Control, add it to the command line like this: /usr/bin/jackd -S