Hello Marcus,
I don't have experience with FluidSynth on ARM, so this is more general in scope.
The ARM platform needs to have some sort of FPU in order to achieve any sort of realtime output (I know a lot of them do nowadays, so probably not an issue). As to latency, there are a lot of variables. FluidSynth itself can achieve very low latency (I think it would be the buffer fill size, which I believe is 64 samples by default - 64 samples a 48KHz is about 1.3ms). It really comes down to the rest of the system. FluidSynth will need to enable realtime SCHED_FIFO scheduling. The Linux kernel will play a huge role in achieving low latency without underruns. For very low latency, the Linux kernel will likely need to be built to be fully preemptive and may need to be patched (although I believe a lot of preemption related stuff has made it back into the stock kernel source). According to one information source, the preemption related code is platform independent and should work just as well on ARM as X86 (
http://lwn.net/images/conf/rtlws11/papers/proc/p11.pdf). The audio interface and drivers will most likely end up determining the lowest underrun free latency possible on a given architecture, provided there aren't other drivers which cause latency spikes.
I run the Ubuntu lowlatency Kernel on my laptop, which appears to not have PREEMPT-RT enabled and I can achieve 20ms without much issue (or somewhere there abouts, don't quote me on that ;-) Lower than that and I start to get occasional underruns, increasing the lower I go. This is with a USB Edirol 24 bit interface. I also have to turn off my Ethernet device, due to some driver or IRQ priority issue, which causes latency spikes at periodic intervals (like every 2 minutes or something).
Hope that helps.
Best regards,
Element