Hello,
I suspect the system is running out of CPU resources. You can confirm that by running top or some other utility to show the CPU usage and correlate it with when the problem occurs.
Also, does the Orange Pi have a floating point unit? If not, that will greatly increase the CPU usage, due to FPU emulation.
In any case, you can set the polyphony maximum voice count to be low enough to not overwhelm the system. When FluidSynth runs out of voices it will start terminating the ones it thinks are the least noticeable. You do this by passing the "-o synth.polyphony" option. For example "fluidsynth -o synth.polyphony=32" to set the total number of voices to 32 (defaults to 256). The release phase of the instruments you are using also affect how long voices are used. If the release phase is excessively long or longer than needed for the sound, then it results in unnecessary voice usage. Using the synth.polyphony setting should help with that though anyways, but you may want to check the instruments you are using with a SoundFont editor to see if the release times are long.
Best regards,
Element Green
P.S. Your English is fine, no problems here :-)