I've seen reports of this, for example here:
https://lists.gnu.org/archive/html/lilypond-devel/2016-09/msg00045.html
I have the same issue on OSX 10.12.3, using 2.19.48-1
From running with --verbose, I see it is rebuilding the font cache each time. I tried various things like checking permissions and also removing the font cache to see if it would like it if were rebuilt with 2.19. But it also didn't have an impact.
2.18.2-1 doesn't have this issue.
Would it be appropriate for me to add this as a bug in the sourceforge page? It does not result in incorrect output, just an extremely long run time while it rebuilds the cache. For example, 50 seconds vs 2 seconds.