#+options: tex:dvipng is not good because it turns all your latex symbol markup into images, which are inserted into the web page as images, i.e., it's not doing MathJax. Also bad because they don't really fit the text lines, either. I use
# this makes MathJax work
#+OPTIONS: tex:t
I just now found this caption weirdness because I started a new .org file with no latex symbol markup besides in the caption. Quite bizarre that when my file sees other latex symbols, the caption symbols then render correctly.
BTW: What does
executing Scheme code block (square)...
=> #<void>
org-babel-script-escape: ‘org-babel-script-escape’ expects a string
mean? I'm suddenly getting this in an org file which has scheme babel code blocks, images, and lots of latex symbol markup. It first popped up when I tried to C-c C-c a scheme code block.