There is a Firefox bug causing that, it's quite difficult to consistently work around. I usually hear audio just fine, though, which makes me think it might be platform specific problems as well...

https://bugzilla.mozilla.org/show_bug.cgi?id=1178751