I've never seen that. All I get is a small blue speaker icon in the bottom right. Tapping it allows me to select the output.
Maybe it's the bluetooth profile of my headset that's the problem. It's annoying as my headset doesn't have any volume controls.
I've just checked this on my 3GS (was using the 3G before) and it's the same, however, the sound delay on video seems to be gone with the 3GS. Or maybe it's the 3.1 that's fixed it.
I still need the volume control though.