It seems that totem is broken quite badly on Ubuntu Hardy Heron at the moment. Video playback stutters along at around 0.25 frames a second and totem itself seg faults after clicking the seek bar a few times. Bug duly filed as part of an existing bug which I think is related.
You can see below the current state of totem on my laptop.
Lets hope it gets fixed soon!