I posted this over at http://www.topsony.com/forum/mobile-phones-f4/w580i-video-controls-t14025.html
Since I had this question too I figure it's good to spread the answer far and wide for teh good of the intarnets.
To Fast Forward video on the w580i using the included video player in portrait mode just hold down right on the directional pad, the button that has >> on it, hold down left (<<) to rewind. You can see the time go by in the bar at the top and let go when you get to the correct minute:second.
When you are in landscape mode you hold down the down button to fast forward and you hold down the up button to rewind..
Sony assumes you have turned the phone on it's side when you use landscape mode, that's why the FF button changes. Also, you don't have the nice bar to indicate the time position as you are fast forwarding so you have to just guess when to stop holding the button or switch back to portrait mode to fast forward or rewind.