I agree, you really need a test drive to be sure.
The only thing that'll throw you (besides you getting used to the superb brakes ;-) ), is the CVT (continuously variable transmission). If you're used to "driving by sound" (know the speed that you're doing based on the sound of the engine), then driving the Prius at first can be a bit disconcerting. The engine will rev when you are going up an incline, or try to accelerate. Don't worry, the engine goes to whatever RPM is most efficient for the amount of load that you're putting on it, and it won't stress itself (rev-limiter to protect transmission).
Also, since you don't have any gear changes to let you know what speed you're doing, you can very quickly be speeding without realizing it (particularly when taking a left turn on a busy road). Many people have thought the car slow at acceleration, until they actually looked at the speedometer... (Or because they were afraid of the engine sounds, they let off of the gas pedal...)
Personally, I've never had a car pass me after a red light yet at an intersection that goes from 2 lanes to 1 immediately following the light (that I didn't purposely let them pass me). No tickets, either. 8)
Then, if you're not ready for it, the engine shuts off when you come to a stoplight and you think that you just stalled (and instinctively go to turn the key again)...
Anyhow, my dealership was great when it came to a test drive. I let them photocopy my license, then they handed me the keys and said "have fun." My husband and I (no salesperson with us) took the demo car out for some errands around town... It was at least an hour or so later that we brought the car back, and at least a 20 mile drive (didn't keep track). We immediately fell in love with the car based on the test drive.