I would imagine that the 2004 Prius would be excellent in stop-and-go driving. That's where it really shines.
You can get good mileage with a diesel, but you will cause a lot of pollution also.
Reliability: You'll never find a more reliable car than a Toyota or a Honda.
Handling: I live in Fargo, ND. We don't get large snow accumulations, but we do get a lot of compacted snow and glare ice. The 2004 Prius with VSC (vehicle stability control -- an option included in the #7 and #9 packages) handles marvellously on slippery roads.
An all-wheel drive has two advantages: it can plow through deeper snow, and it can get going quicker on ice. But the traction control (standard) in the Prius works really well. And the drawback of AWD is that you think you can go through anything, so you go where you shouldn't, and you get stuck more often. I once owned a jeep, and I got stuck more often because I over-estimated what the car could do.
What kind of stop-and-go is your route? If there are a lot of very short, slow segments (from one house to the next) the Prius will often creep on electric power ("stealth" mode) saving on gas. If there are longer stops where you have to get out of the car, the gas engine shuts off, saving gas in comparison with a diesel or conventional car. Note that the Prius has much less wear and tear on starts because it is designed to stop and start a lot in normal operation, and uses a whole different method of starting.
In my opinion, the only situation where a Prius would not be ideal is if you must travel on muddy or heavily-snow-covered roads where AWD is really necessary.
Note also that the 2004 Prius is a hatchback, and the back seats fold down, giving you an enormous amount of cargo space.