Recently, I have been playing with my site to improve its load speed since >5sec was pretty slow and I wanted to learn a bit more on the dark magic of load time optimization. I was able to improve the page load speed to less than 1s (most of the time) using various techniques I found online. In short, here is a brief list of the things that I found to be quite useful in the optimization process, and you can google for more detailed step-by-step tutorials.
Benchmark tools: http://tools.pingdom.com http://www.webpagetest.org
Both of them show you a waterfall graph of the resources being loaded, and are pretty consistent to use as a baseline.
Guideline: Yslow plugin http://developer.yahoo.com/yslow/
This plugin detects the elements on the current page and gives you a score according to a set of optimizations that you should have. It serves as a perfect guideline to what you have already done and what else you can do to optimize more. Luckily in wordpress, we have several plugins that with a few clicks can get you a decent score and significantly improve you page load time.
Some specific things I found useful for WordPress:
1. Uninstall unnecessary plugins or plugins that have resources which take forever to load. It is very common for people to start with a brand new installation and quickly grow to a site with 30+ plugins in a few days of operation. But keep in mind that every plugin will incur some performance overhead and security issues, thus you need to think about whether you really need this plugin before you keep it. =D
2. Make sure your template is optimized. Even though many performance plugins can automatically optimize a lot of things to improve page load time, template optimization is not one of them. A poorly designed template, such as those that load 30+ small images for every page, can greatly increase the number of requests a client has to make to load one page and incur a huge network overhead. CSS sprite is a common trick to combine multiple small images into a big one, thus greatly reducing the number of requests, but it often has to be done manually and can be very tricky to get right without breaking the existing template, depending on the complexity and design pattern of your template.
That’s all I have for now, and hope it helps. =D