Marc,
When I look at your page load process, I see some major inefficiencies. While I wasn't able to replicate your precise issue, here are some observations:
http://www.webpagetest.org/result/12042 ... 1/details/. This report shows that some of your individual images take more than 15 seconds to load. This is just too high.
Also, YSlow reports that your site is loading an alarming 90 JS files. Loading each file requires making a server request and adds an overhead. My guess is that most of your scripts are coming from plugins. Suffusion loads just suffusion.js and jquery-cycle, in addition to some common scripts such as JQuery. My suggestion would be to make use of a caching plugin to combine these scripts. Also use something like Smush.it to optimize your images.
I also see that your site is running a total of 120 queries. This is approximately 100 more than what a standard install of Suffusion uses. Even with featured content turned on I don't get that many hits (the Suffusion page uses just 30). Now, the Magazine layout has some inherent inefficiencies in how it does things, because it has to run multiple queries (featured content, headlines, excerpts, each category block). So you might not be able to get the figures down by much, but I believe about 25% reduction is possible.