You can use the free program 7-Zip to do this:
I believe you have to download their latest Beta version 9.0x to use it with Windows 7.
The latest versions as of 12/12/2009 can be downloaded from here:
If you have 32-Bit Windows 7 then use this:
http://downloads.sourceforge.net/sevenzip/7z909.exe
If you have 64-Bit Windows 7 then use this:
http://downloads.sourceforge.net/sevenzip/7z909-x64.msi
After 7-Zip is installed you can select multiple .zip files, right-click on them, navigate to the 7-Zip pop-up menu, and choose the:
Extract to "*" option .
This will extract each .zip file into its own folder named after the zip file.