Most of these books should be available online in PDF form for free. Specifically the FAA books. All covers below link to the Amazon site.
MUST HAVE BOOKS
Airplane Flying Handbook
Easily my favorite study guide. Easy to read, great illustrations. A must have book for any Private Pilot student, and its cheap.
Look for the latest edition.
Pilot's Handbook of Aeronautical Knowledge
Another one of my favorite study guides. Again, easy to read, great illustrations. Covers more on flying theory and medical conditions. A must have book for any Private Pilot student, and its cheap.
Look for the latest edition.
FAR/AIM 2011: Federal Aviation Regulations/Aeronautical
This information is all online, but the book is cheap and instructors like to see you with one every once in a while. Mostly reference, nothing you'll sit and read probably.
Look for the latest edition.
NICE TO HAVE BOOKS IF...
Say Again, Please
If you're struggling with the radio, this was a nice book to reference. Not exactly gospel but a good get you over the hump book. I also recommend visiting an ATC tower.
Aviation Weather Services: Advisory Circular AC00-45F
If you're struggling with WX. Well, you're screwed. Not really, but these two books were heavy reading. More like a White Page than a reference book.
Aviation Weather (FAA Handbooks)
Another book by NOAA. Look around the web for a PDF version.
Pilot's rules of thumb: Rules of thumb, easy aviation math, handy formulas, quick tips
If you're struggling with formulas, or just like to work the numbers a new way.