Andrew Kantor has a well researched column in USA Today on what you can (and can not) photograph in the US. He also links to multiple sites, including one with a handy brochure written for photographers that summarizes the rules and another written with Missouri in mind, but which appears to generally apply elsewhere.
In case you aren't aware of it, US privacy laws place restrictions on taking pictures or making videos based on a person's reasonable expectations of privacy. There are also restrictions about taking pictures of military installations and the like - and after 911, we can all appreciate that.
(Note: I am not a lawyer and can not give legal advice. Like Kantor says, I am researching and reporting, not giving a legal opinion, so take my comments as holding no legal weight whatsoever.)
Hat tip Geek Press.