Michael Flynn's podcast, Legal Lad's Quick and Dirty Tips for a More Lawful Life, gives an excellent overview of the legalities of making an audio recording of someone in the USA when you are not a government agency performing lawful surveillance. When is it legal and when is it not? The short answer is: it depends - on whether there was a reasonable expectation of privacy, what state the conversation takes place in, and whether all parties being recorded gave consent.
For instance, if one is having a loud conversation on the street (a public place), how can one expect privacy from being overheard? If one can be overheard, how can he then complain that a record (or recording) was made of what he said? An example from daily life, that we can all probably identify with, is the standard phrase we hear when we dial technical support or customer service - "All calls are recorded for quality assurance." In other words - "We're recording you - you've been warned". By being advised and not hanging up, you will likely been seen as having given implied consent.
Legal Lad, being a lawyer as well as having good communication skills, gives a more thorough and educational overview of this subject.