That's a wide open question and you will probably get many different answers. I prefer RME interfaces. They have a few USB interfaces depending on how many inputs and outputs you want. Take a look at the BabyFace or the Fireface UC. If you are on a Mac I would also look at Apogee, either the One or the Duet.
Take a look at Audacity or Reaper if you don't want to pay a lot for a DAW. On the Mac you can use Garage Band with decent results as well. At least starting out until you get some experience and can try out different DAWs. DAWs are interfaces are just like guitar amps, everyone has there favorite and nothing else compares.
Starting out mics you can't go wrong with a Shure SM57 or Senheiser e609 or e906.