I'm one of the biggest cheapskates you'll find when it comes to some things, and I was able to get a decent setup without spending a fortune. I've been doing my own recording off and on since the 90's and I can tell you that it is easier now than ever to do home recording. I have a regular HP computer with a generic soundcard and still am able to do great recordings. Here are some suggestions:
Recording Software - I use Reaper and recommend you getting it. You get to use it for 60 days free of charge (what I'm doing now actually). Then you buy for $60. Great product!
Interface - You have to find a way to get your amp, guitar tones, etc. into the computer. I use the Fractal Axe-FX II via a USB jack but you can use numerous tools, etc. You can use something like a LINE 6 POD, some folks mic their amps and use a mixer, etc., etc. It really depends on what you want to do. Just realize that micing your own gear takes a bit more patience than just plugging in a device that emulates tones, etc. Not really hard but you want to get the right mic placement, etc.
Down the road if you decide to mess with drums, you can get something like EZDrummer to make grooves you can jam along to. I recommend holding off on that for awhile and start out just trying to record a basic guitar part.