It depends on what you want to accomplish:
There are basically two routes to go and a hybrid:
PC with Software
You use a VST host and do all the aplifiery bits (the tone shaping, distortion etc.) inside the computer in software.
Software wise you will need a VST host application of your choice (really, any will do), and effects, amplifier modelers and guitar cabinet modeler plugins.
There is both great free and commercial software for this out there: A good starting point would be the excellent free "le poulin" plugins together with effects plugins you may find googling for free "VST effects". Search the interwebs for "cabinet impulse files", which are basically models of guitar cabinets and can be used by most cabinet modeling software (I recommend Le poulin's "Lecab").
In your VST host the signal chain then goes something like this: Input -> Effects -> Amp -> Effects -> Cab simulator -> Effects -> Output
Of course you are free to put any or none effects wherever you want (Reverb generally goes after cab simulator, while distortion goes before amp etc.).
If you want to foray into commercial software, there are three major contenders I know of: Amplitube (which also offers some of it's plugins free), Guitar Rig and Revalver. These come packaged with everything you need effects, amp and modeler wise. At least some of them can also be used in a standalone fashion (you just run the program, no VST plugin host needed).
Hardware wise you can run directly into your sound card, but you will have high latency issues and bad sound quality. You can get yourself a USB audio interface (essentially an external sound card) to alleviate this. They set you back about 130€ if you want a decent one (a cheap one might do the job, too).
- You can play with headphones or computer speakers and it will sound
good at any volume.
- Unlimited flexibility Layout wise. You can use each plugin multiple times without having to buy it again. You can do stuff like reverb after cabinet simulator where you would have to mic a cabined and record it to do in hardware.
- Some of the free stuff is professional studio grade. Sound quality is way above practice amps.
- You can easily record yourself, practice over it, write yourself some drum loops to play along etc. The sky is the limit. Perfect for improvising.
- You are not locked into a limited number of sounds.
- Can't gig with it.
- Suffers from feature overload. You tend to tinker more and play less.
- Might be complicated if you are bad with computers.
- Lacks the mojo.
- Can't gig with it (well, you can, but I wouldn't. It's easier to just buy an amp).
The cheapo practice amp
You get yourself a little practice amp, like a roland cube, Line6, blackstar ID, etc. You can get additional pedals or multieffects units, though most such amps offer some built in effects. Not much to say here. Most of these also have headphone outs.
- Easy to use.
- Can gig or just go to a friend and play together.
- Takes more space.
- Might not sound as good as software.
- If you want additional good effects, you have to buy them.
The hybrid approach
Get yourself a little practice amp that has a USB audio out. Make sure that it has a "dry" output (that is, the unaltered guitar signal, basically acting as the audio interface). Now you can use both approaches with one device! Most of the newer practice amps have a USB output, I myself have a Yamaha THR-10, which does the job, but I have to confess that nowadays I only use it as an audio interface and never on it's own, as I have other physical amps when not at my PC.
It depends on what you are looking for, at least for my uses (which include recording and jamming with myself) the software setup is superior.
PS: If you use your PC's sound card to connect a guitar, get yourself the free ASIO4All driver. It's a low latency driver so the VST program can directly access your sound card. If you just use the regular driver, there is a noticeable delay between hitting a string and sound coming out of the speaker.