I am going to record music with the guitar and bass guitar. However, I am still undecided between using an amp with microphone, and an amp simulator. Which of the two would be a better choice when sound quality is concerned?
When using an amp with microphone, it means that you play the guitar through a physical amp, and using a microphone to direct this sound into your recording system.
An amp simulator is software that literally simulates an amp; you plug your guitar into your computer (through an interface for better quality), and this sound is modified by the software that you use.
Quality-wise, you can't just say that one is better than the other, so I will try to give you a list of common advantages to each option.
Advantages of using an amp with microphone:
(Note: the first two points are definitely up to debate, as high quality amp simulators will allow you to closely imitate these possibilities as well.)
Advantages of using an amp simulator:
The answer is: It depends.
A good amp connected to a good cab in a good room will sound good through a well placed good mic connected to a good preamp. I don't believe that any amp sim can beat that yet. But that's a lot of variables, many things can go wrong, it's not very hard but certainly not trivial.
To me, a good amp simulator sounds way better than a mediocre amp connected to a mediocre cab in a mediocre room through a mediocre mic placed in a mediocre way connected to a mediocre preamp. And as long as you have a good way to provide a good dry signal to the amp sim, it's also way more consistent and convenient.
Besides, amp sims are constantly improving. The earliest examples were just jokes and I think that's what gave them a bad name. But the latest generation amp sims are almost impossible to tell apart from a real amp when listened to. But they do still feel different when you actually play.
So, choice is yours; if you can, try both ways and listen.
Why not do both? Use a DI unit such as a Radial JDV or similar and run one output into your amp which is mic'd and the other into your audio interface and record the direct dry signal at the same time, then simply add an amp sim plugin and choose a tone to complement your amp tone.