This a an excellent question, Rockin' Cowboy. If you can wrap your head around these ideas, it will help you to understand most Western music. These 'rules' should not be thought of as rules, but descriptions of common practice. (I took two years of music theory at my local community college, my teacher was a concert pianist, and a fantastic musician and teacher.)
When we say that a piece of music is in C major, what we mean is that C major is the most prominent key of the piece. Most pieces start and end in the same key, but often modulate to other keys.
Here are the most common ways in which music modulates to other keys.
Modulation to 'closely related keys'. This means to a key with one sharp or flat difference in the key signature. In practice, this means that the tonic (the note with the key is named after) shifts up, or down, a fifth. That is five scale tones. So, in the Key of C major (no sharps or flats), the two closely related keys are G major (one #) and F major (one b). c,d,e,f,g (up five scale tones) c,b,a,g,f (down five scale tones). In the key of F, Bb is the IV chord. So, this is one way in which an Bb could occur in a piece of music in the key of C major. (or in the key of G major, and F major chord, the IV chord in the key of C major, a closely related key in G) Note: Taking away a sharp is the same as adding a flat, and taking away a flat is the same as adding a sharp. Example: If we are in 3 flats and we add a sharp, we are now in 2 flats.
Additionally, if we are in C major (no sharps or flats) we could modulate to 1#, but instead of going to G major, we could go to e minor. Or 1b, to d minor.
Another common modulation is to the relative minor. That is, the minor key with the same key signature. The relative minor of C major is 'a' minor. (the convention is to capitalize the note name of a major key, and use small cases for minor keys) in a minor key, a major V chord is used. In 'a' minor, the V chord is an E major chord. In the key of C, you will commonly find that an E major chord is used, usually followed by an a minor chord. If an e minor chord is used in 'a' minor, this is called the mode of 'a' minor, the KEY of 'a' minor uses an E major chord. The chord based on the fifth scale degree is called the 'Dominant chord'. That would be G or G7 in the key of C major, or the key of c minor. A g minor chord in the mode of c minor. The tonic of the relative minor key is 6th scale degree of the relative major key. The tonic of the relative major key is the 3rd scale degree of the relative minor key.
Another common modulation is to the 'Parallel Minor'. That is a minor key that has the same tonic as the Major key. In C major, the parallel minor is c minor.
Here, we need to explain how chords are formed, this is commonly called 'stacking thirds', that is skipping every other note in the key. For example, if we stack thirds in the key of C major, starting on the tonic, we get the notes, c, e, g. This is the tonic, a C major chord, containing the 1st(tonic), the third, and the fifth notes in the C major scale. If we want to make 7th chords, we use for notes; c, e, g, b, creating a C maj7 chord. The tonic chord is denoted with a Roman numeral I. In a minor key, the tonic is denoted with a small case Roman numeral i.
If we 'stack thirds' staring on the 2nd note of the C major scale, we get; d, f, a, an d minor chord, designated by the Roman numeral ii, lower case because it is minor.
All major keys follow this pattern, I ii iii IV V vi viio(diminished)
All minor keys follow this pattern, i ii(dim) III iv V VI VII
Notice that the VII chord in c minor is Bb major. I we are in G major, an F chord could be borrowed from g minor, the VII chord of g minor.
There is also the concept of 'Secondary Dominants'. The most common secondary dominant is the dominant of the dominant. In C major, the Dominant is a G major chord. The dominant of the dominant, denoted V/V is a D major chord. You will often find music in the key of C will have a D major chord (not in the Key of C major, C major has a minor ii chord i.e. d minor) followed by a G major chord i.e. V/V V.
Secondary dominants in the key of C major are:
V/ii - Amaj or A7, usually followed by dm
V/iii B7 - em
V/IV C7 - F (common) Note that C7 does not occur in the key of C, but the key of F.
V/V Dmaj - Gmaj
V/vi Emaj - am
V/viio Fmaj - b dim (same as IV viio) Note: vii0 has same function( dominant function) as the V chord.
Another common modulation is to the relative major of the parrellel minor. For example; if we are in C major, the parallel minor is c minor, it's relative major is Eb major.
Or, we can modulate to the parallel major of the relative minor. In C major, relative minor is 'a' minor, parallel major is A major.
If we are in a minor key, the relative minor of the parallel major. For example: If we are in c minor, the relative major is C major, it's relative minor is 'a' minor.
Or, to the parallel minor of the relative major, in c minor, there relative major is Eb major, the parallel minor; eb minor.
There is also something called the 'Protestant Bump'. This is usually done in a major key, we simply modulate up a half-step or a whole-step. That is, for example from C major to Db major or D major.
Here is another explanation for your example; an Fmaj chord occurring in the key of G. This is not uncommon in rock or country music. A song may be in the key of G, but use the I,IV and V chords from the key of C i.e. G,F, and C. One might analyze this as mixolydian mode harmonized I ii iiidim IV v vi VII. Note that these are the same chords that occur in C major, but the numbers have changed.
In C major: C, dm, em, F, G, am b dim.
In G mixo: G, am, b dim, C, dm, em, F
There is one more very important idea that modern musicians must take into account. That is the influence of the blues. A blues progression throws out the 'rules' discussed above (with the exception of the last idea, the mixolydian harmonization) and uses an entirely different basis.
A blues progression uses Dominant 7 chords for the I, IV and V chords i.e. I7, IV7, V7. These chords are often played over with the same minor blue scale. Or the same major blues scale, or a both at the same time, sometimes referred to as mixo-dorian, or scales with the same tonic as the root of the chord being played over.
This pretty much sums up the music theory of harmony and modulation (non-jazz).