Let's see what the music21 Python library thinks about the example chord progressions.
from music21 import stream, harmony
s = stream.Stream()
for d in chords_string.replace('b', '-').strip().split():
print(guess_key('C Dm Am F'))
print(guess_key('Am Em F G'))
The output of our program:
For the A minor song, music21 agreed with you, but what's with the other one? Did the computer make a mistake, is F major wrong? Are you wrong and is Rihanna wrong? No, nobody is wrong. F major is one possible guess out of many, based on too little information.
Using different rhythmic emphasis, and the melody and depending on the listener, and even the mixing of the song, it is possible to use that exact chord sequence in a song that gets perceived by someone as being in C major or F major, and their relative minor keys Am and Dm as well.
Counter-examples that show it is not possible to determine a key from chord and pitch lists
Here is an example song with C - Dm - Am - F in F or Dm.
And here's another song with C - Dm - Am - F, but this time in C.
I deliberately avoided using the giveaway notes B and Bb anywhere in the melodies. And it's still a subjective issue. Maybe you hear those in a different key than I did. At least Dekkadeci felt the key was still C even with this melody. I certainly didn't. But from what I can see, with this counter-example, I demonstrated that in the general case, it is not possible to "correctly" determine a key from a chord sequence. And it doesn't help even if you're given the sets of all pitches in the songs. The pitch sets are the same in both songs: C, D, E, F, G, A.
But how then do you know what key you should say the song is in?
Just like you already did - it is what it feels like to you. Saying that a song is in a key is a subjective statement. If you say "this is in A minor", it means that you personally and subjectively feel that A minor is the most prominent center-of-balance of that song's harmony, and it's the least misleading thing to say. For a lot of songs it's a compromise anyway, and in a lot of Western pop music the center of balance is not very strongly on either the major key or its relative minor key.
Personally, for most intents and purposes, I consider A minor and C major to be the same key. A person standing on two feet, sometimes their weight is slightly more on the left foot, sometimes on the right. But our language for talking about music has evolved in old times where people could only afford to wear a shoe in one foot at a time, so when we say that a person is standing, it is not possible to say that without specifying either the left or the right foot as the one that the person is standing on.
You have this question because of limitations of our language conventions. If there was an established and known way for saying "the key with C and Am being the joint center of balance without either being significantly more dominant", you would use that idiom. But there isn't, so you don't, and you have this problem.
To develop a sensitivity to this major-or-minor aspect, play existing songs and see how they have been categorized to either major or minor by more experienced people. You can only learn this by doing. It is about skill and experience, not knowledge and logic that you could calculate.
To address the literal question in the title: Is it possible to determine the major/minor key of a song by just looking at the chords?
Sometimes yes, sometimes no. If you see things like Am - Dm - B7 - E7 - Am, you can be pretty sure that it's in A minor. But even the existence of E7 - Am somewhere doesn't mean it's in A minor. Sometimes it is completely ambiguous. Sometimes the tonic chord is not even included in the song at all, the harmony goes around it without ever going "home". You have to listen to it. Here's a song that's clearly in A minor, but there is no Am chord in the whole song, not even once.
(Some theory purist / language correctness police might insist that it must be not said to be in a minor key because there's no leading tone. I say that languages evolve, and the modern de-facto meaning of "key" doesn't require a leading tone.)
Algorithms can do more, if they have more information
All that said, it is possible to get better guesses with algorithms, but you'll have to give them more than just a list of chords and a set of pitches. I took my example songs above and exported them from Sibelius a MIDI files, in different key signatures to demonstrate that Music21 is not just looking at the given key signature (in the MIDI metadata event).
The new program
from music21 import midi
mf = midi.MidiFile()
And the output of the program is:
Here's one of the songs imported to GuitarPro:
So, Music21 is able to figure out that the song is in F, despite the deceptive key signature meta data. But only if it has the melody as well as the chords. I don't know what the key-guessing is based in, maybe some statistical calculation, i.e. how many times each pitch is used. I haven't tried, but I'm pretty sure it wouldn't place the Santrofi song in A minor.
By popular request, here's one more song that starts with a C chord, does not use a Bb note anywhere, and is solidly in F or Dm.