As Todd states in his comment, there isn't "one way" to notate rhythms. After all, notation is just a partial representation of the sounds produced, and several may be possible and equally correct. I have an answer over here for more on this.
Therefore, the best notation will be the clearest expression of artistic intent. Some guidelines I suggest:
- Which note gets the beat?
I'd start instead by determining which notes are the most common and near the middle of the used durations. Most pieces rely heavily on about three durations - make these your quarter notes, eighths, and sixteenths if it's a faster feeling piece, and halves, quarters, and eighths if it feels slower. You want to make as many of the notes as possible easy to count, so if you have some much faster notes than "average", use that to inform your decision.
Once you get the rhythm of a passage roughed in, hopefully you'll start to see patterns. Your ear will tell you if a particular bit of music feels the same as another, so you can start to put in your barlines. If not, go on to the next question and come back to this.
- How do you identify syncopation vs odd time signatures?
Go for simplicity here. If the pattern keeps changing chances are you're looking at syncopation. If it's more regular, you'd be better off writing in that meter. You may try it with two time signatures and see which is easier to read.
Don't forget about time signatures such as 3+3+2/8. At slow speeds, this could feel like the meter is constantly shifting between 3 and 2, but with careful notation it can be clearer to read as 4/2 or 4/4. Also remember that it is possible to have a hemiola going through many measures, but that the underlying pulse might be the same.
When in doubt, choose the voice with the most repetition of pitches to look for clues as to what's going on. If you have a percussion or bass part, they usually are providing a rhythmic structure for other voices. If multiple voices keep having cross rhythms, at least one of them is syncopated.
- How do you identify time signatures?
In one ethnomusicology class, we were introduced to an eastern dance form in 5 that was described indigenously "limping dance". This has stuck with me as a way to identify odd time signatures - can a passage be danced in a "limping" way with some regularity? For example, a dance in slow five (think Mars) would feel like two dance steps with a limping third. A faster five might feel like two uneven steps.
Once you figure out whether you're dealing with an odd time signature or not, start tapping along. How many pulses are there in the measure? Are some strong and weak? If you're not sure, tap at double the speed of the main rhythmic progression and use that as the subdivision of the beat.
The number on the bottom of the time signature is determined by legibility. Are there a whole pile of notes in a measure? It's x/2 or x/4. Lots of notes that cover whole measures? x/8. Keep it easy to read.
Ultimately it all comes down to communicating intent effectively. It make take a few tries and some time away from the piece to get it right.