I think I know the trap you have fallen into.
In the score below I circled the first note of each measure in green.
Those four notes map out the
D major chord. Naturally you might play
D major for each bar. If you are using basic guitar chord, those
D chords will be in root position.
However, in the classical style of this music the melody
^3 ^2 ^1 - that's the
F# E D in measures 3 to 4 - would not be harmonized with chords
D | D. Measure 3 needs to be a chord of the dominant or the
D chord in 2nd inversion.
If you are not dealing with classical style, @ggcg's answer about
I IV V will serve you well!
The rest of what I have written is just detail about this tune could be harmonized.
A common method for harmonizing a melody will tell you to match melody tones to corresponding chords from the scale. Example, in D major, take the note
F#, it is found in the three triads
F#m. One of those chords is a possibility for a harmonization. However, if you do that for all of the tones of a melody, you quickly have a big, jumbled list of possible chords with no sense of how to select from the possibilities. The problem with that is it focuses on a matrix of pitches instead of starting with rhythm and phrasing as the basis for harmonization.
There is a deep connection between harmony, rhythm, and the barline that should be satisfied with a good harmonization.
Before selecting chords assess the rhythm of the music especially the latent harmony of the line. Where does the line pause? Are any chords outlined by the line?
Especially in this style of music phrases end with cadences. If you identify those phrases and select appropriate cadences first, you know where the music needs to go structurally.
Also, we want to think about levels of harmony. The idea is that harmony can be simplified or expanded without loosing the essence of the harmonic functions. Example, a single chord
I could be expended to
I IV6/4 I. We can use this idea to sort of sketch up from basic harmony to a final harmonization, or to break down an existing harmonization to see its broad organization.
Let's take this existing harmonization and work backwards from the finished product to see the principle applied.
Initially, forget about pitches. Where are the long notes? Measures 2 and 4 get the long notes. Both have a dotted quarter note on beat one. These are our resting points. They should be our harmonic destinations/goals. How about shortest notes? the last notes of measures 1 and 2 are a 16th note and 8th note respectively. Whatever happens on those notes can be considered harmonically subordinate to the main notes. They are details we can gloss over for the moment.
Now let's consider pitches. The first note is
D the tonic. Long note, measure 2 is
A the dominant. Long note, measure 4 is
D the tonic. Also, the end of the phrase - measures 3 to 4 - gives
^3 ^2 ^1 that should leap off the page as a cadential formula.
A general sketch of our pitches so far would fit the rough sketch
I |V I which I tried to indicate with brackets at the bottom.
Work backward from the cadence and fill in the detail. Essentially we want to write the cadence and then connect it to the beginning. The melody segment
^3 ^2 ^1 is a common compound cadence formula. Zero creativity is involved, just fill in the formula. (I used parenthesis to show that the
I6/4 can be viewed as a
V with an appoggiatura above, but it really doesn't matter, it's just a cadential formula.) That takes care of 2 of 4 measures.
The next question is how to treat the melody segment
^1 ^7 ^6 ^5 for measures 1 and 2. The
^5 in bar 2 is the tricky note. It could be part of a tonic chord or a dominant chord. (We will skip over why the mediant chord wouldn't be used.) This is the moment where a mere matrix of melody tones and matching chords will fail us. We aren't simply matching melody to possible chords. We are trying to connect to whatever chord comes next! This is why it was important to find the cadence first. When we know where we are going we can decide how to get there..
^5 in measure 2 could be harmonized with a dominant chord, but then it would be setting up a repeat of the dominant chord in measure 3 and that would cause the harmony to become static. Better to use the tonic chord
I in measure 2 as a prolongation of the opening tonic chord in bar 1. Now we have gone from a harmonic sketch to the particular harmony for each of our 4 bar phrase
I | I | V | I.
The remaining material is harmonically subordinate. Because this material isn't essential we actually have more flexibility with how it could be handled. This harmonization gives us three common devices:
- treat a melody note as a non-chord tone, the
^7 passing tone in measure 1.
- prolong a chord with a pedal 6/4 (or passing chord), the
IV6/4 between the two
I chords of measures 1 and 2.
- use a pre-dominant chord (chord
ii6 in measure 2.
Finally the basic method is...
- identify phrases and fill in cadences
- sketch in the primary chords
- fill in the subordinate material