I am not sure about being "quirky". At the end of the day you have to like what you've done and since music is an art form no one can say it's "wrong".
However, there are some classic patterns in western music that serve a purpose or function and this determines how chords follow each other in succession. I am not sure where you learned harmony theory, if you are self taught, by example, or from a book. But the first thing to know is how the I, IV, and V or V7, lay over the major scale. These are really the only three chords you need to harmonize a melody in a given key (major key as a reference).
I --> (1, 3, 5)
IV --> (4, 6, 1)
V7 --> (5, 7, 2, 4)
No surprise that the arpeggios of the chords are the notes they harmonize (most of the time). There is no law that says you can't harmonize the 2 with a IV chord, in fact this would be a IV6 and sound cool. But the classic approach to beginner harmony is to start with the above map.
Now it helps to understand the movement of the intervals from one note to another as that serves a purpose. It is often said in western music that the movement from IV-->I and V-->I have a sense of closure of finality. These are called cadences, or resolutions (I may be using vocabulary incorrectly). The V7-->I has the strongest sense of finality from among all possible changes. The IV-->I is often called the Amen cadence and appears a lot in church music. All of these changes have a few things in common among which are chromaticism, 4-->3 and/or 7-->8 in terms of scale degrees of the key signature. In fact the V7-->I has both of these. The IV--> only has 4-->3. To make the IV-->I cadence stronger some people move to the minor IV chord IV-->iv-->I. This introduces a chromatic walk down from 6-->b6-->5 in terms of key scale degrees.
This is not the only feature of a cadence but it is one that is very often and easily capitalized on. So, rather than just add "quirky" things, I'd encourage you to think about ways to introduce meaningful chord movement that follows either IV-->I or V-->I.
There are a few things you can do to "add" to your harmonies.
The first is to introduce what I'd call a cycle extension (or back cycle) in the progression, a very common trick. As as example let's say you are going to move from I to IV. Try putting the I7 right before the IV chord. This is a V7-->I cadence in the key of the 4th, and introduces an accidental relative to the original key, namely the b7. You can do this to your heart's desire and many scores are filled with such extensions. For example, C-->E7-->Amin-->C7-->F-->G7-->C, which is really just C-->Amin-->F-->G7 with the V7 placed strategically before the Amin and F chords.
Another way to spice up your chord progressions is through chord substitutions. This is very common in Jazz but applies to all forms of music. The idea starts with understanding how chords are related to each other. For example any Major chord, and its relative minor are so compatible they can be interchanged. Specifically, the chords I6 and vi-7 are identical. Next is any Major chord and the min a 3rd above it. These two add to create a Maj7, example I + iii = IMaj7. So the group of chords (I, iii, vi) can, in theory be thought of as serving the same function. More intricate substitutions include the b5 (or tritone) sub, which is most commonly applied to a V7 chord but could be inserted anywhere. This creates a lot of chromaticism. Following a circle of 5ths (or 4ths) like B7-->E7-->A7-->D7-->Gmaj, and applying this sub will get you B7-->Bb7-->A7-->Ab7-->Gmaj.
There are endless extensions of the things I've listed but in short these all follow a simple set of rules in harmony that attempt to create close movement into chord tones.
If you haven't looked at any books I'd recommend a few:
[1] Basic Theory-Harmony, A Text and Work Book for the School Musician
By Joseph Paulson, Irving Cheyette
[2] How to Create Jazz Chord Progressions by Chuck Marohnic
[3] Modulation by Max Reger