I would say the "rhythmic center" (or perhaps "metrical center" would be better) would just be the notated barlines and the requisite downbeats.
But to get more advanced, the music theorist Harald Krebs has developed a system for this, a system commonly referred to as "metrical dissonance."
In short, a piece of music has a notated time signature, and within that time signature is a primary metrical layer. But at times, other metrical layers may be at odds with the metrical layer suggested by the time signature.
In 4/4, for instance, there may be a metrical layer that operates in 3/4. Because these two layers are of different cardinalities (one is a "4-layer," the other a "3-layer"), this is called a "grouping" dissonance.
But now imagine we're in 4/4, and we have another 4/4 metrical layer that is just misaligned with the written time signature. (For instance, maybe we have instruments playing their downbeats on beat 2, while other instruments are playing their downbeats on beat 1.) Since these metrical layers have the same cardinalities (both "4-layers") but are just misaligned, this is called a "displacement" dissonance.
These dissonance types are of course in opposition to the idea of "metrical consonance," which is when the metrical layers of a piece are all aligned.
I'm simplifying the theory considerably, but it's all laid out in his book Fantasy Pieces, with musical examples emphasizing metrical dissonance in the music of Robert Schumann.