In contrast to pitch, rhythm refers to the length and accentuation of a sequence of notes. If we focus exclusively on the durations, what would be a proper term?
Example: 1/4, 1/2, 1/4
no tempo, no time signature, no accents, just a sequence of relative durations.
Update: In the example above, 1/2 could be a rest (a minim), it's not that silence is not allowed, but the question is about the temporal microstructure of the sequence.
Update: Put it in a more precise way: I'm formalizing the combinatorics of how a bar (given a signature) can be organized in notes of durations that sum to that bar length. Say, a 3/4 defines a bar of an overall relative time of a dotted-half, what are the possible arrangements of notes (and rests) for this bar to a maximum of (say) two dotted-notes? The question then is: What am I studying in the scope of music? Rhythm, as I understand it, is too broad a term for just that.
Update: According to this manual of music theory, "rhythm" is the word for it. But accentuation is also a fundamental part of rhythm.