Time signatures and bars are not there arbitrarily, nor just to help count your way through a piece. They are there to provide guidance on the rhythm of the piece. Where it is accented, where it breathes.
Some composers do write pieces with no time signature or bars, as an indication that there should be no consistent rhythm. Eric Satie did this for several pieces.
Let's try doing what you've suggested with a well known nursery rhyme, first in 4/4:
| 1 2 3 4 | 1 2 3 4 |
| Half a pound of tupenny rice. | Half a pound of trea- cle. |
| 1 2 3 4 | 1 2 3 4 |
| That's the way the money goes. | Pop goes the wea- sel |
You can see, this puts one line of verse in each bar.
Half a pound of tupenny rice.
Half a pound of treacle.
That's the way the money goes.
Pop goes the weasel.
If you perform it with a strong accent on the first beat, and weak accent on the third beat of each bar, the accents fall where you'd expect them, if you were speaking the line normally: "Half a pound of tup-enny rice."
Now what if we divide it into 3 beat bars instead:
| 1 2 3 | 1 2 3 | 1 2 3 |
| Half a pound of tupenny | rice. Half a pound of | trea- cle. That's the |
| 1 2 3 | 1 2 3 | 1
| way the money goes. | Pop goes the wea- | sel
Half a pound of tupenny
rice. Half a pound of
treacle. That's the
way the money goes.
Pop goes the wea-
sel.
As you can see, it makes a nonsense of the verse. Try clapping a 3/4 oom-pah-pah rhythm and chanting the verse along to it.
Or, going in the opposite direction, something that should be in 3/4:
| 1 2 3 | 1 2 3 | 1 2 3 | 1 2 3 |
| Here comes a | can-dle to | light you to | bed |
| 1 2 3 | 1 2 3 | 1 2 3 | 1 2 3 |
| Here comes a | chop-per to | chop off your | head |
... into 4/4
| 1 2 3 4 | 1 2 3 4 | 1 2 3 4 |
| Here comes a can- | dle to light you | to bed |
| 1 2 3 4 | 1 2 3 4 | 1 2 3 4 |
| Here comes a chop- | per to chop off | your head |
So, "Here comes a can-dle to light you to bed" becomes "Here comes a can-dle to light you to bed."
Of course, not all music has lyrics that you can "break" in this way. But even instrumental music has rhythm, accents, breathing space and flow, which change if you put the bar lines in different places.
It is possible to adapt a tune from one time signature to another while keeping the accented notes at the start of the bars - but this necessarily involves modifying the rhythm of the notes, for example, changing two half-notes into a single note. For example, listen to Bob Dylan's Highway 61 Revisited, which is in 4/4, then PJ Harvey's adaptation in 3/4 (or 12/8, depending on how you choose to count it).