First, what's the instrument you want to play? If you are interested in it, you'll be more willing to practice, and, therefore, more likely to become a proficient musician using whatever instrument that is. For the most part, I wouldn't worry so much about how easy the instrument is to learn if you're really wanting to learn it anyway. I'd argue that you can teach yourself pretty well any instrument given enough time and effort.
Also, "portable" and "cheap" are very relative terms, so I'll try to give you (and whomever else reads this) a variety of ideas to help you find what best fits your needs. Also, anyone that wants to add to this - be my guest.
The harmonica is a great choice (I have no clue how to play it). It's about as compact as an instrument can get, and it can go surprisingly well with many songs. No other equipment is needed to amplify the sound, either. I've seen fairly decent, brand-name harmonicas go for less than $20.
The acoustic guitar is one I'd personally recommend, but I don't know your taste or situation. It has a moderate learning curve. It's compact in the sense that it can be carried in a car or in a case. You may be thinking "guitars aren't cheap," but, starting out, you can get a guitar for less than $100 at times. You can easily find one for less than $200 (compare this to "cheap" wind/brass instruments). The main thing is making sure the neck is in good condition and has a consistent tuning down the neck. Good strings can make even a cheap acoustic guitar sound fairly good. Again, there's nothing needed to amplify the sound. Finally, there are a HUGE amount of online resources helping you to learn various songs for the guitar. Also, I've yet to find a single song that doesn't sound good with the sound and note range a guitar offers.
Singing is musical, and is the most "compact" music-making method named. However, I believe that the learning curve for non-natural-born-singers tends to be quite steep (someone please comment if I'm wrong - this is personal experience with other musicians). Amplifying the sound isn't always necessary, but when you add in accompanying music, you'll need something to play the music (e.g. other musicians/instruments, or a sound system). That harms the s "portability" advantage for singing.
The recorder is certainly cheap and rivals the harmonica in portability. There are some really great songs that can be played on the recorder too! However, the recorder tends to have more limited resources to encourage learning than some of the other instruments. Also, there may be fewer songs/genres you see "fit" the sound of the recorder well. I personally see the recorder as a good short-term instrument to learn a couple songs on rather quickly.
Also, there is the keyboard. It is probably only compact when compared to an actual piano or a full-sized harp, but it is still portable in the sense you can carry it in car/truck fairly easily. A keyboard can be purchased cheaply, but there tends to be a strong relation between realistic sound and price. I haven't found a keyboard that sounds great without any help for less than a few hundred dollars. There is the option of MIDI connectivity, which can make a horrible sounding keyboard sound pretty darn good, but then you need more cables, software, hardware (often a laptop), etc. - in other words, probably not what most people consider cheap or portable. However, all that equipment can fit comfortably in most small rooms. Also, many schools, churches, and other places have pianos or keyboards for you to play and learn on. There are, like the guitar, a seemingly endless amount of resources to help one to learn piano. Also, pretty well all songs sound great on a good keyboard (sometimes modification of the song is needed). Also, I'd say learning the chords of a couple songs is really easy on the piano. Timing the two hands and intricate melodies are usually what trip up players, but learning chords on a piano is pretty easy. The melodic parts can be added later.
In the end, it's still up to you. What do you want to play, and how much effort will go into it? How much money are you willing to invest or not invest? How much portability is really needed? You'll get it.