You could play music for a wide variety of instruments on your horn. It would still work, for example, to play music published for Eb alto saxophone, violin, Bb trumpet, anything really. If it feeds your soul to play it, go for it.
If you want to play along with someone else, the key of your instrument matters. Say you were playing an aria from a Bb clarinet book on your horn and you wanted a friend to play the piano part that accompanies the solo. That wouldn't work, because the music is printed to work for the clarinet, which transposes a major second, not a perfect fifth. The piano book would be expecting a C to come out, the clarinet part would have a printed D... Your horn when you play that D would make a G come out. It would sound terrible and make no sense. You mention the English horn. That would work because the transposition is the same. In this same aria, the piano book would expect a C. The e.h. book would print a G, and when you play that G on your horn a C would come out.
TL;DR
If you are playing alone, play anything that is reasonably possible for you ar your current skill level. If you are playing with others, play music published for instruments in F. You'll have the most success with French horn music, but e.h. music will probably work. The only time it wouldn't is if the e.h. is written higher than you can play on your horn.