The circle of fifths is showing us what keys are most closely related. Keys that are adjacent have all but one note in common. This means that modulating from one key to an adjacent one will have a smooth transition more easily than less closely related keys, if "properly" executed. This can be boring, as Laurence suggested, but can also provide just what you're looking for, a slight change, not too dramatic. A "proper" modulation will typically be set up with a standard sort of cadence and once you're in the new key, you would typically try to thoroughly establish that key, so as to make sure it's not a brief tonicization, often times by playing an additional cadence and/or emphasizing the notes and chords in the new key that were not in the old one, sometimes emphasizing the new notes in the melody. By "proper" I basically just mean that it will be done as seamlessly and convincingly as possible. This approach generally comes from the Classical tradition and is sometimes utilized in more modern styles but more often it is not. A lot of Pop modulations are very sudden, not set up by a cadence and feel very dramatic. So in a literal sense, proper would not be an appropriate word to use, as it implies other approaches are wrong.
As to your question about modulations as they apply to the different modes of the closely related keys, there are a few things to consider. Often times when playing in a mode, we are utilizing a different sort of harmonic structure, usually not using any sort of V-I relationships, as those tend to define functional harmony and take away from the modal feeling. With this in mind, it may be good to pay attention to how you are tonicizing your different keys/modes. If you go from a functional approach to a modal one, then you'll find that your sense of tonic within the mode may not be as well or as easily established. The V-I relationship within a functional setting gives a very strong sense of tonic, especially compared to a modal cadence, so transitioning from functional to modal will likely feel much less convincing than modal to functional.
You'll also want to consider how modal and functional harmony are rather different in their actual execution. Functional harmony feels like it has a much stronger sense of what is supposed to happen where and when (not that you can't stray from it), so when you switch between the two, everything in the modal world may feel a lot less satisfying, or feel like it needs something extra to make it live up to the other section. This could become a great tool within your compositional tool belt, as it could allow you to have very strong emotional content, such as modal feeling more bland and therefore making the functional more emotional than would have been within an exclusively functional environment. You just need to be very aware of the differences between the overall feel of these two approaches and be sure to use them with clear intention, whether or not it has to do with emotion or some other intention.
In your example, you mention that you are starting in E Phrygian and suggest that E Aeolian could be a good destination. This would likely be easy to accomplish, however, it won't really feel like much has changed, as both modes are minor and are both based around E. This isn't to say that you couldn't consider it a modulation, just that it will feel much less like a modulation than a slight change in texture.
In one of your comments, you mentioned that you are more interested in algorithms of music. I'd be careful with this to some extent, depending on what you intend to do with the music in the end, as it could lead you to a very boring place, which doesn't seem to be a concern to you at the moment but may become more so later on. The problem with looking at music in terms of algorithms is that you miss so much of what makes it so interesting. Sure, you can come up with a melody and a harmony and a whole arrangement through a strictly theory based or algorithm based approach but it's probably going to sound like a nursery rhyme, ie, totally boring and predictable. Using modulations within this will take it away from that a bit but ultimately it will likely just feel like you have a nursery rhyme in two different keys kind of stuck together. If you do wish to make truly artistic music, there needs to be more than algorithms. After all, some of the best music has been in direct dissent to the theory of the time or had no real theory to describe it. I'm very driven by theory myself but it always comes back around to what sounds good. I may come up with some intricate idea that I think is really cool because of the theory, then come back and listen to it the next day only to realize that it doesn't sound all that good and I don't like it anymore. The theory driving it may be sound but the sound isn't any good and it's really all about what sounds good in the end.