The following part from chapter 84 of the Harry Potter and the Methods of Rationality fanfic describes a tune that appears to be designed to be unnerving.
The humming started as a simple children's lullaby, the one that in Muggle Britain begins, Lullaby, and goodnight...
This tune was hummed, without variation, over and over, for seven minutes, to establish the underlying pattern.
Then began the elaborations upon the theme. Phrases hummed too slow, with long pauses in between, so that the listener's mind helplessly waits and waits for the next note, the next phrase. And then, when that next phrase comes, it is so out of key, so unbelievably awfully out of key, not just out of key for the previous phrases but sung at a pitch which does not correspond to any key, that you would have to believe this person had spent hours deliberately practicing their humming just to acquire such perfect anti-pitch. <...>
And this horrible, horrible humming is impossible to ignore. It is similar to a known lullaby, but it departs from that pattern unpredictably. It sets up expectations and then violates them, never in any constant pattern that would permit the humming to fade into the background. The listener's brain cannot prevent itself from expecting the anti-musical phrases to complete, nor prevent itself from noticing the surprises.
The only possible explanation for how this mode of humming came to exist is that it was deliberately designed by some unspeakably cruel genius who woke up one day, feeling bored with ordinary torture, who decided to handicap himself and find out whether he could break someone's sanity just by humming at them.
Since most of the non-magical ideas in HP&MOR are supposed to be scientifically plausible, I think this kind of tune might actually be composable. So I wonder, has anyone ever tried to compose such a thing? Did they succeed, or has it proved to be very hard or impossible?