This seems to be a problem with the way LilyPond handles the so-called grobs. The system bracket obviously has a height of zero, so that it does not affect the clipped area, but instead gets clipped itself.
Anyways, as soon as you type notes into the lower staff that reach below the lowest staff line or if you append things below staff (such as dynamics or lyrics), the problem will disappear, because then, there is something below the lowest staff line that LilyPond has to render and the clipping is correct.
So, the one way to solve this is to tell LilyPond that there is something it has to render below the lowest staff line. This can be done for example by telling LilyPond to print transparent bar numbers below the staff groups like this:
<lilypond>
\layout {
#(layout-set-staff-size 25)
\context {
\Score
barNumberVisibility = #all-bar-numbers-visible % print bar numbers in every line
\override BarNumber.direction = #DOWN % print bar numbers below staff
\override BarNumber.transparent = ##t % make bar numbers transparent
}
}
\new StaffGroup <<
\bar "" % force printing of first bar number
\new Staff = "up" {
\voiceOne
c'1 d' e' f' g' a' b' c''
}
\new Staff = "down" {
\clef tenor
\voiceTwo
c'1 c' c' c' c' c' c' c'
}
>>
</lilypond>
If you render this with the same command you used, namely lilypond-book --output=out -f html book.lytex
(assumed your original file is called “book.lytex”), you will get the following result:
Edit
After another round of googling, I found a better (i.e. cleaner) solution that re-calculates the height of the system bracket. It seems that this solution also works in your set up. Placing the original code into your snippet would look like this:
<lilypond>
\layout {
#(layout-set-staff-size 25)
\context {
\Score
\override SystemStartBracket.after-line-breaking =
#(lambda (grob)
(if (defined? 'default-toplevel-book-handler)
(let ((Y-off (ly:grob-property grob 'Y-extent)))
(ly:grob-set-property! grob 'Y-extent
(cons (- (car Y-off) 1.7) (+ (cdr Y-off) 1.7))))))
}
}
\new StaffGroup <<
\new Staff = "up" {
\voiceOne
c'1 d' e' f' g' a' b' c''
}
\new Staff = "down" {
\clef tenor
\voiceTwo
c'1 c' c' c' c' c' c' c'
}
>>
</lilypond>