The Penderwicks by Jeanne Birdsall (1994) Yearling NBA

Four sisters and their father go on vacation, but the house they usually vacation in has been sold and at the last second, a house in a different place than they've gone before is available.  There, they find a mansion, but it's not what they're renting, it's the cottage in the far back of the property with a full Victorian garden in-between.  The girls, of course, though they're told not to go into the garden, are inextricably drawn when they discover a hidden tunnel in the wall.  There are other discoveries as well, the oldest girl finds herself attracted to the gardener.  The next oldest, the tomboy, finds a boy her age living in the mansion, who is under the thumb of his mother and her terrible boyfriend, who want to send the boy away to a military school, because his grandfather was a general.  The gardener has rabbits which gets the youngest, shyest girl to come every day just to feed the rabbits. Then there is the other girl, who writes stories based on what she sees and imagines around her.