If a garden is too close to the street, people won't use it because it isn't private enough. But if it is too far from the street, then it won't be used either, because it is too isolated.
HERE HERE --> Courtyards Which Live HERE --> Garden Growing Wild HERE --> Entrance Transition
Do not place the garden fully in front of the house, nor fully to the back. Instead, place it in some kind of half-way position, side-by-side with the house, in a position which is half-hidden from the street, and half-exposed.
Try Lineup Viewer