My mum's got a small terraced newbuild house with one built on by the previous owners.
I find the living room dark. Definitely much darker than it should be, given that it has patio doors. It does provide some free heat on sunny winter days, but it also provides lots of heat that you don't want on hot summer days, into the living room and beyond into the rest of the house.
She sometimes uses it as a dining room, when there are too many visitors for the dining kitchen. I find it weird siting in there, overlooking neighbours both ways. The garden slopes down from the house, so it's quite elevated. Check planning requirements - my mum's has been there long enough to have de facto planning permission. If I'd been either of her neighbours I'd have phoned the council when it was built. It must be very odd for either neighbour to sit in their garden, overlooked by the side windows. Even if nobody's in there.
It's useful space. But I'd say an extension would be much better, perhaps with lots of windows on the side(s) that face into your garden only, together with a solid roof. You could add a lantern to get some light from above without as much heat. There are many graduations between conservatory and room. Perhaps see if you could add light tunnels or more windows into the adjoining room to compensate for the darkness you'll inevitably get.