Brickwork is inherently self-supporting, so if the breast is bonded into the wall it projects from, it is OK.
The issue tends to be with the innards of the chimney, as the brickwork that forms the flue tends not to be tied to anything - so the idea is that gallows brackets support a plate or suchlike to form a tray on the underside of the remaining breast to reduce the risk of things dropping down. But typically with such a narrow breast as this, bricks which are not bonded are still held in place by friction.
Your problem will be that whomever comes to look at this may not want to commit to say it is OK, and may want further work not for your benefit but more for the benefit of their insurance cover.
