Sadly for you, and for us, Gas Safe Registration does not guarantee any practical ability.
It's sad for you because the job is lousy, the chap doing it was both hamfisted and couldn't give a damn (just the sealant round the pipe entry shows this).
It's sad for the installation trade because one or more of us has lost a morning's work quoting for this job, only for us to be rejected in favour of a dodgy tradesman.
Gas Safe are unlikely to be much help as the flue has screws in it and the compression joints are perfectly allowable if they are accessible. In other words, their inspector would be dismayed but wouldn't have enough to enforce upgrades of the general installation.
You will find Trading Standards useless too.
We have had people like this apply for jobs with us in the last decade when gas installers wanted £40Kpa to turn up - and believe me, they can't learn or be rehabilitated, because we've tried. If they were happy to leave this mess on Day 1, any further work will be awful. Contrary to the common advice, don't get him back.
My advice would be to put it down to experience and pay someone else to put it right. Preferably someone sober and not as high as a kite.