I'd suggest you step aside from blame-throwing, as you'll probably never resolve anything if you're dealing with cowboys as it appears.
How are you on the payments side? Have you paid for more or less work than has been done?
My approach would be to write to the builder to say the contract is terminated due to breach of contract by him, he has no further involvement and you won't be making any further payments. If he demands payment then invite him to sue, then you'd need to show up at the small claims court and prove that what he built wasn't fit for purpose, which shouldn't be difficult once you get it properly assessed. But in all likelihood you'll never hear from him again.
Find a decent SE and/or builder. Give them the freedom to decide what needs demolishing and what can stay and propose a solution.
Just concentrate on how to get the building itself resolved, leave the builder and his dodgy SE mate to find another mug to scam.