Air brick is not damaged by nest which is similar to paper . You can buy a can of wasp nest killer .It’s a spray foam which you could spray on nest from both sides which will kill everything in it in a few minute .
Then you should be able to vacuum out the nest .