That's All Very Well But... is a best of compilation of the band McCarthy. Initially planned for release in 1991 but shelved after the Midnight Music label went bankrupt. It finally surfaced in 1996 on Cherry Red who took over the rights to a number of artists from the defunct Midnight Music label (among them The Wolfhounds, The Snake Corps, The Essence and Sad Lovers & Giants). "That's All Very Well But..." is a somewhat haphazardly organized but still essential collection of McCarthy singles, rarities and Peel Sessions. The release presents many of the band's best moments at combining political passion with sparkling Indie pop music. The Peel Session tracks, taken from three different appearances in total, generally come across in slightly crisper fashion than the other studio cuts. It includes an alternate version of "Should The Bible Be Banned" which is a touch quieter with more prominent acoustic guitar.