Gregg Wallace is facing a mounting backlash after claiming allegations of sexually inappropriate behaviour made against him came from a “handful of middle-class women of a certain age”.