The vandalism in Ballarat comes as security has been increased near statues of significant historical figures.
Police are searching for the vandals who removed the heads from two statues of Australian prime ministers and damaged 18 others in Ballarat early on Thursday morning.