Campaigners have renewed calls for more affordable housing and rent controls after it emerged flats in a former public building were being let for up to £2,500 a month.