The housing minister tweeted just after 1:30pm today saying:
"Been asked to step aside from Housing for a younger generation. Disappointing but it's been a great eleven years on frontbencher.(sic)"
Mr Prisk joked about the uncertainty around his job at the Conservative Party conference in Manchester last week and is one of several mid-ranking ministers whose position was under threat.
Mr Prisk took on the role of housing minister from Grant Shapps only 12 months ago.
As yet it is not clear yet who will replace him at the Department for Communities and Local Government (DCLG).
The move comes as building regulations minister Don Foster was moved from his post at DCLG to become Liberal Democrat chief whip.