Found the instructions for this here.

apt-get purge desktop-base
update-initramfs -u
update-grub