Disclaimer: I hope this is relevant for this sub, as a student I wanted to see a discussion of how people work.
I have access to an always-up university cluster and do everything on a remote server. I use Emacs’s Tramp package to edit files on the server and have an always running SSH connection to the server for bash, IPython, and even jupyter notebooks (accessed on my browser via SSH tunneling).
This workflow fails whenever I have no stable internet connection, and also does not translate well to billed-by-the-minute services like AWS. What is your workflow for doing ML research and experiments? Do you first test locally and then push everything to the server (if so, what do you use for this? Ex: git, rsync, etc.) and run experiments there or do you live on the server as I do?