Get back to Ready/Ready

Try1:

1. run sudo nodectl leave -p dag-l1 ; sudo nodectl stop -p dag-l1 ; sudo nodectl leave -p dag-l0 ;sudo nodectl stop -p dag-l0
2. Shut down your VM from the cloud side.  This way it really shuts down, clears out its memory file and the VM executable.  Basically a power off if it were a physical machine.
3. Power back on and login
4. run sudo nodectl execute_starchiver --datetime -p dag-l0
5. Wait 5 minutes
6. sudo nodectl upgrade --ni

Explanation - We are basically stopping L1/L0 cleanly and then shutting down the node.  Your nodes are processes on a Virtualization HyperVisor called KVM.  If you reboot then that processes never goes away. If you power off you get a clean process and the memory file is normally recreated. After that you Starchiver it, wait 5 minutes and upgrade nodectl. Let me know how it goes as I'm trying to find that as close to perfect procedure.  

1. Shutdown vs. Reboot Behavior

• Shutdown:

  • When you shut down a VM, it’s like powering off a physical machine. The guest OS inside the VM performs a clean shutdown of all its processes, network services, and file systems. It writes everything to disk, cleans up temporary files, and powers off gracefully.

  • The VM's resources are completely released on the host side, and the virtual CPUs, RAM, and disk are freed up until the VM is started again.

• Reboot:

  • When you reboot a VM, the guest OS essentially performs a restart but doesn't fully power off the virtual machine. The VM's state is still there in memory (RAM), and most resources remain allocated. After the VM reboots, it resumes from a fresh boot state.

  • It’s a bit faster than a full shutdown because it doesn't have to go through the entire cleanup and shutdown process. It just restarts the OS and resets the virtual hardware environment.