virtualbox drivers are/were a constant source of kernel panics on MacOS and Linux too, so that should not be underestimated.
IIRC they also disabled ASLR kernel wide.
Additionally, and perhaps less important: USB3 is a commercial feature of virtualbox, there are stories of companies getting C&D letters (or Audits/Invoices) from Oracle because a developer had installed the virtualbox extensions..
If you use virtualbox with this new backend, you can use it concurrently with qemu (and a few other virtualisation tools).
KVM is also part of Linux itself, so there’s a lot less haste with setting it up.