Swapping one USB keyboard (or mouse) for another USB keyboard (or mouse) is safe to do while the computer is on.
Since drivers for plug-and-play peripherals are included in Windows, as I originally wrote, these drivers were 'installed and saved' when Windows was installed.
These drivers were supplied to Microsoft by the peripheral manufacturers themselves. Microsoft has neither the proprietary knowledge nor the legal right to write drivers for peripherals that other companies build.
Over time, peripheral manufacturers may supply Microsoft with newer versions of the drivers they originally provided to Microsoft to include in Windows. The updated drivers are 'installed and saved' by Windows Update.
Thus, Windows may have more than one installed and saved driver for any given peripheral, and in that case Windows uses the most recent version of the driver. The earlier versions are still installed and saved - this is necessary for a situation where an updated driver causes problems. If that happens you can revert to an earlier version of the driver.
All this discussion of drivers only applies to the drivers that Microsoft receives from device manufacturers for inclusion in Windows. Device manufacturers commonly include additional software for their devices that you can install yourself. This is common for keyboards and mice, where the manufacturer's software may give you the ability to remap keys, use special function keys and remap mouse buttons, for examples.