I find that each app serves its purpose but rarely do I need to flop back-n-forth between DJIGO and Litchi.... I stay 95% of the time in Litchi and everything works just fine and dandy.... don't know where you got the expectation that you'd be able to swap back and forth while flying as simply as selecting one app or the other with no other action/step needed... Litchi/VC Tech never claims that you could... they have no control over the behavior of Android USB attachment to an app... as long as you always select the "use once" option, you technically could switch back and forth but may also have to unplug/plug the cable to force the Android OS to change the USB association...
Litchi is a great app that has seen significant enhancements and features added in only a couple of months (since the previous Firmware introduction).