Grounding 2 separate battery banks

This is my first post. I've been following this forum since I built my small off grid system about 16 months ago. Thanks for all the great discussion and info. You guys have saved me a ton of aggravation and money.

My existing system is installed in a small beach house in Belize and consists of 750w array, into a TS 45a mppt controller, into 2 Dekka 12v 8g8d 225ah gel batteries in parallel for a 450ah bank. (Bank #1) My power consumption is a modest 1450wh daily, split 55% day, 45% night. I have a couple of small 12v loads directly from battery bank to a 12v distribution panel (hence the 12v system) along with 120v circuits powered by a GoPower 3000w inverter to a 120v distribution panel with the neutral bonding screw removed. The battery negative and 120v neutral are bonded inside the inverter and my single point of system grounding (per inverter instructions) is from the inverter chassis ground terminal to a ground rod driven 10 ft into the sand under the roof drip line of the house. All components are isolated by appropriately sized breakers.

The system has operated flawlessly for over a year, until I recently replaced the propane fridge that finally gave up the ghost with a modest electric one. This added approx 1400w to my daily power needs split 50/50 between day and night. I decided to add 1000w more panels into a TS 60a mppt controller into 2 additional 225ah batteries parallel for a separate 450ah bank. (Bank #2) 

I did not want to add the new batter's to the old ones, or have 4 batteries in parallel, so I am keeping the banks separate. I will leave the 12v loads on Bank #1 and run the inverter from Bank #2. I would like to install a battery selector switch to be able to utilize Bank #1 to the inverter when needed. 

Can I buss the negatives of the two banks together (they are charged by separate controllers) and only switch the positive leads to the inverter, or do I need a switch for the negatives to the inverter as well? (positive switch off,  switch negative to other bank, switch positive to other bank). If I switch both it will leave the bank serving the 12v loads floating with no ground, which I think is ok (?)