Here's my take on it after I looked into this a bit for myself. I'm no expert, just another guy that wants to take his Phantom everywhere.
When researching, I found that something like the Goal Zero Yeti 150 will only carry enough juice to fully recharge two Phantom batteries assuming you don't use it for anything else. Going up from there the Yeti 400 will do six to seven charges and the Yeti 1250 (100lb. monster battery) will do twenty Phantom recharges.
You can recharge the any of the above with solar panels, but even the small Yeti 150 battery takes 11-22 hours to charge with Goal Zero's largest panels (Boulder 30). All of them can be charged in short order via wall socket at home prior to departure.
By my estimation it's only borderline practical and would depend heavily on what you want accomplish and how much you can carry. If you're going to be camped in a single location for a week or more, I think that having a GZ Yeti battery and a panel or two would keep you in the air. With good sun exposure you could effectively charge one battery per day but at that rate, you'd want to have two or three batteries with you just so you'll have at least one at the ready at all times.
For short trips, or for activities like rafting or wilderness backpacking where you won't have reliable dawn-to-dusk sun exposure on your panels, you'd be better served packing extra Phantom batteries. Or at least that's the conclusion I came to.