Your ballast dilemma is not new. If you want to see how other engineers have solved the issue of moveable ballast take a look at the Lake seaplanes and the the homebuilt helicopters. A friend who owned Lake 250 I think had movable ballast for flying solo or with passengers. I recall that a couple of friends who built Rotorway helicopters had some sort of moveable ballast.

Of course, ballast that has no other function steals performance so if you can find a way to say make the main battery mount on a sliding tray so that it can be moveable ballast, rather than just using chunks of lead, you can minimize the penalty.

By the way, if you make the ballast removable, you will have problems hopping a friend over to another airport and then dropping them off to return solo. That's why ballast almost always is permanently attached somehow.

Best of luck,

Wes
N78PS