Well, in that case, the partitions that are not used will get unloaded as soon as the memory is required for other columns/partitions to work on.
Unfortunately for you, your dominant client will tend to use most of the space. So the effect of memory saving can safely be expected to be smallish.
- Lars