The short version appears to be first to do the kinds of things that Kim Eldridge has been suggesting (centralized follow up systems) as well as more full-time faculty and more remediation programs. Class size doesn’t seem to matter as much (at least from a student retention perspective, though I can tell you as someone who taught 240 students my first semester at JBU, class size does indeed matter from a faculty perspective).
http://www.insidehighered.com/news/2010/12/10/productivity
“In addition to call centers and the greater use of full-time faculty, remediation holds up well as a cost-effective way to improve college completion, Harris said. Yet, strategies like cutting class sizes can drive rankings, increase prestige and garner attention.”