One of the technical questions I had at Blizzard was pertaining to locking down access to a file(s) so that only a given user could download them. I creatively suggested an SSL'ed vhost and some form of auth to get to the file(s). When asked about scp, I stated that scp required a valid shell, which isn't always desired. This means that they can logon to the box.
This has been bugging me since yesterday, as it's a bit of a hack. What if you have a centralised auth system and can state which shell to use? There must be some way to only allow scp. Having done some research, it appears that there is! Enter "scponly". The only downside is that they'd need somewhere to logon to, in order to change passwords in the instance(s) of password expiry, should you wish to go down that route. Perhaps, in this case, it's better for a sysadmin to track and perform account changes though.
When I had to do this a while ago, I used rssh.
http://www.pizzashack.org/rssh/
It did a pretty good job of it.
Is it worth considering pam-mysql and nss-mysql for authentication, so you can provide a password changing facility via web control panel?
Then again, why would you want them to be able to change passwords? Surely you'd restrict their authentication methods to key only?
Ah, also how handy! I've added it to my list of stuff to play with.
I'd suggest that yes, you would limit it to a key, but aren't always suitable though :)