dbtfeed tipped me to this, answering someone else's question on IRC.

If you need to run multiple simultaneous or successive ssh connections to the same host, OpenSSH 4 and later have a mechanism to do this with minimal overhead. ControlMaster lets you do the key exchange once, and routes successive connections through that session.