First thing to check is rDNS - make sure the box can resolve the IP addresses used to connect to it. That's often why ssh/telnet/ftp/etc hang up - waiting for rDNS results to write log records.