Just in case you are using xdm with pam_ssh, recent xdm does not set SSH_AUTH_SOCK, because pam_getenvlist() will not called. I posted this problem at freedesktop Bugzilla. You can see details and my patch at: https://bugs.freedesktop.org/show_bug.cgi?id=11020