Problem with libfetch, pkg, and proxying?

[Alan Amesbury]

Given FreeBSD-9.1-RELEASE, 'pkg' installed from ports, and a pkg.conf that points to a proxy, it appears 'pkg' is ignoring the proxy setting for HTTPS URLs.

The contents of /usr/local/etc/pkg.conf consists of:


  pkg_env {
	http_proxy: http://proxyhost.fqdn:3128/
  }



'uname -srm' = "FreeBSD 9.1-RELEASE-p19 amd64".  It's not running GENERIC, but I don't think that's relevant.  :-)

Network traffic shows the host uses the proxy correctly for the initial HTTP callout to the local package repository, but tries to connect directly when it receives an HTTP redirect to HTTPS.  This is borne out in output from 'truss', which shows (with some data redacted):

			.
			.
			.
72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) = 0 (0x0)
72869: sendto(5,"\M-)W\^A\0\0\^A\0\0\0\0\0\0\apro"...,44,0x0,NULL,0x0) = 44 (0x2c)
72869: clock_gettime(0,{1413835372.386244672 })  = 0 (0x0)
72869: kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,EV_ONESHOT,0,0xcb,0x0},1,{5.000000000 }) = 1 (0x1)
72869: recvfrom(5,"\M-)W\M^A\M^@\0\^A\0\^A\0\^B\0"...,65536,0x0,{ AF_INET 128.101.101.101:53 },0x7fffffff77dc) = 203 (0xcb)
72869: close(5)                                  = 0 (0x0)
72869: close(4)                                  = 0 (0x0)
72869: kqueue(0x7e6bfa380,0x7e7496000,0x10000058,0x7e7486000,0x10000,0x1) = 4 (0x4)
72869: socket(PF_INET,SOCK_DGRAM,0)              = 5 (0x5)
72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) = 0 (0x0)
72869: sendto(5,"\M-)X\^A\0\0\^A\0\0\0\0\0\0\apro"...,44,0x0,NULL,0x0) = 44 (0x2c)
72869: clock_gettime(0,{1413835372.388397497 })  = 0 (0x0)
72869: kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,EV_ONESHOT,0,0x69,0x0},1,{5.000000000 }) = 1 (0x1)
72869: recvfrom(5,"\M-)X\M^A\M^@\0\^A\0\0\0\^A\0\0"...,65536,0x0,{ AF_INET 128.101.101.101:53 },0x7fffffff77dc) = 105 (0x69)
72869: close(5)                                  = 0 (0x0)
72869: close(4)                                  = 0 (0x0)
72869: madvise(0x7e7496000,0x10000,0x5,0x95,0x7fffffff7830,0x62c1b0) = 0 (0x0)
72869: madvise(0x7e7476000,0x10000,0x5,0x75,0x7fffffff7d10,0xffffffff) = 0 (0x0)
72869: madvise(0x7e7486000,0x10000,0x5,0x85,0x7fffffff7d10,0x62c1b0) = 0 (0x0)
72869: socket(PF_INET,SOCK_STREAM,6)             = 4 (0x4)
72869: connect(4,{ AF_INET [PROXY]:3128 },16) = 0 (0x0)
72869: fcntl(4,F_SETFL,O_NONBLOCK)               = 0 (0x0)
72869: fcntl(4,F_SETFD,FD_CLOEXEC)               = 0 (0x0)
72869: setsockopt(0x4,0xffff,0x800,0x7fffffff9144,0x4,0x0) = 0 (0x0)
72869: setsockopt(0x4,0x6,0x4,0x7fffffff9458,0x4,0x0) = 0 (0x0)
			.
			.
			.
72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) = 0 (0x0)
72869: sendto(5,"\M-)Y\^A\0\0\^A\0\0\0\0\0\0\thor"...,42,0x0,NULL,0x0) = 42 (0x2a)
72869: clock_gettime(0,{1413835372.458693385 })  = 0 (0x0)
72869: kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,EV_ONESHOT,0,0xc9,0x0},1,{5.000000000 }) = 1 (0x1)
72869: recvfrom(5,"\M-)Y\M^A\M^@\0\^A\0\^A\0\^B\0"...,65536,0x0,{ AF_INET 128.101.101.101:53 },0x7fffffff77dc) = 201 (0xc9)
72869: close(5)                                  = 0 (0x0)
72869: close(4)                                  = 0 (0x0)
72869: kqueue(0x7e6bfa380,0x7e7496000,0x10000058,0x7e7486000,0x10000,0x1) = 4 (0x4)
72869: socket(PF_INET,SOCK_DGRAM,0)              = 5 (0x5)
72869: connect(5,{ AF_INET [NAMESERVER]:53 },16) = 0 (0x0)
72869: sendto(5,"\M-)Z\^A\0\0\^A\0\0\0\0\0\0\thor"...,42,0x0,NULL,0x0) = 42 (0x2a)
72869: clock_gettime(0,{1413835372.461001593 })  = 0 (0x0)
72869: kevent(4,{0x5,EVFILT_READ,EV_ADD|EV_ONESHOT,0,0x0,0x0},1,{0x5,EVFILT_READ,EV_ONESHOT,0,0x67,0x0},1,{5.000000000 }) = 1 (0x1)
72869: recvfrom(5,"\M-)Z\M^A\M^@\0\^A\0\0\0\^A\0\0"...,65536,0x0,{ AF_INET 128.101.101.101:53 },0x7fffffff77dc) = 103 (0x67)
72869: close(5)                                  = 0 (0x0)
72869: close(4)                                  = 0 (0x0)
72869: madvise(0x7e7496000,0x10000,0x5,0x95,0x7fffffff7830,0x62c1b0) = 0 (0x0)
72869: madvise(0x7e7476000,0x10000,0x5,0x75,0x7fffffff7d10,0xffffffff) = 0 (0x0)
72869: madvise(0x7e7486000,0x10000,0x5,0x85,0x7fffffff7d10,0x62c1b0) = 0 (0x0)
72869: socket(PF_INET,SOCK_STREAM,6)             = 4 (0x4)
72869: connect(4,{ AF_INET [NOT_PROXY]:443 },16) ERR#60 'Operation timed out'
			.
			.
			.



The connection timed out because connections to hosts other than the proxy aren't allowed.  However, my reading of fetch(3) and fetch(1) suggests that the environment variable for http_proxy should cover HTTP and HTTPS URLs.  Tests using lynx were different; lynx apparently uses ${PROTOCOL}_PROXY where ${PROTOCOL} is the URL type, and HTTP and HTTPS are different.

Is this behavior correct?  I don't think it is.  Regardless, is there a way to get 'pkg' to use HTTPS URLs through a proxy?

Thanks in advance for any help/insights you can provide!


-- 
Alan Amesbury
University Information Security