local pkgng repo

[Mark Saad]

All
 I am working on setting up a local repo for pkgng. I want to have a
local copy of the quarterly packages, so I can archive them as needed
and to run the repo inside of a private network.

I am using FreeBSD 9.3-RELEASE amd64 whit pkg v 1.4.10

Here is what I did

1. Mirror the contents of the public mirror to my mirror
 curlmirror.pl -s 100000000000 -o 2014Q4
http://pkg.freebsd.org/freebsd:9:x86:64/quarterly

2. Setup thttpd to serve this directory
thttpd -p 8080 -r -d /mnt/pkgng/ -l /tmp/thttpd-pkgng.log

3. Setup a client to use this new repo

[root at roseta ~]# cat /usr/local/etc/pkg/repos/About.conf
About: {
  url: "http://nyprdcpkgng1:8080/${ABI}/quarterly",
  mirror_type: "http",
  signature_type: "fingerprints",
  fingerprints: "/usr/share/keys/pkg",
  enabled: yes
}

4. Disabled the master repo on the client
[root at roseta ~]# cat /usr/local/etc/pkg/repos/FreeBSD.conf
FreeBSD: { enabled: no }

5. Tested that I can browse to the http server via a real client.
XXX>XXX>XXX>XXX - - [17/Feb/2015:21:05:10 +0000] "GET
/freebsd%3a9%3ax86%3a64/ HTTP/1.1" 200 25000 "http://nyprdcpkgng1/"
"Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0) Gecko/20100101 Firefox/31.0"
XXX>XXX>XXX>XXX- - [17/Feb/2015:21:05:14 +0000] "GET
/freebsd%3a9%3ax86%3a64/2014Q4/ HTTP/1.1" 200 1652
"http://nyprdcpkgng1/freebsd%3a9%3ax86%3a64/" "Mozilla/5.0 (X11; FreeBSD
amd64; rv:31.0) Gecko/20100101 Firefox/31.0


6. Then test the client

[root at roseta ~]# pkg update
Updating About repository catalogue...
pkg: http://nyprdcpkgng1:8080/FreeBSD:9:amd64/quarterly/meta.txz: Not Found
pkg: repository About has no meta file, using default settings
pkg: http://nyprdcpkgng1:8080/FreeBSD:9:amd64/quarterly/packagesite.txz:
Not Found
pkg: Unable to update repository About
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
All repositories are up-to-date.


7. Check the logs on the pkgng mirror to see what happened.
XXX>XXX>XXX>XXX - - [18/Feb/2015:17:13:06 +0000] "GET
/FreeBSD:9:amd64/quarterly/meta.txz HTTP/1.1" 404 0 "" "pkg/1.4.10"
XXX>XXX>XXX>XXX - - [18/Feb/2015:17:13:06 +0000] "GET
/FreeBSD:9:amd64/quarterly/packagesite.txz HTTP/1.1" 404 0 "" "pkg/1.4.10"
XXX>XXX>XXX>XXX - - [18/Feb/2015:17:21:15 +0000] "GET
/FreeBSD:9:amd64/quarterly/meta.txz HTTP/1.1" 404 0 "" "pkg/1.4.10"
XXX>XXX>XXX>XXX - - [18/Feb/2015:17:21:15 +0000] "GET
/FreeBSD:9:amd64/quarterly/packagesite.txz HTTP/1.1" 404 0


Where do I stand
---------------------


So in step 5 I noted that the request is for freebsd%3a9%3ax86%3a64 and
not freebsd:9:x86:64 ok not sure why its html encoded but it still works

secondly in step 7 the client appears to translate the ${ABI} in my
config to FreeBSD:9:amd64 and not freebsd:9:x86:64 which is strange.


What I tried .
-------------------
So in the sake of trying sill things that should not matter I ln'd
freebsd:9:x86:64 to  FreeBSD:9:amd64

Then I went to my client and tested it

[root at roseta ~]# pkg update
Updating About repository catalogue...
Fetching meta.txz: 100%    968 B   1.0kB/s    00:01
Fetching packagesite.txz: 100%    5 MiB   5.3MB/s    00:01
Processing entries: 100%
About repository update completed. 23741 packages processed
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
[root at roseta ~]#

That looks like it was ok let me try to install a package

[root at roseta ~]# pkg instal figlet
Updating About repository catalogue...
About repository is up-to-date.
Updating FreeBSD repository catalogue...
FreeBSD repository is up-to-date.
All repositories are up-to-date.
The following 1 packages will be affected (of 0 checked):

Installed packages to be REINSTALLED:
        figlet-2.2.5 [About] (options changed)

The process will require 550 B more space.
115 KiB to be downloaded.

Proceed with this action? [y/N]: y
pkg:
http://nyprdcpkgng1:8080/FreeBSD:9:amd64/quarterly/All/figlet-2.2.5.txz:
Not Found

so I tried to get the file manually from a browser and I checked the
logs and noted this


URL from the pkgng client
XXX>XXX>XXX>XXX - - [18/Feb/2015:17:42:28 +0000] "GET
/FreeBSD:9:amd64/quarterly/All/figlet_2.2.5.txz HT
TP/1.1" 200 118512 "" "Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0)
Gecko/20100101 Firefox/31.0"

URL from browsing to the package from /
XXX>XXX>XXX>XXX - - [18/Feb/2015:17:42:58 +0000] "GET
/FreeBSD:9:amd64/quarterly/All/figlet-2.2.5.txz HT
TP/1.1" 404 0 "" "Mozilla/5.0 (X11; FreeBSD amd64; rv:31.0)
Gecko/20100101 Firefox/31.0"


The issue here is in the name of the package "figlet_2.2.5.txz vs
figlet-2.2.5.txz" the char after the t is wrong but no idea why. Its
like this all over the place.


I am going to redownload the repo and see if I get the same odd - vs _
issue .

Secondly does anyone have any comments on this ? Is there any major foot
shooters I hit ?

-- 
Mark saad | nonesuch at longcount.org