libfetch ftp patch for less latency
John Baldwin
jhb at freebsd.org
Thu Mar 29 18:14:47 UTC 2007
On Thursday 29 March 2007 01:53:33 pm Rui Paulo wrote:
> Nate Lawson wrote:
> > To speed up pkg_add and other ftp command channel performance over slow
> > links, change this:
> > [...]
> > All ftp servers I've ever seen support a full path when changing down
> > dirs. This might be a DOS ftp server thing however. In any case, if
> > there is an error to the all-in-one CWD, the code reverts back to legacy
> > behavior of multiple CWDs.
>
> Yes, I think this behavior is acceptable. If the pathname fails it only
> adds one more CWD, but reduces the time on the successful cases (which
> are many if you consider the Ports collection).
>
> Dag-Erling Smørgrav wrote:
> > No. This is a violation of the FTP protocol.
> > [...]
> > When the all-in-one CWD fails, you're SOL. You have no idea what
> > state the server is in, and you have to start over.
>
> About the possible violation of the RFC, I quote:
>
> pathname
>
> Pathname is defined to be the character string which must be
> input to a file system by a user in order to identify a file.
> Pathname normally contains device and/or directory names, and
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> file name specification. FTP does not yet specify a standard
> ^^^^^^^^^^^^^^^^^^^^^^^
> pathname convention. Each user must follow the file naming
> conventions of the file systems involved in the transfer.
>
>
> CHANGE WORKING DIRECTORY (CWD)
>
> This command allows the user to work with a different
> directory or dataset for file storage or retrieval without
> altering his login or accounting information. Transfer
> parameters are similarly unchanged. The argument is a
> pathname specifying a directory or other system dependent
> ^^^^^^^^
> file group designator.
>
>
> I'm not sure why you consider this a violation of the protocol.
> Also, considering the reply codes to the CWD command, only 421 seems to
> change the server state.
>
> Have you seen servers that change their state by issuing giving invalid
> arguments to the CWD command ?
http://www.freebsd.org/cgi/query-pr.cgi?pr=83278
In this case IIRC, the problem was the leading / as the path fetched was
supposed to be relative to the user's home directory, but instead fetch sent
an absolute path and failed.
See also http://www.ietf.org/rfc/rfc1738.txt which is the RFC for URLs and
explains exactly how the <url-path> for ftp URLs should be interpreted:
<quote from="rfc1738">
3.2.2. FTP url-path
The url-path of a FTP URL has the following syntax:
<cwd1>/<cwd2>/.../<cwdN>/<name>;type=<typecode>
Where <cwd1> through <cwdN> and <name> are (possibly encoded) strings
and <typecode> is one of the characters "a", "i", or "d". The part
";type=<typecode>" may be omitted. The <cwdx> and <name> parts may be
empty. The whole url-path may be omitted, including the "/"
delimiting it from the prefix containing user, password, host, and
port.
The url-path is interpreted as a series of FTP commands as follows:
Each of the <cwd> elements is to be supplied, sequentially, as the
argument to a CWD (change working directory) command.
If the typecode is "d", perform a NLST (name list) command with
<name> as the argument, and interpret the results as a file
directory listing.
Otherwise, perform a TYPE command with <typecode> as the argument,
and then access the file whose name is <name> (for example, using
the RETR command.)
Within a name or CWD component, the characters "/" and ";" are
reserved and must be encoded. The components are decoded prior to
their use in the FTP protocol. In particular, if the appropriate FTP
sequence to access a particular file requires supplying a string
containing a "/" as an argument to a CWD or RETR command, it is
necessary to encode each "/".
For example, the URL <URL:ftp://myname@host.dom/%2Fetc/motd> is
interpreted by FTP-ing to "host.dom", logging in as "myname"
(prompting for a password if it is asked for), and then executing
"CWD /etc" and then "RETR motd". This has a different meaning from
<URL:ftp://myname@host.dom/etc/motd> which would "CWD etc" and then
"RETR motd"; the initial "CWD" might be executed relative to the
default directory for "myname". On the other hand,
<URL:ftp://myname@host.dom//etc/motd>, would "CWD " with a null
argument, then "CWD etc", and then "RETR motd".
</quote>
--
John Baldwin
More information about the freebsd-current
mailing list