misc/111715: request ftpd conform to RFC 2640

Mon Apr 16 18:50:02 UTC 2007

>Number:         111715
>Category:       misc
>Synopsis:       request ftpd conform to RFC 2640
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Apr 16 18:50:02 GMT 2007
>Closed-Date:
>Last-Modified:
>Originator:     Zhang Weiwu
>Release:        FreeBSD 6.1 Stable RELEASE
>Organization:
Real Softservice
>Environment:
FreeBSD exupery.realss.com 6.1-RELEASE FreeBSD 6.1-RELEASE #1: Sun Dec  3 00:54:09 CST 2006     root at hatch.realss.com:/usr/obj/usr/src/sys/HATCH  i386

>Description:
Compare to those who only use English for a life time, probably we Asian
users feel much more painful for character-set issue. To Asian users the
UTF8 compliance as defined in RFC2640 is very important. As FreeBSD is
an international project, UTF8 and related RFC compliance is AFAIK basic
requirement. The attached path added an option -8 to ftpd, when used, it
indicate the file system on server is using UTF-8 and the FTP client
(e.g. smart ftp, filezilla, lftp, gftp ...) should convert the filenames
and path names to local character set (which they do fine). FTP server
tell client to do character conversion if necessary by including "UTF8"
in FEAT.

RFC2640 is the only way to make international ftp server: it is the only
standard way to deliver non-latin filenames to different OSs. a.k.a.
Suppose you set up an FTP server and you want users use the Chinese file
names for both Simplified Chinese version of Windows user and
Traditional Chinese version of Windows.

P.S. RFC 2640 is based on RFC2389 which also gets implemented here. Although I tried my best compare the command-set of FreeBSD ftpd and RFC959 to implementation  RFC2389, the implementation of RFC2389 may still be incorrect.
>How-To-Repeat:

>Fix:
See my patch. I don't have FreeBSD 6.2 to try (and upgrade is not easy on my fully loaded server) and hope this patch can be applied to 6.2's ftpd directly.

Patch attached with submission follows:

diff -u ftpd.FreeBSD-6.1/ftpcmd.y ftpd/ftpcmd.y

--- ftpd.FreeBSD-6.1/ftpcmd.y	Thu Nov 18 21:46:29 2004
+++ ftpd/ftpcmd.y	Mon Apr 16 19:27:31 2007
@@ -95,6 +95,7 @@
 extern	int usedefault;
 extern  char tmpline[];
 extern	int readonly;
+extern	int utf8_mode;
 extern	int noepsv;
 extern	int noretr;
 extern	int noguestretr;
@@ -135,7 +136,7 @@
 	ABOR	DELE	CWD	LIST	NLST	SITE
 	STAT	HELP	NOOP	MKD	RMD	PWD
 	CDUP	STOU	SMNT	SYST	SIZE	MDTM
-	LPRT	LPSV	EPRT	EPSV
+	LPRT	LPSV	EPRT	EPSV	FEAT
 
 	UMASK	IDLE	CHMOD	MDFIVE
 
@@ -687,6 +688,16 @@
 			if ($4 != NULL)
 				free($4);
 		}
+	| FEAT CRLF
+		{
+			lreply(211, "Features:");
+			printf(" EPSV\r\n");
+			printf(" EPRT\r\n");
+			printf(" MDTM\r\n");
+			printf(" SIZE\r\n");
+			if (utf8_mode) printf(" UTF8\r\n");
+			reply(211, "End");
+		}
 	| SYST check_login CRLF
 		{
 			if ($2) {
@@ -1112,6 +1123,7 @@
 	{ "NLST", NLST, OSTR, 1,	"[ <sp> path-name ]" },
 	{ "SITE", SITE, SITECMD, 1,	"site-cmd [ <sp> arguments ]" },
 	{ "SYST", SYST, ARGS, 1,	"(get type of operating system)" },
+	{ "FEAT", FEAT, ARGS, 1,	"(get features supported)" },
 	{ "STAT", STAT, OSTR, 1,	"[ <sp> path-name ]" },
 	{ "HELP", HELP, OSTR, 1,	"[ <sp> <string> ]" },
 	{ "NOOP", NOOP, ARGS, 1,	"" },
diff -u ftpd.FreeBSD-6.1/ftpd.8 ftpd/ftpd.8
--- ftpd.FreeBSD-6.1/ftpd.8	Sun Jan 29 21:21:05 2006
+++ ftpd/ftpd.8	Mon Apr 16 20:03:35 2007
@@ -40,7 +40,7 @@
 .Nd Internet File Transfer Protocol server
 .Sh SYNOPSIS
 .Nm
-.Op Fl 46ADdEhMmOoRrSUvW
+.Op Fl 468ADdEhMmOoRrSUvW
 .Op Fl l Op Fl l
 .Op Fl a Ar address
 .Op Fl P Ar port
@@ -78,6 +78,11 @@
 is specified, accept connections via
 .Dv AF_INET6
 socket.
+.It Fl 8
+Server is running in UTF-8 mode, an RFC2640 compliant  client should convert
+all paths and messages from server from UTF-8 to client's local character-set.
+Note enalbing this option does not make server do any filesystem encoding
+conversion, it only tell client to do so.
 .It Fl A
 Allow only anonymous ftp access.
 .It Fl a
@@ -250,6 +255,7 @@
 .It DELE Ta "delete a file [RW]"
 .It EPRT Ta "specify data connection port, multiprotocol"
 .It EPSV Ta "prepare for server-to-server transfer, multiprotocol"
+.It FEAT Ta "give server feature information"
 .It HELP Ta "give help information"
 .It LIST Ta "give list files in a directory" Pq Dq Li "ls -lgA"
 .It LPRT Ta "specify data connection port, multiprotocol"
diff -u ftpd.FreeBSD-6.1/ftpd.c ftpd/ftpd.c
--- ftpd.FreeBSD-6.1/ftpd.c	Thu Mar  9 17:12:44 2006
+++ ftpd/ftpd.c	Tue Apr 17 01:26:06 2007
@@ -128,6 +128,7 @@
 int	restricted_data_ports = 1;
 int	paranoid = 1;	  /* be extra careful about security */
 int	anon_only = 0;    /* Only anonymous ftp allowed */
+int	utf8_mode = 0;    /* server file system is in UTF-8 */
 int	guest;
 int	dochroot;
 char	*chrootdir;
@@ -308,7 +309,7 @@
 	openlog("ftpd", LOG_PID | LOG_NDELAY, LOG_FTP);
 
 	while ((ch = getopt(argc, argv,
-	                    "46a:AdDEhlmMoOp:P:rRSt:T:u:UvW")) != -1) {
+	                    "468a:AdDEhlmMoOp:P:rRSt:T:u:UvW")) != -1) {
 		switch (ch) {
 		case '4':
 			family = (family == AF_INET6) ? AF_UNSPEC : AF_INET;
@@ -316,6 +317,10 @@
 
 		case '6':
 			family = (family == AF_INET) ? AF_UNSPEC : AF_INET6;
+			break;
+
+		case '8':
+			utf8_mode = 1;
 			break;
 
 		case 'a':

>Release-Note:
>Audit-Trail:
>Unformatted:
