Sudden trouble with net/rdesktop

[Yuri Pankov]

Andrea Venturoli wrote:
> On 11/12/20 5:18 PM, Yuri Pankov wrote:
> 
>>> Could it be something that changed on remote side, e.g. any recent 
>>> updates?
> 
> Sure.
> Most targets are Windows 10 machines, so they possibly updated to a 
> newer version, but I don't know.
> One is Windows 2012, though, so I don't think that changed that much.

OK, I was able to reproduce this; actually, it hit that assertion for 
all Windows 10 20H2 installs I have, all are updated to latest patches, 
so I can't tell when this started to happen.  Adding a debug print 
before that assert() shows the following:

$ DISPLAY=mercury:0.0 
/usr/ports/net/rdesktop/work/rdesktop-1.9.0/rdesktop orion
len=128 size=128
Assertion failed: ((len * 2) < size), function _utils_data_to_hex, file 
utils.c, line 501.

So we need the len of 128 we need size of digest buf to be > 256, the 
following patch worked for me:

polaris:ypankov:/usr/ports/net/rdesktop$ cat files/patch-utils.c
--- utils.c.orig        2020-11-13 12:40:51 UTC
+++ utils.c
@@ -584,7 +584,7 @@ _utils_cert_get_info(gnutls_x509_crt_t cert, char *out
  {
         char buf[128];
         size_t buf_size;
-       char digest[128];
+       char digest[512];
         gnutls_x509_dn_t dn;
         time_t expire_ts, activated_ts;


Also attached so you could simply drop it to files/ directory (if the 
list will pass it through).  Though I must admit I have no idea what 
changed exactly on Windows side caused the digest size to grow that much.

>>> As a workaround, try increasing the buf size to e.g. 512 in 
>>> utils.c:_utils_cert_get_info().
>>
>> Sorry, looks like I'm reading that backwards, and increasing the buf 
>> size will actually make it worse.
>>
>> BTW, did you previously have rdesktop compiled without DEBUG option, 
>> so that assert() simply didn't fire, and now it's ON?
> 
> Actually I built it in poudriere *without* DEBUG option.
> 
> I might as well enable it and investigate, then.

Sorry, that was another shot in the dark, without actually trying 
something, and wrong one at it.
-------------- next part --------------
--- utils.c.orig	2020-11-13 12:40:51 UTC
+++ utils.c
@@ -584,7 +584,7 @@ _utils_cert_get_info(gnutls_x509_crt_t cert, char *out
 {
 	char buf[128];
 	size_t buf_size;
-	char digest[128];
+	char digest[512];
 	gnutls_x509_dn_t dn;
 	time_t expire_ts, activated_ts;