From nobody Thu Jan 05 14:41:13 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Nnq0T5NvWz2qxyx; Thu, 5 Jan 2023 14:41:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Nnq0T59Bmz3v3F; Thu, 5 Jan 2023 14:41:13 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1672929673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t3RvYgpFOsrm9L/XvL2povZy6D37Prk7OEwZ41OEWI8=; b=V6kTHAensV/Dg28FJIC166htB+UoLAsMGmqeDeq1X4hWUh4VQbX999oogBpUXFO0lK+a8i /q4qS1xfmyDjLwLcLozsi4hVhY1bMu6OlxlX3PlPd9Rh8g7rfswu1xCC2XNU3z4F/4fLxT Rwy+xoU2OofN8MYqcVAZDFZWuC44Z4D1vL8UdOyPoXTgmGcFCu89/XKJkfJKNs2Jlx+Q/Q 36aoKxmiSPPFgpXUagS0Z33wsjCHSZ6HcEQNXQClkY5UvfWKdIjrUUxP+ZARgTfrejWnYw ZFXZ44Ur1VUfTJZYpIQ1ITHymhoydaxaEB91tUB5Dh23sKax5UgmC43r5XANjw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1672929673; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=t3RvYgpFOsrm9L/XvL2povZy6D37Prk7OEwZ41OEWI8=; b=tVys9zGIyYoSnEQ2MD8TpK4xOOm/SdNVAqPT4Wa0Vr3FQIw8btq2KB41TBDizv9INMY78i a3qlv970U2ZvuxfW/Chl9+wwPDqBjRKZ/QflGK+f0w0kfgFoiEeaNxz2fYtFhPUzngKgNq kiRpPOdZm/WY836iKNUGAsplRb2z6nXGFqCnkGVkoH1Ejr7UYeleuuxZdXA/PcmpPvXE7q hmZhoeVfZeuhh47pI0DJ0BYIkokZ48Jm8DWM/oDosOmoddGI7UZN9KmKaCRTaIlK77xgzk 64G+jAUF1mTDddsCxVT8+CrHm/qAF6k2WVIFiw/10Znug27svq/b9XxCdFqYdg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1672929673; a=rsa-sha256; cv=none; b=ldmEfVKOzsU62r9xvLyXcDEkEMWHzf2DhXjmZT0G38rgWDGag5tC4ucSThMABmJ0eHTeYI LO9eIPUAnJiOMqxWqcVcXl6AVWyew3GvSYqEMi+m1cOBfSLqq6eLau+0rlfOXqmpgNuPbp pLvPdO2/I4reQ10avas5Pnz/Qb1LqAr35VqUD0W14FUtaRvXu6Gvas2Pox65qOd7BLwuqB fFbxaiGYPZCE/C0RW1rUZC3G0ekPPebu1VE68ippDzVnBs+zdsf4RJgXbEpy66yGPDtyyd lC+e3/cSklILdih+bmTwIW3V1cF3evVL5N+32t4qFu97yE1bWO9aThr9oL8WMA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Nnq0T4FCQzvB3; Thu, 5 Jan 2023 14:41:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 305EfDhw055197; Thu, 5 Jan 2023 14:41:13 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 305EfDSX055196; Thu, 5 Jan 2023 14:41:13 GMT (envelope-from git) Date: Thu, 5 Jan 2023 14:41:13 GMT Message-Id: <202301051441.305EfDSX055196@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Cy Schubert Subject: git: 13202ee5cc4b - stable/12 - sqlite3: Vendor import of sqlite3 3.40.0 List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: cy X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 13202ee5cc4b1b36d7a99f94c21e6aa168494dc5 Auto-Submitted: auto-generated X-ThisMailContainsUnwantedMimeParts: N The branch stable/12 has been updated by cy: URL: https://cgit.FreeBSD.org/src/commit/?id=13202ee5cc4b1b36d7a99f94c21e6aa168494dc5 commit 13202ee5cc4b1b36d7a99f94c21e6aa168494dc5 Author: Cy Schubert AuthorDate: 2022-11-28 00:03:49 +0000 Commit: Cy Schubert CommitDate: 2023-01-04 15:29:13 +0000 sqlite3: Vendor import of sqlite3 3.40.0 Changes at https://www.sqlite.org/releaselog/3_40_0.html Obtained from: https://www.sqlite.org/2022/sqlite-autoconf-3400000.tar.gz (cherry picked from commit ac50343d44f8dff1efe667b4713de4b1351a19e1) --- contrib/sqlite3/configure | 20 +- contrib/sqlite3/configure.ac | 2 +- contrib/sqlite3/shell.c | 5165 ++++++++++++++++++++++++++-------- contrib/sqlite3/sqlite3.c | 4901 ++++++++++++++++++++++---------- contrib/sqlite3/sqlite3.h | 117 +- contrib/sqlite3/sqlite3ext.h | 8 +- contrib/sqlite3/sqlite3rc.h | 2 +- contrib/sqlite3/tea/Makefile.in | 189 +- contrib/sqlite3/tea/configure | 2069 +++++--------- contrib/sqlite3/tea/configure.ac | 94 +- contrib/sqlite3/tea/pkgIndex.tcl.in | 15 +- contrib/sqlite3/tea/tclconfig/tcl.m4 | 1153 ++++---- contrib/sqlite3/tea/win/makefile.vc | 13 +- 13 files changed, 8887 insertions(+), 4861 deletions(-) diff --git a/contrib/sqlite3/configure b/contrib/sqlite3/configure index 16251265e4f4..1f9c2c17e787 100755 --- a/contrib/sqlite3/configure +++ b/contrib/sqlite3/configure @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.71 for sqlite 3.39.3. +# Generated by GNU Autoconf 2.71 for sqlite 3.40.0. # # Report bugs to . # @@ -621,8 +621,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='sqlite' PACKAGE_TARNAME='sqlite' -PACKAGE_VERSION='3.39.3' -PACKAGE_STRING='sqlite 3.39.3' +PACKAGE_VERSION='3.40.0' +PACKAGE_STRING='sqlite 3.40.0' PACKAGE_BUGREPORT='http://www.sqlite.org' PACKAGE_URL='' @@ -1367,7 +1367,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures sqlite 3.39.3 to adapt to many kinds of systems. +\`configure' configures sqlite 3.40.0 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1438,7 +1438,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of sqlite 3.39.3:";; + short | recursive ) echo "Configuration of sqlite 3.40.0:";; esac cat <<\_ACEOF @@ -1563,7 +1563,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -sqlite configure 3.39.3 +sqlite configure 3.40.0 generated by GNU Autoconf 2.71 Copyright (C) 2021 Free Software Foundation, Inc. @@ -1833,7 +1833,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by sqlite $as_me 3.39.3, which was +It was created by sqlite $as_me 3.40.0, which was generated by GNU Autoconf 2.71. Invocation command line was $ $0$ac_configure_args_raw @@ -3106,7 +3106,7 @@ fi # Define the identity of the package. PACKAGE='sqlite' - VERSION='3.39.3' + VERSION='3.40.0' printf "%s\n" "#define PACKAGE \"$PACKAGE\"" >>confdefs.h @@ -15314,7 +15314,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by sqlite $as_me 3.39.3, which was +This file was extended by sqlite $as_me 3.40.0, which was generated by GNU Autoconf 2.71. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -15373,7 +15373,7 @@ ac_cs_config_escaped=`printf "%s\n" "$ac_cs_config" | sed "s/^ //; s/'/'\\\\\\\\ cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config='$ac_cs_config_escaped' ac_cs_version="\\ -sqlite config.status 3.39.3 +sqlite config.status 3.40.0 configured by $0, generated by GNU Autoconf 2.71, with options \\"\$ac_cs_config\\" diff --git a/contrib/sqlite3/configure.ac b/contrib/sqlite3/configure.ac index e3883098896c..aadce30bf20c 100644 --- a/contrib/sqlite3/configure.ac +++ b/contrib/sqlite3/configure.ac @@ -10,7 +10,7 @@ # AC_PREREQ(2.61) -AC_INIT(sqlite, 3.39.3, http://www.sqlite.org) +AC_INIT(sqlite, 3.40.0, http://www.sqlite.org) AC_CONFIG_SRCDIR([sqlite3.c]) AC_CONFIG_AUX_DIR([.]) diff --git a/contrib/sqlite3/shell.c b/contrib/sqlite3/shell.c index e66ae0874f52..6280ebf6108f 100644 --- a/contrib/sqlite3/shell.c +++ b/contrib/sqlite3/shell.c @@ -34,6 +34,8 @@ /* This needs to come before any includes for MSVC compiler */ #define _CRT_SECURE_NO_WARNINGS #endif +typedef unsigned int u32; +typedef unsigned short int u16; /* ** Optionally #include a user-defined header, whereby compilation options @@ -55,6 +57,15 @@ # define SQLITE_OS_WINRT 0 #endif +/* +** If SQLITE_SHELL_FIDDLE is defined then the shell is modified +** somewhat for use as a WASM module in a web browser. This flag +** should only be used when building the "fiddle" web application, as +** the browser-mode build has much different user input requirements +** and this build mode rewires the user input subsystem to account for +** that. +*/ + /* ** Warning pragmas copied from msvc.h in the core. */ @@ -94,6 +105,14 @@ # define _LARGEFILE_SOURCE 1 #endif +#if defined(SQLITE_SHELL_FIDDLE) && !defined(_POSIX_SOURCE) +/* +** emcc requires _POSIX_SOURCE (or one of several similar defines) +** to expose strdup(). +*/ +# define _POSIX_SOURCE +#endif + #include #include #include @@ -247,20 +266,21 @@ static void setTextMode(FILE *file, int isOutput){ # define setTextMode(X,Y) #endif -/* -** When compiling with emcc (a.k.a. emscripten), we're building a -** WebAssembly (WASM) bundle and need to disable and rewire a few -** things. -*/ -#ifdef __EMSCRIPTEN__ -#define SQLITE_SHELL_WASM_MODE -#else -#undef SQLITE_SHELL_WASM_MODE -#endif - /* True if the timer is enabled */ static int enableTimer = 0; +/* A version of strcmp() that works with NULL values */ +static int cli_strcmp(const char *a, const char *b){ + if( a==0 ) a = ""; + if( b==0 ) b = ""; + return strcmp(a,b); +} +static int cli_strncmp(const char *a, const char *b, size_t n){ + if( a==0 ) a = ""; + if( b==0 ) b = ""; + return strncmp(a,b,n); +} + /* Return the current wall-clock time */ static sqlite3_int64 timeOfDay(void){ static sqlite3_vfs *clockVfs = 0; @@ -549,6 +569,7 @@ static void utf8_width_print(FILE *pOut, int w, const char *zUtf){ int i; int n; int aw = w<0 ? -w : w; + if( zUtf==0 ) zUtf = ""; for(i=n=0; zUtf[i]; i++){ if( (zUtf[i]&0xc0)!=0x80 ){ n++; @@ -692,7 +713,7 @@ static char *local_getline(char *zLine, FILE *in){ if( stdin_is_interactive && in==stdin ){ char *zTrans = sqlite3_win32_mbcs_to_utf8_v2(zLine, 0); if( zTrans ){ - int nTrans = strlen30(zTrans)+1; + i64 nTrans = strlen(zTrans)+1; if( nTrans>nLine ){ zLine = realloc(zLine, nTrans); shell_check_oom(zLine); @@ -719,7 +740,7 @@ static char *local_getline(char *zLine, FILE *in){ ** be freed by the caller or else passed back into this routine via the ** zPrior argument for reuse. */ -#ifndef SQLITE_SHELL_WASM_MODE +#ifndef SQLITE_SHELL_FIDDLE static char *one_input_line(FILE *in, char *zPrior, int isContinuation){ char *zPrompt; char *zResult; @@ -739,7 +760,7 @@ static char *one_input_line(FILE *in, char *zPrior, int isContinuation){ } return zResult; } -#endif /* !SQLITE_SHELL_WASM_MODE */ +#endif /* !SQLITE_SHELL_FIDDLE */ /* ** Return the value of a hexadecimal digit. Return -1 if the input @@ -828,9 +849,9 @@ static void freeText(ShellText *p){ ** quote character for zAppend. */ static void appendText(ShellText *p, const char *zAppend, char quote){ - int len; - int i; - int nAppend = strlen30(zAppend); + i64 len; + i64 i; + i64 nAppend = strlen30(zAppend); len = nAppend+p->n+1; if( quote ){ @@ -989,10 +1010,10 @@ static void shellAddSchemaName( const char *zName = (const char*)sqlite3_value_text(apVal[2]); sqlite3 *db = sqlite3_context_db_handle(pCtx); UNUSED_PARAMETER(nVal); - if( zIn!=0 && strncmp(zIn, "CREATE ", 7)==0 ){ + if( zIn!=0 && cli_strncmp(zIn, "CREATE ", 7)==0 ){ for(i=0; inInit>in.mx ) return 0; + c = RE_START-1; } if( pRe->nState<=(sizeof(aSpace)/(sizeof(aSpace[0])*2)) ){ @@ -3971,6 +4021,10 @@ static int re_match(ReCompiled *pRe, const unsigned char *zIn, int nIn){ if( pRe->aArg[x]==c ) re_add_state(pNext, x+1); break; } + case RE_OP_ATSTART: { + if( cPrev==RE_START ) re_add_state(pThis, x+1); + break; + } case RE_OP_ANY: { if( c!=0 ) re_add_state(pNext, x+1); break; @@ -4052,7 +4106,9 @@ static int re_match(ReCompiled *pRe, const unsigned char *zIn, int nIn){ } } for(i=0; inState; i++){ - if( pRe->aOp[pNext->aState[i]]==RE_OP_ACCEPT ){ rc = 1; break; } + int x = pNext->aState[i]; + while( pRe->aOp[x]==RE_OP_GOTO ) x += pRe->aArg[x]; + if( pRe->aOp[x]==RE_OP_ACCEPT ){ rc = 1; break; } } re_match_end: sqlite3_free(pToFree); @@ -4207,7 +4263,6 @@ static const char *re_subcompile_string(ReCompiled *p){ iStart = p->nState; switch( c ){ case '|': - case '$': case ')': { p->sIn.i--; return 0; @@ -4244,6 +4299,14 @@ static const char *re_subcompile_string(ReCompiled *p){ re_insert(p, iPrev, RE_OP_FORK, p->nState - iPrev+1); break; } + case '$': { + re_append(p, RE_OP_MATCH, RE_EOF); + break; + } + case '^': { + re_append(p, RE_OP_ATSTART, 0); + break; + } case '{': { int m = 0, n = 0; int sz, j; @@ -4262,6 +4325,7 @@ static const char *re_subcompile_string(ReCompiled *p){ if( m==0 ){ if( n==0 ) return "both m and n are zero in '{m,n}'"; re_insert(p, iPrev, RE_OP_FORK, sz+1); + iPrev++; n--; }else{ for(j=1; jsIn.i+1>=pRe->sIn.mx ){ - re_append(pRe, RE_OP_MATCH, RE_EOF); - re_append(pRe, RE_OP_ACCEPT, 0); - *ppRe = pRe; - }else if( pRe->sIn.i>=pRe->sIn.mx ){ + if( pRe->sIn.i>=pRe->sIn.mx ){ re_append(pRe, RE_OP_ACCEPT, 0); *ppRe = pRe; }else{ @@ -4468,6 +4528,67 @@ static void re_sql_func( } } +#if defined(SQLITE_DEBUG) +/* +** This function is used for testing and debugging only. It is only available +** if the SQLITE_DEBUG compile-time option is used. +** +** Compile a regular expression and then convert the compiled expression into +** text and return that text. +*/ +static void re_bytecode_func( + sqlite3_context *context, + int argc, + sqlite3_value **argv +){ + const char *zPattern; + const char *zErr; + ReCompiled *pRe; + sqlite3_str *pStr; + int i; + int n; + char *z; + + zPattern = (const char*)sqlite3_value_text(argv[0]); + if( zPattern==0 ) return; + zErr = re_compile(&pRe, zPattern, sqlite3_user_data(context)!=0); + if( zErr ){ + re_free(pRe); + sqlite3_result_error(context, zErr, -1); + return; + } + if( pRe==0 ){ + sqlite3_result_error_nomem(context); + return; + } + pStr = sqlite3_str_new(0); + if( pStr==0 ) goto re_bytecode_func_err; + if( pRe->nInit>0 ){ + sqlite3_str_appendf(pStr, "INIT "); + for(i=0; inInit; i++){ + sqlite3_str_appendf(pStr, "%02x", pRe->zInit[i]); + } + sqlite3_str_appendf(pStr, "\n"); + } + for(i=0; (unsigned)inState; i++){ + sqlite3_str_appendf(pStr, "%-8s %4d\n", + ReOpName[(unsigned char)pRe->aOp[i]], pRe->aArg[i]); + } + n = sqlite3_str_length(pStr); + z = sqlite3_str_finish(pStr); + if( n==0 ){ + sqlite3_free(z); + }else{ + sqlite3_result_text(context, z, n-1, sqlite3_free); + } + +re_bytecode_func_err: + re_free(pRe); +} + +#endif /* SQLITE_DEBUG */ + + /* ** Invoke this routine to register the regexp() function with the ** SQLite database connection. @@ -4492,12 +4613,19 @@ int sqlite3_regexp_init( rc = sqlite3_create_function(db, "regexpi", 2, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, (void*)db, re_sql_func, 0, 0); +#if defined(SQLITE_DEBUG) + if( rc==SQLITE_OK ){ + rc = sqlite3_create_function(db, "regexp_bytecode", 1, + SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_DETERMINISTIC, + 0, re_bytecode_func, 0, 0); + } +#endif /* SQLITE_DEBUG */ } return rc; } /************************* End ../ext/misc/regexp.c ********************/ -#ifndef SQLITE_SHELL_WASM_MODE +#ifndef SQLITE_SHELL_FIDDLE /************************* Begin ../ext/misc/fileio.c ******************/ /* ** 2014-06-13 @@ -6767,8 +6895,8 @@ SQLITE_EXTENSION_INIT1 #endif /* typedef sqlite3_int64 i64; */ /* typedef unsigned char u8; */ -typedef UINT32_TYPE u32; /* 4-byte unsigned integer */ -typedef UINT16_TYPE u16; /* 2-byte unsigned integer */ +/* typedef UINT32_TYPE u32; // 4-byte unsigned integer // */ +/* typedef UINT16_TYPE u16; // 2-byte unsigned integer // */ #define MIN(a,b) ((a)<(b) ? (a) : (b)) #if defined(SQLITE_COVERAGE_TEST) || defined(SQLITE_MUTATION_TEST) @@ -10049,6 +10177,10 @@ static char *idxAppendText(int *pRc, char *zIn, const char *zFmt, ...){ */ static int idxIdentifierRequiresQuotes(const char *zId){ int i; + int nId = STRLEN(zId); + + if( sqlite3_keyword_check(zId, nId) ) return 1; + for(i=0; zId[i]; i++){ if( !(zId[i]=='_') && !(zId[i]>='0' && zId[i]<='9') @@ -11275,7 +11407,12 @@ void sqlite3_expert_destroy(sqlite3expert *p){ /************************* End ../ext/expert/sqlite3expert.c ********************/ #if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_ENABLE_DBPAGE_VTAB) -/************************* Begin ../ext/misc/dbdata.c ******************/ +#define SQLITE_SHELL_HAVE_RECOVER 1 +#else +#define SQLITE_SHELL_HAVE_RECOVER 0 +#endif +#if SQLITE_SHELL_HAVE_RECOVER +/************************* Begin ../ext/recover/dbdata.c ******************/ /* ** 2019-04-17 ** @@ -11349,16 +11486,20 @@ void sqlite3_expert_destroy(sqlite3expert *p){ ** It contains one entry for each b-tree pointer between a parent and ** child page in the database. */ + #if !defined(SQLITEINT_H) /* #include "sqlite3ext.h" */ /* typedef unsigned char u8; */ +/* typedef unsigned int u32; */ #endif SQLITE_EXTENSION_INIT1 #include #include +#ifndef SQLITE_OMIT_VIRTUALTABLE + #define DBDATA_PADDING_BYTES 100 typedef struct DbdataTable DbdataTable; @@ -11380,11 +11521,12 @@ struct DbdataCursor { /* Only for the sqlite_dbdata table */ u8 *pRec; /* Buffer containing current record */ - int nRec; /* Size of pRec[] in bytes */ - int nHdr; /* Size of header in bytes */ + sqlite3_int64 nRec; /* Size of pRec[] in bytes */ + sqlite3_int64 nHdr; /* Size of header in bytes */ int iField; /* Current field number */ u8 *pHdrPtr; u8 *pPtr; + u32 enc; /* Text encoding */ sqlite3_int64 iIntkey; /* Integer key value */ }; @@ -11577,14 +11719,14 @@ static int dbdataClose(sqlite3_vtab_cursor *pCursor){ /* ** Utility methods to decode 16 and 32-bit big-endian unsigned integers. */ -static unsigned int get_uint16(unsigned char *a){ +static u32 get_uint16(unsigned char *a){ return (a[0]<<8)|a[1]; } -static unsigned int get_uint32(unsigned char *a){ - return ((unsigned int)a[0]<<24) - | ((unsigned int)a[1]<<16) - | ((unsigned int)a[2]<<8) - | ((unsigned int)a[3]); +static u32 get_uint32(unsigned char *a){ + return ((u32)a[0]<<24) + | ((u32)a[1]<<16) + | ((u32)a[2]<<8) + | ((u32)a[3]); } /* @@ -11599,7 +11741,7 @@ static unsigned int get_uint32(unsigned char *a){ */ static int dbdataLoadPage( DbdataCursor *pCsr, /* Cursor object */ - unsigned int pgno, /* Page number of page to load */ + u32 pgno, /* Page number of page to load */ u8 **ppPage, /* OUT: pointer to page buffer */ int *pnPage /* OUT: Size of (*ppPage) in bytes */ ){ @@ -11609,25 +11751,27 @@ static int dbdataLoadPage( *ppPage = 0; *pnPage = 0; - sqlite3_bind_int64(pStmt, 2, pgno); - if( SQLITE_ROW==sqlite3_step(pStmt) ){ - int nCopy = sqlite3_column_bytes(pStmt, 0); - if( nCopy>0 ){ - u8 *pPage; - pPage = (u8*)sqlite3_malloc64(nCopy + DBDATA_PADDING_BYTES); - if( pPage==0 ){ - rc = SQLITE_NOMEM; - }else{ - const u8 *pCopy = sqlite3_column_blob(pStmt, 0); - memcpy(pPage, pCopy, nCopy); - memset(&pPage[nCopy], 0, DBDATA_PADDING_BYTES); + if( pgno>0 ){ + sqlite3_bind_int64(pStmt, 2, pgno); + if( SQLITE_ROW==sqlite3_step(pStmt) ){ + int nCopy = sqlite3_column_bytes(pStmt, 0); + if( nCopy>0 ){ + u8 *pPage; + pPage = (u8*)sqlite3_malloc64(nCopy + DBDATA_PADDING_BYTES); + if( pPage==0 ){ + rc = SQLITE_NOMEM; + }else{ + const u8 *pCopy = sqlite3_column_blob(pStmt, 0); + memcpy(pPage, pCopy, nCopy); + memset(&pPage[nCopy], 0, DBDATA_PADDING_BYTES); + } + *ppPage = pPage; + *pnPage = nCopy; } - *ppPage = pPage; - *pnPage = nCopy; } + rc2 = sqlite3_reset(pStmt); + if( rc==SQLITE_OK ) rc = rc2; } - rc2 = sqlite3_reset(pStmt); - if( rc==SQLITE_OK ) rc = rc2; return rc; } @@ -11636,17 +11780,30 @@ static int dbdataLoadPage( ** Read a varint. Put the value in *pVal and return the number of bytes. */ static int dbdataGetVarint(const u8 *z, sqlite3_int64 *pVal){ - sqlite3_int64 v = 0; + sqlite3_uint64 u = 0; int i; for(i=0; i<8; i++){ - v = (v<<7) + (z[i]&0x7f); - if( (z[i]&0x80)==0 ){ *pVal = v; return i+1; } + u = (u<<7) + (z[i]&0x7f); + if( (z[i]&0x80)==0 ){ *pVal = (sqlite3_int64)u; return i+1; } } - v = (v<<8) + (z[i]&0xff); - *pVal = v; + u = (u<<8) + (z[i]&0xff); + *pVal = (sqlite3_int64)u; return 9; } +/* +** Like dbdataGetVarint(), but set the output to 0 if it is less than 0 +** or greater than 0xFFFFFFFF. This can be used for all varints in an +** SQLite database except for key values in intkey tables. +*/ +static int dbdataGetVarintU32(const u8 *z, sqlite3_int64 *pVal){ + sqlite3_int64 val; + int nRet = dbdataGetVarint(z, &val); + if( val<0 || val>0xFFFFFFFF ) val = 0; + *pVal = val; + return nRet; +} + /* ** Return the number of bytes of space used by an SQLite value of type ** eType. @@ -11683,9 +11840,10 @@ static int dbdataValueBytes(int eType){ */ static void dbdataValue( sqlite3_context *pCtx, + u32 enc, int eType, u8 *pData, - int nData + sqlite3_int64 nData ){ if( eType>=0 && dbdataValueBytes(eType)<=nData ){ switch( eType ){ @@ -11727,7 +11885,19 @@ static void dbdataValue( default: { int n = ((eType-12) / 2); if( eType % 2 ){ - sqlite3_result_text(pCtx, (const char*)pData, n, SQLITE_TRANSIENT); + switch( enc ){ +#ifndef SQLITE_OMIT_UTF16 + case SQLITE_UTF16BE: + sqlite3_result_text16be(pCtx, (void*)pData, n, SQLITE_TRANSIENT); + break; + case SQLITE_UTF16LE: + sqlite3_result_text16le(pCtx, (void*)pData, n, SQLITE_TRANSIENT); + break; +#endif + default: + sqlite3_result_text(pCtx, (char*)pData, n, SQLITE_TRANSIENT); + break; + } }else{ sqlite3_result_blob(pCtx, pData, n, SQLITE_TRANSIENT); } @@ -11755,6 +11925,7 @@ static int dbdataNext(sqlite3_vtab_cursor *pCursor){ rc = dbdataLoadPage(pCsr, pCsr->iPgno, &pCsr->aPage, &pCsr->nPage); if( rc!=SQLITE_OK ) return rc; if( pCsr->aPage ) break; + if( pCsr->bOnePage ) return SQLITE_OK; pCsr->iPgno++; } pCsr->iCell = pTab->bPtr ? -2 : 0; @@ -11818,7 +11989,7 @@ static int dbdataNext(sqlite3_vtab_cursor *pCursor){ if( bNextPage || iOff>pCsr->nPage ){ bNextPage = 1; }else{ - iOff += dbdataGetVarint(&pCsr->aPage[iOff], &nPayload); + iOff += dbdataGetVarintU32(&pCsr->aPage[iOff], &nPayload); } /* If this is a leaf intkey cell, load the rowid */ @@ -11865,7 +12036,7 @@ static int dbdataNext(sqlite3_vtab_cursor *pCursor){ /* Load content from overflow pages */ if( nPayload>nLocal ){ sqlite3_int64 nRem = nPayload - nLocal; - unsigned int pgnoOvfl = get_uint32(&pCsr->aPage[iOff]); + u32 pgnoOvfl = get_uint32(&pCsr->aPage[iOff]); while( nRem>0 ){ u8 *aOvfl = 0; int nOvfl = 0; @@ -11885,7 +12056,8 @@ static int dbdataNext(sqlite3_vtab_cursor *pCursor){ } } - iHdr = dbdataGetVarint(pCsr->pRec, &nHdr); + iHdr = dbdataGetVarintU32(pCsr->pRec, &nHdr); + if( nHdr>nPayload ) nHdr = 0; pCsr->nHdr = nHdr; pCsr->pHdrPtr = &pCsr->pRec[iHdr]; pCsr->pPtr = &pCsr->pRec[pCsr->nHdr]; @@ -11899,7 +12071,7 @@ static int dbdataNext(sqlite3_vtab_cursor *pCursor){ if( pCsr->pHdrPtr>&pCsr->pRec[pCsr->nRec] ){ bNextPage = 1; }else{ - pCsr->pHdrPtr += dbdataGetVarint(pCsr->pHdrPtr, &iType); + pCsr->pHdrPtr += dbdataGetVarintU32(pCsr->pHdrPtr, &iType); pCsr->pPtr += dbdataValueBytes(iType); } } @@ -11938,6 +12110,18 @@ static int dbdataEof(sqlite3_vtab_cursor *pCursor){ return pCsr->aPage==0; } +/* +** Return true if nul-terminated string zSchema ends in "()". Or false +** otherwise. +*/ +static int dbdataIsFunction(const char *zSchema){ + size_t n = strlen(zSchema); + if( n>2 && zSchema[n-2]=='(' && zSchema[n-1]==')' ){ + return (int)n-2; + } + return 0; +} + /* ** Determine the size in pages of database zSchema (where zSchema is ** "main", "temp" or the name of an attached database) and set @@ -11948,10 +12132,16 @@ static int dbdataDbsize(DbdataCursor *pCsr, const char *zSchema){ DbdataTable *pTab = (DbdataTable*)pCsr->base.pVtab; char *zSql = 0; int rc, rc2; + int nFunc = 0; sqlite3_stmt *pStmt = 0; - zSql = sqlite3_mprintf("PRAGMA %Q.page_count", zSchema); + if( (nFunc = dbdataIsFunction(zSchema))>0 ){ + zSql = sqlite3_mprintf("SELECT %.*s(0)", nFunc, zSchema); + }else{ + zSql = sqlite3_mprintf("PRAGMA %Q.page_count", zSchema); + } if( zSql==0 ) return SQLITE_NOMEM; + rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pStmt, 0); sqlite3_free(zSql); if( rc==SQLITE_OK && sqlite3_step(pStmt)==SQLITE_ROW ){ @@ -11962,6 +12152,25 @@ static int dbdataDbsize(DbdataCursor *pCsr, const char *zSchema){ return rc; } +/* +** Attempt to figure out the encoding of the database by retrieving page 1 +** and inspecting the header field. If successful, set the pCsr->enc variable +** and return SQLITE_OK. Otherwise, return an SQLite error code. +*/ +static int dbdataGetEncoding(DbdataCursor *pCsr){ + int rc = SQLITE_OK; + int nPg1 = 0; + u8 *aPg1 = 0; + rc = dbdataLoadPage(pCsr, 1, &aPg1, &nPg1); + assert( rc!=SQLITE_OK || nPg1==0 || nPg1>=512 ); + if( rc==SQLITE_OK && nPg1>0 ){ + pCsr->enc = get_uint32(&aPg1[56]); + } + sqlite3_free(aPg1); + return rc; +} + + /* ** xFilter method for sqlite_dbdata and sqlite_dbptr. */ @@ -11979,19 +12188,28 @@ static int dbdataFilter( assert( pCsr->iPgno==1 ); if( idxNum & 0x01 ){ zSchema = (const char*)sqlite3_value_text(argv[0]); + if( zSchema==0 ) zSchema = ""; } if( idxNum & 0x02 ){ pCsr->iPgno = sqlite3_value_int(argv[(idxNum & 0x01)]); pCsr->bOnePage = 1; }else{ - pCsr->nPage = dbdataDbsize(pCsr, zSchema); rc = dbdataDbsize(pCsr, zSchema); } if( rc==SQLITE_OK ){ + int nFunc = 0; if( pTab->pStmt ){ pCsr->pStmt = pTab->pStmt; pTab->pStmt = 0; + }else if( (nFunc = dbdataIsFunction(zSchema))>0 ){ + char *zSql = sqlite3_mprintf("SELECT %.*s(?2)", nFunc, zSchema); + if( zSql==0 ){ + rc = SQLITE_NOMEM; + }else{ + rc = sqlite3_prepare_v2(pTab->db, zSql, -1, &pCsr->pStmt, 0); + sqlite3_free(zSql); + } }else{ rc = sqlite3_prepare_v2(pTab->db, "SELECT data FROM sqlite_dbpage(?) WHERE pgno=?", -1, @@ -12004,13 +12222,20 @@ static int dbdataFilter( }else{ pTab->base.zErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pTab->db)); } + + /* Try to determine the encoding of the db by inspecting the header + ** field on page 1. */ + if( rc==SQLITE_OK ){ + rc = dbdataGetEncoding(pCsr); + } + if( rc==SQLITE_OK ){ rc = dbdataNext(pCursor); } return rc; } -/* +/* ** Return a column for the sqlite_dbdata or sqlite_dbptr table. */ static int dbdataColumn( @@ -12054,11 +12279,12 @@ static int dbdataColumn( case DBDATA_COLUMN_VALUE: { if( pCsr->iField<0 ){ sqlite3_result_int64(ctx, pCsr->iIntkey); - }else{ + }else if( &pCsr->pRec[pCsr->nRec] >= pCsr->pPtr ){ sqlite3_int64 iType; - dbdataGetVarint(pCsr->pHdrPtr, &iType); + dbdataGetVarintU32(pCsr->pHdrPtr, &iType); dbdataValue( - ctx, iType, pCsr->pPtr, &pCsr->pRec[pCsr->nRec] - pCsr->pPtr + ctx, pCsr->enc, iType, pCsr->pPtr, + &pCsr->pRec[pCsr->nRec] - pCsr->pPtr ); } break; @@ -12128,56 +12354,3170 @@ int sqlite3_dbdata_init( return sqlite3DbdataRegister(db); } -/************************* End ../ext/misc/dbdata.c ********************/ +#endif /* ifndef SQLITE_OMIT_VIRTUALTABLE */ + +/************************* End ../ext/recover/dbdata.c ********************/ +/************************* Begin ../ext/recover/sqlite3recover.h ******************/ +/* +** 2022-08-27 +** +** The author disclaims copyright to this source code. In place of +** a legal notice, here is a blessing: +** +** May you do good and not evil. +** May you find forgiveness for yourself and forgive others. +** May you share freely, never taking more than you give. +** +************************************************************************* +** +** This file contains the public interface to the "recover" extension - +** an SQLite extension designed to recover data from corrupted database +** files. +*/ + +/* +** OVERVIEW: +** +** To use the API to recover data from a corrupted database, an +** application: +** +** 1) Creates an sqlite3_recover handle by calling either +** sqlite3_recover_init() or sqlite3_recover_init_sql(). +** +** 2) Configures the new handle using one or more calls to +** sqlite3_recover_config(). +** +** 3) Executes the recovery by repeatedly calling sqlite3_recover_step() on +** the handle until it returns something other than SQLITE_OK. If it +** returns SQLITE_DONE, then the recovery operation completed without +** error. If it returns some other non-SQLITE_OK value, then an error +** has occurred. +** +** 4) Retrieves any error code and English language error message using the +** sqlite3_recover_errcode() and sqlite3_recover_errmsg() APIs, +** respectively. +** +** 5) Destroys the sqlite3_recover handle and frees all resources +** using sqlite3_recover_finish(). +** +** The application may abandon the recovery operation at any point +** before it is finished by passing the sqlite3_recover handle to +** sqlite3_recover_finish(). This is not an error, but the final state +** of the output database, or the results of running the partial script +** delivered to the SQL callback, are undefined. +*/ + +#ifndef _SQLITE_RECOVER_H +#define _SQLITE_RECOVER_H + +/* #include "sqlite3.h" */ + +#ifdef __cplusplus +extern "C" { #endif -#if defined(SQLITE_ENABLE_SESSION) /* -** State information for a single open session +** An instance of the sqlite3_recover object represents a recovery +** operation in progress. +** +** Constructors: +** +** sqlite3_recover_init() +** sqlite3_recover_init_sql() +** +** Destructor: +** +** sqlite3_recover_finish() +** +** Methods: +** +** sqlite3_recover_config() +** sqlite3_recover_errcode() +** sqlite3_recover_errmsg() +** sqlite3_recover_run() +** sqlite3_recover_step() */ -typedef struct OpenSession OpenSession; -struct OpenSession { - char *zName; /* Symbolic name for this session */ - int nFilter; /* Number of xFilter rejection GLOB patterns */ - char **azFilter; /* Array of xFilter rejection GLOB patterns */ - sqlite3_session *p; /* The open session */ -}; +typedef struct sqlite3_recover sqlite3_recover; + +/* +** These two APIs attempt to create and return a new sqlite3_recover object. +** In both cases the first two arguments identify the (possibly +** corrupt) database to recover data from. The first argument is an open *** 20644 LINES SKIPPED ***