From nobody Sat Apr 16 22:13:02 2022 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 0D68411CDE89 for ; Sat, 16 Apr 2022 22:13:21 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: from mail-qk1-x732.google.com (mail-qk1-x732.google.com [IPv6:2607:f8b0:4864:20::732]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4KgnX02zNLz3rJK for ; Sat, 16 Apr 2022 22:13:20 +0000 (UTC) (envelope-from sodynet1@gmail.com) Received: by mail-qk1-x732.google.com with SMTP id s4so8971883qkh.0 for ; Sat, 16 Apr 2022 15:13:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=+0Za3+nd4f+SGj83R/Qoq+DwceS1EZSftK3YgP3+txc=; b=QDBd+qttT+Bfr7tOvhrfczMKh1bepfrPkhSdljKii/VpULJXnoygg6mNR5iITiiPgD zk8E8tUwaAFxaCVBrFFC2ZU3hwKgHU2f8TRVQ/NRxz7wXpw+us2vXuF7E5ARKs+422fg joVRCqG6hc9WYap1ZriQjbs11HyeG7/6uoQPctii35ki611yqDcP7DDDQZc5CyvWdPpX 6/zed/1bgQoOtLFYA90QJUQsBZeFEWV/BMTigegBMAqd0haxCIqCrmjK6cfZfMsxawMV 9e9EhmsSd/A4ui30j7scVfxP7lLWS0WCrq6Dh7bQ9E9N1OIioZENr8amkwV04dgXHjBT z7vA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=+0Za3+nd4f+SGj83R/Qoq+DwceS1EZSftK3YgP3+txc=; b=DaI7jP6jpriIj/iNCD4sVnTYvsQoaNZwShMSsf5HYRI7feLbqdc31WNeNSqDrQTY2H 4tv5P9Jf3So3z0VRX0zGrPvYOijGmIQrjpmFYGhNPob5YchZ4kN4298ixcayadYk87d/ p5CvH/wD2eEDxWiFPlSP4Q66RNxsn+g0hlsAzbGexxwf6l7fHTJi3f7LQxe4xAb8RXF8 CAQFray7XlXCKkoPPU4LPNKRymNFOW7gRn6MkJa02KUINj32igbf1Q59fAOSTS+51dNi scMS5uB6OhlL08sDl+h1A705zCzFfT7vuF+mfd3p+eb2gSfCydbVwGQXlGdEHzDaaPvf RoiA== X-Gm-Message-State: AOAM532UmIbeViZ8rZsSoBfoJ/tP6M3HifRVHWzCf+MV+aJBFKvujmSI 6yf2RXLckphZ9VFNXyfJntRtwjPYXmK3InOuvMWrqYWi X-Google-Smtp-Source: ABdhPJzxTUojoYvzMGkNKLDgBxL+1UF4GKoRvaVM9t0YV0OBGNlpB/WLl32OMfX51w6RJd4zttunUqwg89rfQy4fCuA= X-Received: by 2002:a05:620a:2a12:b0:69c:5ab9:a08d with SMTP id o18-20020a05620a2a1200b0069c5ab9a08dmr3000160qkp.326.1650147194134; Sat, 16 Apr 2022 15:13:14 -0700 (PDT) List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@freebsd.org MIME-Version: 1.0 References: In-Reply-To: From: Sami Halabi Date: Sun, 17 Apr 2022 01:13:02 +0300 Message-ID: Subject: Re: recover deleted file To: Warner Losh Cc: FreeBSD Current Content-Type: multipart/alternative; boundary="00000000000055623b05dcccd574" X-Rspamd-Queue-Id: 4KgnX02zNLz3rJK X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=QDBd+qtt; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of sodynet1@gmail.com designates 2607:f8b0:4864:20::732 as permitted sender) smtp.mailfrom=sodynet1@gmail.com X-Spamd-Result: default: False [-3.99 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::732:from]; NEURAL_HAM_SHORT(-0.99)[-0.995]; MID_RHS_MATCH_FROMTLD(0.00)[]; MLMMJ_DEST(0.00)[freebsd-current]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N --00000000000055623b05dcccd574 Content-Type: text/plain; charset="UTF-8" Hi warner, Thanks for trying :) Actually my use case was (if you read later replies, i gave up since the downtime was too long and couldn't wait more) a VM in ESXi. so all the underlying stuff of the disks/TRIM.. is hidden an inaccessible for me (hosting provider). In my case I tried to recover MariaDB database files, and some tar.gz file of backups of the db that I accidentally deleted all together (mysql* instead of mysql/* , my bad, deleted mysql/ & mysql_backups!!). I stopped all services immediately, so maybe I would succeed if I wasn't time limited. I understand its hard to undelete since no one designed UFS/ZFS to do so.. that why I asked in later replies to see if someone would step in and implement such a "feature" and I suggested some directions/thoughts. As soren@ suggested in later reply it maybe would be easier to implement custom rm script that moves files to "Recycle bin" directory (and empty it after some period) but as a programmer I know that perfection is needed :) so It might start as a simple task and end in many what-if's (unfortunattly I did my last C programming in late 2003!). What amzes me is that this "feature" was asked too much in the last decade or two and no one ever implemented it, maybe it's not needed in daily usage, but in disasters it would be super userful, save admins many time and nerves.. For now I did some backup tools locally and used chflags to mark them undeletable so I wouldn't do that mistake again, plus I rsync them to my home storage.. so probably I would be more resilent to such mistakes in the future.. but the same problem remains.. accidently deleted file(s)/directory(s) are the nightmare of all admins in earth!! Sami On Sun, Apr 17, 2022 at 12:42 AM Warner Losh wrote: > > > On Sat, Apr 16, 2022 at 5:24 AM Sami Halabi wrote: > >> Hi, >> is there anyway easy to restore deleted file by accident in UFS???? >> > > Do you know what the contents of the file is? At least the first, say, > ~32k? > > The problem with unrm for ufs is that the directory entry has the inode > number stored in it. > Without the inode number, you won't get very far. > With the inode number, you can get the first 12 filesystem blocks of the > file and the > first three indirect blocks. Once you have those, you can reconstruct the > file. > > But only if the inode hasn't been zero'd out (which it likely has, another > thing that makes > UFS undelete harder). But all hope isn't lost... UFS has a predictable > allocation algorithm > that lets you get much of the file back (which is why I asked if you know > how it should start: > you can find where it starts in the data blocks and maybe get lucky with > the rest if the > data spills into indirect blocks). > > However, that's only if you don't have TRIM enabled on the filesystem. If > you do, > then UFS will do a BIO_DELETE of the blocks, which means their contents are > likely gone at the drive level. I say likely because there's weasel words > in the ATA > spec that allows a drive to return the prior contents of the blocks, or > all zeros or > the drive's initialization pattern (usually all 1's) when the blocks are > later read. Same > goes for NVMe drives (with the additional constraint it must be > deterministic). So there's > may still be a chance you can read the old contents, but drives that do > that are rare > in my experience (which is admittedly quite narrow). > > But, if you want to use fsdb to try to recover this data, or write your > own tools, > then you should likely have a copy of the daemon book (The Design and > Implementation > of the FreeBSD Operating System). It explains a lot of the finer details > of UFS and > reference to it likely will catch me where my memory isn't quite right in > the above > descriptions. > > So, it's for all these reasons you can't find somebody with a unrm command > for ufs > like you can for DOS or other filesystems. I wish I had a better answer > for you. > > Warner > > >> Sami >> >> -- >> Sami Halabi >> Information Systems Engineer >> NMS Projects Expert, FreeBSD SysAdmin Expert >> Asterisk Expert >> > -- Sami Halabi Information Systems Engineer NMS Projects Expert, FreeBSD SysAdmin Expert Asterisk Expert --00000000000055623b05dcccd574 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Hi warner,
Thanks for trying :)

Actually my use case was (if you read later replies, i gave up since the= downtime was too long and couldn't wait more) a VM in ESXi.
= so all the underlying stuff of the disks/TRIM.. is hidden an inaccessible= =C2=A0for me (hosting provider).

In my case I trie= d to recover MariaDB database files, and some tar.gz file of backups of the= db that I accidentally=C2=A0deleted all=C2=A0together (mysql* instead of m= ysql/*=C2=A0 , my bad, deleted mysql/ & mysql_backups!!). I stopped all= services immediately, so maybe I would succeed if I wasn't time limite= d.

I understand=C2=A0its hard to undelete since no= one designed UFS/ZFS to do so.. that why I asked in later replies to see i= f someone would step in and implement such a "feature" and I sugg= ested=C2=A0some directions/thoughts.

As soren@ sug= gested in=C2=A0later reply it maybe would be easier to implement custom rm = script that moves files to "Recycle bin" directory (and empty it = after some period) but as a programmer I know that perfection is needed :) = so It might start as a simple task and end in many what-if's (unfortuna= ttly=C2=A0I did my last C programming in late=C2=A02003!).

What amzes me is that this "feature" was asked too much = in the last decade or two and no one ever implemented it, maybe it's no= t needed in daily usage, but in disasters it would be super userful, save a= dmins many time and nerves..

For now I did some ba= ckup tools locally and used chflags to mark them undeletable so I wouldn= 9;t do that mistake again, plus I rsync them to my home storage.. so probab= ly I would be more resilent=C2=A0to such mistakes in the future.. but the s= ame problem remains.. accidently deleted file(s)/directory(s) are the night= mare of all admins=C2=A0in earth!!

Sami

On Su= n, Apr 17, 2022 at 12:42 AM Warner Losh <imp@bsdimp.com> wrote:


On Sat, Apr 16, 2022= at 5:24 AM Sami Halabi <sodynet1@gmail.com> wrote:
Hi,
is there anyway easy to= restore deleted file by accident in UFS????
<= br>
Do you know what the contents of the file is? At least the fi= rst, say, ~32k?

The problem with unrm for ufs is t= hat the directory entry has the inode number stored in it.
Withou= t the inode number, you won't get very far.
With the inode nu= mber, you can get the first 12 filesystem blocks of the file and the
<= div>first three indirect blocks. Once you have those, you can reconstruct t= he file.

But only if the inode hasn't been zer= o'd out (which it likely has, another thing that makes
UFS un= delete harder). But all hope isn't lost...=C2=A0 UFS has a predictable = allocation algorithm
that lets you get much of the file back (whi= ch is why I asked if you know how it should start:
you can find w= here it starts in the data blocks and maybe get lucky with the rest if the<= /div>
data spills into indirect blocks).

Howev= er, that's only if you don't have TRIM enabled on the filesystem. I= f you do,
then UFS will do a BIO_DELETE of the blocks, which mean= s their contents are
likely gone at the drive level. I say likely= because there's weasel words in the ATA
spec that allows a d= rive to return the prior contents of the blocks, or all zeros or
= the drive's initialization pattern (usually all 1's) when the block= s are later read. Same
goes for NVMe drives (with the additional = constraint it must be deterministic). So there's
may still be= a chance you can read the old contents, but drives that do that are rare
in my experience (which is admittedly quite narrow).
But, if you want to use fsdb to try to recover this data, or wr= ite your own tools,
then you should likely have a copy of the dae= mon book (The Design and Implementation
of the FreeBSD Operating = System). It explains a lot of the finer details of UFS and
refere= nce to it likely will catch me where my memory isn't quite right in the= above
descriptions.

So, it's for al= l these reasons you can't find somebody with a unrm command for ufs
like you can for DOS or other filesystems. I wish I had a better ans= wer for you.

Warner
=C2=A0
Sami

--
Sami Halabi
Information Systems Engineer
NMS Pr= ojects Expert,=C2=A0FreeBSD SysAdmin Exper= t
Asterisk Expert


--
Sami Hala= bi
Information Systems Engineer
NMS Projects Expert,=C2=A0FreeBSD SysAdmin Expert
Aste= risk Expert
--00000000000055623b05dcccd574--