SD card formatting

Tue Mar 17 01:53:37 UTC 2020

On 3/16/20 2:08 PM, Polytropon wrote:
> On Mon, 16 Mar 2020 12:36:25 -0600, Gary Aitken wrote:
>> On 3/16/20 2:05 AM, Polytropon wrote:
>>> On Sun, 15 Mar 2020 19:06:22 -0600, Gary Aitken wrote:
>>>> On 3/15/20 5:00 PM, Polytropon wrote:
>>>>> On Sun, 15 Mar 2020 15:39:33 -0600, Gary Aitken wrote:
>>>>>> 11.3-RELEASE-p6 GENERIC  amd64
>>>>>>
>>>>>> I'm having trouble reading SD cards formatted in my camera (Olympus
>>>>>> EM1-MkII) or on a Win 7 system.
>>>>>
>>>>> What filesystem is in use? For cards, it's typically FAT.
>>>>>
>>>>>
>>>>>
>>>>>> When attempting to mount, I get the following:
>>>>>>
>>>>>> $ mount -t msdosfs /dev/da0s1 /mnt/memstick mount_msdosfs:
>>>>>> /dev/da0s1: Invalid argument $ mount -t ntfs /dev/da0s1
>>>>>> /mnt/memstick mount: /dev/da0s1: Operation not supported by device
>>>>>>
>>>>>> Cards used without formatting *usually* seem to work.
>>>>>
>>>>> So those are preformatted (sometimes sold this way, sometimes
>>>>> initialized when first used in a camera).
>>>>>
>>>>>
>>>>>
>>>>>> If I look at the cards which don't mount using gpart, I see:
>>>>>>
>>>>>> Card formatted in camera:
>>>>>> $ gpart show -p /dev/da0
>>>>>> =>       63 120944577    da0  MBR  (58G)
>>>>>>            63      32705         - free -  (16M)
>>>>>>         32768  120911872  da0s1  ntfs  [active]  (58G)
>>>>>                                   ^^^^
>>>>>
>>>>> It seems that the card has been accidentally formatted with NTFS. In
>>>>> most cases, cameras cannot use that. If you want to mount it on
>>>>> FreeBSD, use ntfs-3g.
>>>>
>>>> As stated above, this is a card formatted _in the camera_.  So the
>>>> camera can read it fine.  The problem is fbsd can't mount it.
>>>
>>> >From the program output presented, it seems that you're
>>> trying to mount a NTFS partition using the FAT mount option,
>>> which will not work. Try ntfs-3g (needs to be installed;
>>> mount_ntfs is no longer part of the OS).
>>
>> Ah, that's a help, thanks.
>> Ugh:
>>
>> # ntfs-3g /dev/da0s1 /mnt/memstick
>> NTFS signature is missing.
>> # ntfs-3g.probe --readonly /dev/da0s1
>> NTFS signature is missing.
>> # file -r - </dev/da0
>> /dev/stdin: DOS/MBR boot sector; partition 1 : ID=0x7, active,
>> start-CHS (0x2,10,9), end-CHS (0x3ff,254,63), startsector 32768,
>> 124702720 sectors
>> # file -r - < /dev/da0s1
>> /dev/stdin: DOS/MBR boot sector
> 
> You can install ""libfsntfs" and use its identification tool
> to see what you really have here. If the NTFS signature is
> missing, the filesystem either is heavily damaged (and in
> that case, it's strange to see the "Windows" PC and camera
> being able to use it), ot, it's _not_ a NTFS filesystem,
> and some partitioning data is terribly misleading.

Ugh.  Apparently it's a strange beast:

# fsntfsinfo -v /dev/da0
fsntfsinfo 20191221

Unable to open: /dev/da0.
libcfile_file_read_buffer_with_error_code: unable to read from file with error: Invalid argument
libcfile_file_read_buffer: unable to read from file.
libbfio_file_read_buffer: unable to read from file: /dev/da0.
libbfio_file_range_io_handle_read_buffer: unable to read from file IO handle.
libbfio_handle_read_buffer: unable to read from handle.
libfsntfs_check_volume_signature_file_io_handle: unable to read signature.
info_handle_open_input: unable to check volume signature.

/dev/da0s1 gives same result

However, I just discovered this:
   According to
     https://en.wikipedia.org/wiki/File_Allocation_Table#FAT12
   under the section exFAT:
     exFAT is intended for use on flash drives (such as SDXC and Memory Stick XC), where FAT32 is otherwise used. Microsoft's GUI and command-line format utilities offer it as an alternative to NTFS (and, for smaller partitions, to FAT16B and FAT32). The MBR partition type is 0x07 (the same as used for IFS, HPFS, and NTFS). Logical geometry information located in the VBR is stored in a format not resembling any kind of BPB.

So the 0x7 NTFS type may in fact be for exFAT

I've now retried formatting the card several different ways on win7 and
they all failed at the very end with no useful information I finally got
somithing maybe useful with this:

format g: /fs:exFAT
goes through format process & verifies, then asks for label.
When attempting to write label, it fails with
Invalid media or track 0 bad

I'm leaving town tomorrow for a bit so won't be able to look into this
further until I get back.  I also need to scrounge up another card to
test with.

>>>>> Personally, I tend to leave the formatting to the camera which I want
>>>>> to use the card in; the camera "knows best" what it can uderstand.
>>>>> :-)
>>>>
>>>> In this case, the camera is not the problem; cards formatted in the camera
>>>> are usable by the camera.  But they aren't mountable on fbsd.  So I can't
>>>> transfer the files to fbsd, either from the camera or by inserting the
>>>> card in an fbsd usb adapter.
>>>
>>> Depending on the camera, a USB connection can show up as
>>> a mass storage device (leads to /dev/da<something>) or a
>>> MTP interface (leads to /dev/ugen<something>). In many cases,
>>> the "personality" can be seleted by the menu in the camera.
>>
>> It shows up as mass storage, /dev/da0 in this case
> 
> So you can access the SD card "through" the camera, but
> you'll probably experience the same problem, right?

Cards formatted in the camera are usable by the camera and readable on
win7; but not mountable/readable on fbsd using either "mount -t msdosfs"
or "ntfs-3g".  So far, no card formatted by fbsd is usable / recognized
by the camera, but as stated above, the card I'm testing with for
formatting may be bad.  I'm reluctant to try formatting any of the other
cards I have for fear or making them unusable; will try to find one I
can risk when I get back.

>>>> For a card never reformatted (i.e. formatted by the manufacturer, and
>>>> used in the camera); this card is usable by the camera and mountable
>>>> by fbsd:
>>>>
>>>> # file -r - < /dev/da0
>>>> /dev/stdin: DOS/MBR boot sector; partition 1 : ID=0xc, active,
>>>> start-CHS (0x0,130,3), end-CHS (0x3ff,254,63), startsector 8192,
>>>> 31496192 sectors
>>>> # file -r - < /dev/da0s1
>>>> /dev/stdin: , code offset 0x0+3, OEM-ID "        ", sectors/cluster 64,
>>>> reserved sectors 504, Media descriptor 0xf8, sectors/track 63,
>>>> heads 255, hidden sectors 8192, sectors 31496192 (volumes > 32 MB),
>>>> FAT (32 bit), sectors/FAT 3844, serial number 0x0, unlabeled
>>>> # mount_msdosfs /dev/da0s1 /mnt/memstick
>>>> # ls /mnt/memstick
>>>> DCIM                            System Volume Information
>>>
>>> This doesn't actually look like a "mint condition" card. A
>>> directory named "DCIM" is typically created by a camera.
>>> However, _this_ card is FAT-formatted, and therefore can be
>>> mounted with msdosfs as correctly shown above. The directory
>>> name "System Volume Information" suggests it has been in a
>>> "Windows" PC before. COnclusion: Card works both in camera
>>> an in "Windows" PC.
>>
>> Correct, it's not an unused card.  It's been put in the camera,
>> recorded on, and then put in a windows system to transfer images.
> 
> Okay, so that's the configuration to replicate.
> 
> 
> 
>>> So the camera seems (!) to initialize cards as NTFS (strange,
>>> but surely possible), but also understands FAT.
>>
>> That seems to be the case.
>> Since ntfs-3g doesn't seem to like the camera-generated NTFS
>> filesystem, it seems my best option is to try to format with
>> the FAT it understands.
> 
> Fully agree. NTFS seems to be the wrong thing (but gpart
> reports it).
> 
> Just a stupid question:
> 
> Have you tried "newfs_msdosfs -A -F 32 /dev/da0"?
> I mean, without the gpart part? Could that work?

Doesn't work, camera can't deal with it.

>>> All you need to do now is to replicate _that_ formatting.
>>> The output shows "FAT (32 bit)", and the partitioning also
>>> looks reasonable. You should be able to create this either
>>> with gpart + newfs_msdosfs (should be quite standard today),
>>> or the traditional way with fdisk + newfs_msdosfs (archaic,
>>> but should also still work).
>>
>> Tried the following:
>>
>> # gpart create -s MBR /dev/da0
>> da0 created
>> # gpart show -l da0
>> =>       63  120944577  da0  MBR  (58G)
>>            63  120944577       - free -  (58G)
>> # gpart add -t fat32 -a 4K /dev/da0
>> da0s1 added
>> # gpart show -l /dev/da0
>> =>       63  120944577  da0  MBR  (58G)
>>            63          1       - free -  (512B)
>>            64  120944576    1  (null)  (58G)
>> # newfs_msdos -F 32 -A /dev/da0s1
>> /dev/da0s1: 120885440 sectors in 1888835 FAT32 clusters (32768 bytes/cluster)
>> BytesPerSec=512 SecPerClust=64 ResSectors=46 FATs=2 Media=0xf0
>> SecPerTrack=63 Heads=255 HiddenSecs=0 HugeSectors=120944576
>> FATsecs=14761 RootCluster=2 FSInfo=1 Backup=2
>> # mount -t msdosfs /dev/da0s1 /mnt/memstick
>> mount_msdosfs: /dev/da0s1: Invalid argument
>>
>> Obviously I'm missing something?
> 
> You could see if -F 12 or -F 16 changes something. Another
> option is to use traditional fdisk instead of gpart...

Will hold off until I get another (hopefully good) card.
Thanks,

Gary