I have an OEM generated Win CE 5.0 image with TFAT support. I think.
The only reason I think I have TFAT is that my flash is sporadically
becoming corrupt (no power loss that I know of), and I can only fix it
from me CE hardware.

Anyway, something is corrupting my flash rendering it unbootable. I
do no fancy I/O - simple file writes from time to time, lot's of
reads. The only way to fix the CF is to remove all the files (using
the CE hardware) and re-write. Doing this seems to recover the file
system and all is well for days/weeks until it happens again.

Is there a way to detect when TFAT fails to write all of the data
necessary and has to rollback the previous operation?

Is there any special error handling I can implemnent to detect when
TFAT goes south?

Re: Detecting TFAT file system corruption - possible? by Michel

Michel
Mon Feb 04 13:31:07 PST 2008

TFAT is there to prevent the problems you describe. I think your system
does NOT have TFAT enabled (because then the problems you describe won't
happen anymore). TFAT "transacts" all writes to the file system. It uses
a flag to indicate the entire write was succesful. Only with that flag
set to true will the data be used. If that flag is not set (because
power failed during the write) the data will be "rolled-back" / "not used".

I'd ask your OEM to generate an image with TFAT enabled for you.

To verify if TFAT is enabled for your store, lookup the correct profile
under the storage manager registry keys and check if "FormatAsTFAT" is
set to 1.

For instance:

[HKEY_LOCAL_MACHINE\System\StorageManager\Profiles\PCMCIA\Part01]
"Name"="PCMCIA/Compact Flash Device"
"Folder"="Main"
;Indicate this partition can store the hive registry and make it TFAT
"MountAsBootable"=dword:1
"MountAsRoot"=dword:1
"MountPermanent"=dword:1
"AutoFormat"=dword:1
"FormatTfat"=dword:1
"Flags"=dword:410034


Good luck,

Michel Verhagen, eMVP
Check out my blog: http://GuruCE.com/blog

GuruCE
http://GuruCE.com
Consultancy, training and development services.


Charlie wrote:
> I have an OEM generated Win CE 5.0 image with TFAT support. I think.
> The only reason I think I have TFAT is that my flash is sporadically
> becoming corrupt (no power loss that I know of), and I can only fix it
> from me CE hardware.
>
> Anyway, something is corrupting my flash rendering it unbootable. I
> do no fancy I/O - simple file writes from time to time, lot's of
> reads. The only way to fix the CF is to remove all the files (using
> the CE hardware) and re-write. Doing this seems to recover the file
> system and all is well for days/weeks until it happens again.
>
> Is there a way to detect when TFAT fails to write all of the data
> necessary and has to rollback the previous operation?
>
> Is there any special error handling I can implemnent to detect when
> TFAT goes south?

Re: Detecting TFAT file system corruption - possible? by Charlie

Charlie
Mon Feb 04 14:01:29 PST 2008

Michel ,

I am in contact with our OEM. We'll see what they say. At this
point, I am suspecting that TFAT was left out of the build somehow.
I have no storage manager utility on my platform desktop, and I have
no entries such as you describe.

Even so, I have read enough on what TFAT needs to be successful to
give me pause. One quote in particular from 2006 just bothers me:


"TFAT works if the underlying block device supports atomic write
operation (the data is fully updated or it is not changed).
If your flash block driver can lose data during a power down (ex: it
stores in RAM the contents of a block while it's erasing it) TFAT
cannot grant the integrity of your filesystem. "

There is another quote that TFAT was primarily directed at non-
removable storage which doesn't make a whole lot of sense in my book.
WinCE has a file system that cannot be accessed (safely) by Windows.
Huh? <note: anyone want to send me the APIs I'll write the utility>

It would seem to be a house built on a very shaky foundation, but this
is strictly personal perception.

Re: Detecting TFAT file system corruption - possible? by ctacke/>

ctacke/>
Mon Feb 04 14:32:15 PST 2008

> "TFAT works if the underlying block device supports atomic write
> operation (the data is fully updated or it is not changed).
> If your flash block driver can lose data during a power down (ex: it
> stores in RAM the contents of a block while it's erasing it) TFAT
> cannot grant the integrity of your filesystem. "

Why does this bother you? How would TFAT be able to protect these
situations where it thinks that the data has been written? It's been passed
down the the block driver and the block driver says it has, but it's
actually caching the write in RAM. TFAT has no way to know that happened
and understandabley considers the write to be committable.

> There is another quote that TFAT was primarily directed at non-
> removable storage which doesn't make a whole lot of sense in my book.
> WinCE has a file system that cannot be accessed (safely) by Windows.
> Huh? <note: anyone want to send me the APIs I'll write the utility>

TFAT is not FAT - it's a separate file system. There is no TFAT driver for
the desktop that I'm aware of. The source for it ships with PB, so if you
really want an XP driver for it, it could be done, but it's no small task.


--

Chris Tacke, eMVP
Join the Embedded Developer Community
http://community.opennetcf.com




Re: Detecting TFAT file system corruption - possible? by Michel

Michel
Mon Feb 04 18:17:23 PST 2008

Just to point out: you can safely READ TFAT from XP. You can even write
to it, but those writes will not be transacted (of course) and if you
create a new file on the drive in XP, that file will never be transacted
in the future, even when later writing to that file under CE (because
the required data structures are not created during creation on XP).

Note that CE 6.0 comes with exfat, and I believe Vista supports exfat
now (maybe even XP with an update, can't remember exactly).

Good luck,

Michel Verhagen, eMVP
Check out my blog: http://GuruCE.com/blog

GuruCE
http://GuruCE.com
Consultancy, training and development services.


<ctacke/> wrote:
>> "TFAT works if the underlying block device supports atomic write
>> operation (the data is fully updated or it is not changed).
>> If your flash block driver can lose data during a power down (ex: it
>> stores in RAM the contents of a block while it's erasing it) TFAT
>> cannot grant the integrity of your filesystem. "
>
> Why does this bother you? How would TFAT be able to protect these
> situations where it thinks that the data has been written? It's been passed
> down the the block driver and the block driver says it has, but it's
> actually caching the write in RAM. TFAT has no way to know that happened
> and understandabley considers the write to be committable.
>
>> There is another quote that TFAT was primarily directed at non-
>> removable storage which doesn't make a whole lot of sense in my book.
>> WinCE has a file system that cannot be accessed (safely) by Windows.
>> Huh? <note: anyone want to send me the APIs I'll write the utility>
>
> TFAT is not FAT - it's a separate file system. There is no TFAT driver for
> the desktop that I'm aware of. The source for it ships with PB, so if you
> really want an XP driver for it, it could be done, but it's no small task.
>
>

Re: Detecting TFAT file system corruption - possible? by Charlie

Charlie
Tue Feb 05 02:22:11 PST 2008

On Feb 4, 5:32 pm, "<ctacke/>" <ctacke[at]opennetcf[dot]com> wrote:
> > "TFAT works if the underlying block device supports atomic write
> > operation (the data is fully updated or it is not changed).
> > If your flash block driver can lose data during a power down (ex: it
> > stores in RAM the contents of a block while it's erasing it) TFAT
> > cannot grant the integrity of your filesystem. "
>
> Why does this bother you? How would TFAT be able to protect these
> situations where it thinks that the data has been written? It's been passed
> down the the block driver and the block driver says it has, but it's
> actually caching the write in RAM. TFAT has no way to know that happened
> and understandabley considers the write to be committable.
>

A fair statement and one I would agree with.... in my initial reading
of features to add
to our OEM generated image, I "missed" two caveats - the first is the
above, the second
below. My only observation is that the functionality is so dependent
on the block driver
operation that it should be declared much more boldly.


> > There is another quote that TFAT was primarily directed at non-
> > removable storage which doesn't make a whole lot of sense in my book.
> > WinCE has a file system that cannot be accessed (safely) by Windows.
> > Huh? <note: anyone want to send me the APIs I'll write the utility>
>
> TFAT is not FAT - it's a separate file system. There is no TFAT driver for
> the desktop that I'm aware of. The source for it ships with PB, so if you
> really want an XP driver for it, it could be done, but it's no small task.
>
> --
>
> Chris Tacke, eMVP

I understand the difference between file systems. However, no where
have I been able to find
the implied limitation - that tfat was intended for non-removable
media - in the documentation.
It is such a useful feature to have in your system, and it just begs
to be enabled. Yet this issue
is consistently asked in this group. To make matters worse, it
usually works - setting up someone who
has not read all of the documentation. And I mean all.

There are some subtle issues here of responsibility between Microsoft,
the OEM and the customer of the OEM system. I recognize that being
the end customer - it's all mine :). But I wish that in the technical
documentation, MS had pointed out the limitations a bit more clearly.
See my next post.


Perhaps this is just the nature of the environment. Some projects
> Join the Embedded Developer Communityhttp://community.opennetcf.com


Re: Detecting TFAT file system corruption - possible? by Charlie

Charlie
Tue Feb 05 02:47:46 PST 2008

On Feb 4, 9:17 pm, "Michel Verhagen (eMVP)" <mic...@nospam.box> wrote:
> Just to point out: you can safely READ TFAT from XP. You can even write
> to it, but those writes will not be transacted (of course) and if you
> create a new file on the drive in XP, that file will never be transacted
> in the future, even when later writing to that file under CE (because
> the required data structures are not created during creation on XP).
>
> Note that CE 6.0 comes with exfat, and I believe Vista supports exfat
> now (maybe even XP with an update, can't remember exactly).
>
> Good luck,
>
> Michel Verhagen, eMVP

Interesting, Michel I will look for a reference in the documentation.
Most of the comments I have seen indicate that MS is very hesitant to
say just what you said - which makes me a bit paranoid. My practical
experience validates that I can read from the CF and write - which is
what lulled me into a false sense of security.

Is there a way to detect when a file is not transacted? I've not seen
any reference in the documentation to this. I'd also like your
thoughts on how TFAT tracks this - by file? physical area?

The very fact that exfat exists tells me that MS recognized the gap of
needed functionality. Good for them.

-----------------------------------

Let me get back to the question I originally asked and then vectored
myself into a potential flame war - is there any way to detect or
check the TFAT file system for errors, recoverable or otherwise?


Appreciate the discussion... learning by doing.



Re: Detecting TFAT file system corruption - possible? by Henrik

Henrik
Wed Feb 06 00:06:28 PST 2008

On 5 Feb, 11:47, Charlie <cgil...@bravesw.com> wrote:
> Is there a way to detect when a file is not transacted? =A0I've not seen
> any reference in the documentation to this. =A0I'd also like your
> thoughts on how TFAT tracks this - by file? =A0physical area?

The short version is that it's the FAT table that is transacted. TFAT
uses two FAT tables and has code to compare and syncronize them if
they differ. TFAT has no option but to rely on the underlying block
driver to perform the block IO in a transaction friendly way.

> The very fact that exfat exists tells me that MS recognized the gap of
> needed functionality. =A0Good for them.

Well, it probably has more to do with capacity limitations with FAT
than anything else.

> Let me get back to the question I originally asked and then vectored
> myself into a potential flame war - is there any way to detect or
> check the TFAT file system for errors, recoverable or otherwise?

You can allways use scandisc/fatutil lib, but that's a pretty dull
tool when it comes to hunting down file system issues. I spent a
conciderable part of last year fixing file system stability issues and
we ended up writing our own utilities for doing file system and block
driver forensics.

Henrik Viklund
http://www.addlogic.se