question about CreateFile and FILE_FLAG_WRITE_THROUGH  
Author Message
George2





PostPosted: Visual C++ Language, question about CreateFile and FILE_FLAG_WRITE_THROUGH Top

Hello everyone,


Suppose I have a file created with FILE_ATTRIBUTE_NORMAL attribute. I think it means attribute FILE_FLAG_WRITE_THROUGH is not enabled. Is my understanding correct

And I think if I do not enable the FILE_FLAG_WRITE_THROUGH attribute, even if flush operation (function call) will not *truly* dump data to disk. Is my understanding also correct

So, my question is, when will the data be dumped to disk -- only when we close the file


thanks in advance,
George



Visual C++10  
 
 
einaros





PostPosted: Visual C++ Language, question about CreateFile and FILE_FLAG_WRITE_THROUGH Top

Suppose I have a file created with FILE_ATTRIBUTE_NORMAL attribute. I think it means attribute FILE_FLAG_WRITE_THROUGH is not enabled. Is my understanding correct

That's correct.

And I think if I do not enable the FILE_FLAG_WRITE_THROUGH attribute, even if flush operation (function call) will not *truly* dump data to disk. Is my understanding also correct

When you flush the file by use of FlushFileBuffers, it will be flushed to the disk from the system buffers. AFAIK, it doesn't guarantee that the disk cache is written, however, but you won't really notice that (except if the power fails, and the drive doesn't manage to spend its dying moments flushing the internal cache, as I seem to recall that some do).

Note that this isn't an API or SDK forum, though. I believe you've been told before to route such questions to the proper newsgroup



 
 
George2





PostPosted: Visual C++ Language, question about CreateFile and FILE_FLAG_WRITE_THROUGH Top

Thanks einaros!

Do you mean even if I only invoke write API (without flush) when in FILE_FLAG_WRITE_THROUGH mode, the OS buffer and the physical content on disk are the same

Suppose I have a file created with FILE_ATTRIBUTE_NORMAL attribute. I think it means attribute FILE_FLAG_WRITE_THROUGH is not enabled. Is my understanding correct

That's correct.

And I think if I do not enable the FILE_FLAG_WRITE_THROUGH attribute, even if flush operation (function call) will not *truly* dump data to disk. Is my understanding also correct

When you flush the file by use of FlushFileBuffers, it will be flushed to the disk from the system buffers. AFAIK, it doesn't guarantee that the disk cache is written, however, but you won't really notice that (except if the power fails, and the drive doesn't manage to spend its dying moments flushing the internal cache, as I seem to recall that some do).

Note that this isn't an API or SDK forum, though. I believe you've been told before to route such questions to the proper newsgroup

I do not fully agree with this point. I think whether or not FILE_FLAG_WRITE_THROUGH flsg is setted, FlushFileBuffers will always make cache flushed on disk.

Another question is, if FILE_FLAG_WRITE_THROUGH mode is not enabled, how will data lose (I think if FILE_FLAG_WRITE_THROUGH is not enabled, we could invoke flush, and whether or not FILE_FLAG_WRITE_THROUGH is enebled, flush will definitely write OS buffer data to external disk. Is my understanding correct )

BTW: which usergroup do you recommend


regards,
George


 
 
einaros





PostPosted: Visual C++ Language, question about CreateFile and FILE_FLAG_WRITE_THROUGH Top

Do you mean even if I only invoke write API (without flush) when in FILE_FLAG_WRITE_THROUGH mode, the OS buffer and the physical content on disk are the same

When you use FILE_FLAG_WRITE_THROUGH, the content will be written to a OS buffer, but flushed immediately to the drive. If you specify FILE_FLAG_NO_BUFFERING as well, the OS buffer will be omitted, and even the drive cache will be attempted surpassed. There's however no guarantee that the data won't be cached on the drive.

To answer your question: yes, the content will be the same. Either the OS buffer and the drive cache will be the same, the OS buffer and the physical drive data, or both.

I do not fully agree with this point. I think whether or not FILE_FLAG_WRITE_THROUGH flsg is setted, FlushFileBuffers will always make cache flushed on disk.

The point is that there are two or more buffers / caches in action here. Generally speaking, there's an OS buffer, and a drive cache. The combination of FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING will attempt to avoid both these temporary stores. Calls to FlushFileBuffers, on the other hand, will only flush the OS buffer.

Once the data is on the drive, you cannot really tell if it's the cache or the actual data you're seeing. This means that if one application writes some data and flushes the OS buffer, another application which opens the same file will see the changes. The use of FlushFileBuffers alone, FILE_FLAG_WRITE_THROUGH alone or the combination of FILE_FLAG_NO_BUFFERING & FILE_FLAG_WRITE_THROUGH will all do the same in that regard.

Another question is, if FILE_FLAG_WRITE_THROUGH mode is not enabled, how will data lose

If FILE_FLAG_WRITE_THROUGH isn't enabled, you could loose data if the OS or application crashes. I don't know under which specific circumstances this could happen if only the application crashes, as the kernel may still find its way to flush the buffer to disk.

(I think if FILE_FLAG_WRITE_THROUGH is not enabled, we could invoke flush, and whether or not FILE_FLAG_WRITE_THROUGH is enebled, flush will definitely write OS buffer data to external disk. Is my understanding correct )

FILE_FLAG_WRITE_THROUGH doesn't affect FlushFileBuffers, nor does FlushFileBuffers affect FILE_FLAG_WRITE_THROUGH.

FlushFileBuffers *will* flush the OS buffers if called. FILE_FLAG_WRITE_THROUGH *will* cause a flush to happen at every write.

BTW: which usergroup do you recommend

You could try http://msdn.microsoft.com/newsgroups/default.aspx dg=microsoft.public.win32.programmer.kernel&lang=en&cr=US. Those guys should be able to clear up the part about the application crash aswell.



 
 
George2





PostPosted: Visual C++ Language, question about CreateFile and FILE_FLAG_WRITE_THROUGH Top

Thanks einaros!

Do you mean even if I only invoke write API (without flush) when in FILE_FLAG_WRITE_THROUGH mode, the OS buffer and the physical content on disk are the same

When you use FILE_FLAG_WRITE_THROUGH, the content will be written to a OS buffer, but flushed immediately to the drive. If you specify FILE_FLAG_NO_BUFFERING as well, the OS buffer will be omitted, and even the drive cache will be attempted surpassed. There's however no guarantee that the data won't be cached on the drive.

To answer your question: yes, the content will be the same. Either the OS buffer and the drive cache will be the same, the OS buffer and the physical drive data, or both.

I do not fully agree with this point. I think whether or not FILE_FLAG_WRITE_THROUGH flsg is setted, FlushFileBuffers will always make cache flushed on disk.

The point is that there are two or more buffers / caches in action here. Generally speaking, there's an OS buffer, and a drive cache. The combination of FILE_FLAG_WRITE_THROUGH and FILE_FLAG_NO_BUFFERING will attempt to avoid both these temporary stores. Calls to FlushFileBuffers, on the other hand, will only flush the OS buffer.

Once the data is on the drive, you cannot really tell if it's the cache or the actual data you're seeing. This means that if one application writes some data and flushes the OS buffer, another application which opens the same file will see the changes. The use of FlushFileBuffers alone, FILE_FLAG_WRITE_THROUGH alone or the combination of FILE_FLAG_NO_BUFFERING & FILE_FLAG_WRITE_THROUGH will all do the same in that regard.

Another question is, if FILE_FLAG_WRITE_THROUGH mode is not enabled, how will data lose

If FILE_FLAG_WRITE_THROUGH isn't enabled, you could loose data if the OS or application crashes. I don't know under which specific circumstances this could happen if only the application crashes, as the kernel may still find its way to flush the buffer to disk.

(I think if FILE_FLAG_WRITE_THROUGH is not enabled, we could invoke flush, and whether or not FILE_FLAG_WRITE_THROUGH is enebled, flush will definitely write OS buffer data to external disk. Is my understanding correct )

FILE_FLAG_WRITE_THROUGH doesn't affect FlushFileBuffers, nor does FlushFileBuffers affect FILE_FLAG_WRITE_THROUGH.

FlushFileBuffers *will* flush the OS buffers if called. FILE_FLAG_WRITE_THROUGH *will* cause a flush to happen at every write.

BTW: which usergroup do you recommend

You could try http://msdn.microsoft.com/newsgroups/default.aspx dg=microsoft.public.win32.programmer.kernel&lang=en&cr=US. Those guys should be able to clear up the part about the application crash aswell.

I want to confirm that you mean whether or not FILE_FLAG_WRITE_THROUGH is set, when we invoke flush (FileFlushBuffers), the content of the disk and the content of cache should always be the same, right

If it is, why there will be data lost issues (since data integrity is ensured by flush operation)

BTW: the forum you recommended is very helpful.

regards,

George