former issues, revisited: FileSystemWatcher  
Author Message
M. Nicholas





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

My understanding of the FileSystemWatcher is that it can only watch a single directory, and either one file type or all file types (nothing in between). If that is true, I am looking at a different approach, if it is not true, perhaps someone could expand my limited knowledge.

In short, here is what I want to accomplish. I have big files being uploaded to our FTP server (located in our office), by various speed IP’s (including dial-up) to over 30 unique folders, and I want them swept into a central location. Obviously I have to be concerned about whether the files have completed their upload before they are moved. I was thinking about an app that has a timed event to check all of the files in each named directory for the “date last modified” property. If the modified property is more than 5 minutes older than DateTime.Now, move it to the central location.

This seems like a good alternative to FileSytemWatcher to me, but I would welcome any comments or suggestions.

PS. I know this all probably sounds very familiar. I was looking into solutions for this app a few months ago, but had surgery on my right arm and am just now able to type again. ;^(

Thanks,

Mike



Visual Basic9  
 
 
DMan1





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

Hello Mike,

Hope your arm is doing better

Your solution of scanning the directories and checking filestamps is a viable solution.  You can however setup multiple FileSystemWatcher Components to accomplish the same task....

Private Sub FileSystemWatcher_Changed(ByVal sender As System.Object, ByVal e As System.IO.FileSystemEventArgs) Handles FileSystemWatcher1.Changed, FileSystemWatcher2.Changed

'move file to central location

Microsoft.VisualBasic.FileCopy(e.FullPath, "YourDestinationPath")

'if file successfully copied delete original source file

If System.IO.File.Exists("DestinationFile") Then

Microsoft.VisualBasic.Kill(e.FullPath)

End If

 

End Sub



 
 
M. Nicholas





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

Dman1~

Interesting…I’ve done some research on that very topic (multiple FileSystemWatcher) and never found a definitive answer.

Tell me, do you see any adverse effects on creating say…25 or more instances I wonder if there would be any fallout.

The reason I ask is that we have at least 20 directories receiving 2 different file types (.dss and .wav) from our clients. And as I said before, they are big and can take several minutes to upload. Then, we have about 20 directories that we are receiving .doc (from our remote employees) that we want to be swept (for further processing) and later put back on the client’s sites.

I set up FileSystemWatcher once to watch the entire FTP site (because it will only watch one folder) and it was a real mess. The issue became that anytime we moved things around (maintenance, cleanup) it would trigger the FileSystemWatcher.

Under the scenario I just described, do you think the FileSystemWatcher is still the best way to go I honestly don’t know the answer.

I think using timed events may be a way to get very explicit control, but it would be a hell of a lot code (which is okay too).

I would love your (or anyone’s) thoughts.

Much thanks,

Mike


 
 
DMan1





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

Tell me, do you see any adverse effects on creating say…25 or more instances I wonder if there would be any fallout.

As always you need to consider system resources...and I have not tested nor researched the resource consupmtion of a single filesystemwatcher component...however I don't think your requirments are that huge anyway

The reason I ask is that we have at least 20 directories receiving 2 different file types (.dss and .wav) from our clients. And as I said before, they are big and can take several minutes to upload. Then, we have about 20 directories that we are receiving .doc (from our remote employees) that we want to be swept (for further processing) and later put back on the client’s sites.

Set up a watcher for the directories and not the file types....check the actual file type in the created/changed event and process those files with the extensions that you need to deal with...Also the filesystem watcher can be setup to monitor subdirectories also....in other words if any of your 20 directories are subdirectories of another then you only need a single filesystemwatcher for thos directories in the same tree

I set up FileSystemWatcher once to watch the entire FTP site (because it will only watch one folder) and it was a real mess. The issue became that anytime we moved things around (maintenance, cleanup) it would trigger the FileSystemWatcher.

Under the scenario I just described, do you think the FileSystemWatcher is still the best way to go I honestly don’t know the answer.

Yea, I would go the filesystemwatcher route....and be very wise in how they arfe setup and which events you monitor...as far as the maintenance ...that is going to be an issue no matter which route you take...in either case you can stop the service...

HTH



 
 
Anarchy





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

Hi Mike

I've ran into similar problems as you have, where I need multiple FileSystemWatchers and multiple file type to look for, have a look at these threads

http://forums.microsoft.com/MSDN/ShowPost.aspx PostID=620710&SiteID=1

http://forums.microsoft.com/MSDN/ShowPost.aspx PostID=620846&SiteID=1

As you have found, the FSW.created event occurs when the file STARTS being created, not when it had FINISHED been created, which is not much use. Even very small files take time to be written to disk, and if you try to move a file as soon as the created event happens, it will fail. I created a wrapper for FSW which tries (and will fail) to open a newly created file for writing to, sleeps for a few milliseconds and tries again, and raises an ENDCreated event when it works.

re multiple file types, as DMan suggests, set the FSW to look for all types, and simply ignore files you don't want. If you could organise your folders so the FSW can look in subfolders, you could possibly get away with only 1 watcher, although from what I've read, FSW uses virtually no system resources, so having many should be do-able as well.

I'm happy to share my FSW wrapper, just ask here and I'll post it. I'd be very interested in how you are getting on with your app. I write lots of little apps to do pretty much what you are doing, and used to run a app every n minutes and simply look for files. FSW is obviously a better way to go, but it took me a WEEK! to get it working how I wanted it to.


 
 
M. Nicholas





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

Anarchy~

Thanks for your input and the reading material.

My situation is complex, but I’ll try to describe it in as few words as possible. We have 20 clients that upload numerous voice files (.wav and .dss), each to their own FTP location on our server (located in our office). Because of the nature of the content, we are required to keep every client’s files separated (federal HIPAA requirements).

We also have 20 remote employees that we pass the client’s files to. They also have individual locations on our server. Our employees transcribe from the voice files and return to text documents to us, which we in-turn, move back to the client’s FTP location, although in a different sub-directory.

So we are keeping track of three different file types (wav, dss, doc), that are coming and going out of over 40 unique locations AND, except for while we are handling them in our office, can never be mixed together.

After careful consideration and much experimentation, I chose to go with a timed event that searches for specific file types and handles them one directory at a time. I will simply loop this small code through a list of every directory that will receive new files.

Mike


 
 
Anarchy





PostPosted: Visual Basic General, former issues, revisited: FileSystemWatcher Top

Hi Mike,

Sounds like you would need lots of FSWs if you went that route, which could get messy.

The timer approach should be easier, and is just as valid assuming you don't need an instant response every time a file arrives, which is unlikely.

good luck