Tip for Win 7 Users with many many files on network

notarat

notarat

2[H]4U
Joined
Mar 28, 2010
Messages
2,501
I manage about 10TB of storage (not much in terms of space) that contains around 6.8 million files and 3.2 million directories that reside on over a dozen servers geographically located around the country

I'm the only one at my work who tracks the shared drive usage for duplicates, unauthorized files, overly large files, old files, etc.

Since the provider of our storage is not "Us", we're restricted to scanning the shares remotely, an we were never given any tools with which to perform scans.

I've seen posts from people who wanted to know how to go about gathering the information so they too can track these kinds of information regarding their network drives, so I thought I'd share my Windows Power Shell 2.0 script that gathers the information.

The script I use differs from the one posted below only in that I stripped out the stuff I had in there that personalized it to my environment.

Measure-Command {get-childitem -recurse | where-object {$_.PSIscontainer -ne $true} | foreach-object {'"'+ $_.name + '" , "' + $_.fullname + '" , "' + $_.length + '" , "' + $_.Extension + '" , "' + $_.CreationTime + '" , "' + $_.LastAccessTime + '" , "' + $_.LastWriteTime + '"' } >c:\outputfiles\hmm.txt}

Measure-Command {....} is not the scan itself, but will record how long it takes the scan to complete. You can remove that portion if you want...

If you have Power Shell 2.0 loaded:

1- Start the ISE
2 - In the Command Line Frame, enter your drive to be scanned (Ex: d: <enter> )
3 - Paste the code in the code window
4 - Click the Run button.

You'll obviously want to change the path c:\Scans\ to wherever you store your results, and name the file however you want.

You can import that directly into MS Access, or open the txt file from Excel

Caveat: I use Office 2010 (work requirement) so I don't have a problem with spreadsheets larger than 65535 rows. However, I use MS Access since I can create more powerful queries in it.


EDITED: Added Quotes around the values to clear up potential issues when importing into MS Access or MS Excel.
 
Last edited:
Running this at home (Because I'm not allowed to post actual results from Work equipment)

Performance seems to be pretty good overall regardless of environment (remote share or local drive...but I can't post numbers for work)

I would imagine that if you have a Windows 2008 Server for your file shares, you could store the script on the server side and run it remotely.

The output of the command, when run on my gaming machine's 2TB Caviar Black "D:" Drive (containing 281,537 files) is listed below.

Days : 0
Hours : 0
Minutes : 2
Seconds : 8
Milliseconds : 153
Ticks : 1281531899
TotalDays : 0.00148325451273148
TotalHours : 0.0355981083055556
TotalMinutes : 2.13588649833333
TotalSeconds : 128.1531899
TotalMilliseconds : 128153.1899


Equates to roughly 1.2 million files scanned in less than 10 minutes locally. (now there's no reason not to organize your pr0n collection properly.)

Couple things to keep in mind is that

1 - Power Shell 3.0 is supposed to be up to 40% faster, specifically when running the Get-ChildItem cmdlet against remote file shares due to changes in the way it's handled in the 3.0 environment.

(Note: If anyone is familiar with the old Windows utility LS.EXE, that program is about 35% faster than this script (Running in Ps 2.0) so installing and running Ps 3.0 would actually make this script faster than that compiled .exe file)

2 - Your network's latency will affect your scans. Don't expect this speed on a file share located in Denver if you're in New York.
 
It's too bad you don't have access to those servers. We don't bother with powershell for storage tasks as we just use the reporting built in to FSRM. Duplicate files, least accessed files, large files, etc are all reports built in to the system and we just email them out either automatically when a quota limit is hit or we feel one person or department is taking up too much space.

Powershell 3.0 really pissed me off with the variable security crap.
 
The links are to nice programs, but not possible where I work. (*sigh*)

On a different note:
It's my fault for not being clear. When I created the script (for work) my intent here was to post it so people who don't have commercial software for these tasks on their home network could use it.
 
You must log in or register to reply here.
Back
Top