Re: Zero out Free Disk Space

We've been having weekly script club meetings at work where anyone who is interested gets together in a conference room and we all work on scripts together so we all have a chance to learn new techniques while solving our real-world problems at the same time. This week I developed a script to mimic the behavior of the SysInternals utility SDelete. The script will be used to reclaim thin-provisioned space from our SAN, and I wanted to develop it during script club to use as an example of how to use the classes in the System.IO namespace to get better IO performance from PowerShell.

One of my co-workers shared the script from the meeting with Don Jones and he was nice enough to post it on his blog along with a few comments about the script.

Don's blog doesn't seem to support public comments, so I wanted to take a few minutes to respond to his comments here. I've reproduced his comments (in italics below) along with my responses:

I wish the comment-based help was a bit more complete, but I love that they're using it!

This script was written in a "class" setting and I was focusing more on solving the task at hand than producing a share-able script. The parameter block and help content were actually thrown in at the very end of the session just as a quick "finishing touch" and were never even run. I agree that well documented scripts are important and I try to write more complete help on my scripts when I don't have an audience watching me code :)

They're overriding $PercentFree to always be 0.5 - not sure if that's on purpose

This is a bug due to the way I'm trying to teach programming at work. I've been encouraging people to use variables for things even when they are just starting a script and are still in "explore" mode, with the idea that this makes it easy to turn the script into a function just by deleting the explicit variable declarations and moving them up to a parameter block.

Yay! Error handling!

I really wanted to take the opportunity to highlight the proper way to deal with external resources (ie file handles, network ports, etc) for my team. I see a lot of people who either don't bother to close things or don't put the close statements in a finally block and end up leaking resources.

Yay! Variable-replacement-within-double-quotes instead of concatenation!

Really brilliant technique of creating an empty file and then deleting it - clever thinking.

Can't really take credit for this one, just replicating the way sdelete works :)

Kind of a lot of variables. Like, why put 64KB into $ArraySize, and then only use it once? Why not use use the literal 64KB? Doing so would make the script a bit more concise and possibly easier to read.

This again comes from my programming style. At the time I thought there was potential we might want to make $ArraySize a parameter so I built it from the start so that it would be easy to convert down the line if we decided to go that way. Also I wanted to make it very easy to modify so we could do some performance testing to determine what the optimal block size to write was. In general I try to avoid hard coding any values into the meat of my scripts to make them easier to modify if the needs change down the road.

After Further Review

The script that Don was kind enough to review was written live during class and I never even got to the point of saving the script to a file in my editor, much less cleaning it up to make it "enterprise class" or "share worthy". Some other rough areas I have noticed in it after having time to review it are:

• It should probably test to make sure the file doesn't exist already so it doesn't clobber existing data
• It should test to make sure the $Root path/volume exists
• It should validate that $PercentFree is between 0 and 1
• The delete command should probably be moved up into the finally block to ensure the file gets deleted
• I also would consider putting the whole thing in a process block, adding a "Name" alias to $Root, and enabling "ValueFromPipelineByPropertyName" so you could do:
  
  gwmi Win32_Volume | Write-ZeroesToFreeSpace

The "Real" Script

I've sort of posted these last two blog entries in reverse order, as my previous post is the cleaned up version of the script that addresses Don's comments as well as my additional comments posted above.

Thanks again Don for taking the time to review our little script and give us the opportunity to share a little bit of our development process with the community!

A PowerShell Alternative to SDelete

Problem Background

Many storage appliances support thin provisioning by not storing large blocks of zeros on disk, thus saving expensive physical disks for actual data. One of the challenges of this feature is that Windows (or most file systems really) does not zero out space that was used by files that have been deleted. This means that over time as files are written an deleted, the amount of physical disk that is saved by thin provisioning grows smaller.

One popular solution (really the only one we've been able to find) is to use the SysInternals utility SDelete's -z switch to write zeros to all the free space, thus allowing the storage appliance to use all that empty space for something else. The challenge with sdelete is that it only supports volumes mounted as drive letters, not volumes that are mounted to folders. Another concern with SDelete is that it appears to fill up the entire disk, if only for that brief second between finishing writing the file out and deleting the file. We were concerned with using this tool at work on a workload like SQL that might not take kindly to a disk filling up.

Our storage administrator tried to solve this problem using the standard PowerShell techniques of redirecting null strings to files and found the performance to be somewhat lacking. I don't want to get into a detailed analysis of PowerShell IO characteristics (mostly because I haven't spent much time researching how PowerShell does IO), but instead just post that the solution is to turn to the System.IO .Net namespace and perform the IO operations the "hard way".

The Script

Here is the script I came up with using a FileStream to write out the zero file. We informally compared a couple of runs between this script and sdelete and saw roughly the same performance.

Write-ZeroesToFreeSpace.ps1

001 002 003 004 005 006 007 008 009 010 011 012 013 014 015 016 017 018 019 020 021 022 023 024 025 026 027 028 029 030 031 032 033 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 052 053 054 055 056 057 058 059 060 061 062 063 064 065 066 067 068 069 070 071 072 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 099 100 101 102

<# .SYNOPSIS  Writes a large file full of zeroes to a volume in order to allow a storage  appliance to reclaim unused space. .DESCRIPTION  Creates a file called ThinSAN.tmp on the specified volume that fills the  volume up to leave only the percent free value (default is 5%) with zeroes.  This allows a storage appliance that is thin provisioned to mark that drive  space as unused and reclaim the space on the physical disks.   .PARAMETER Root  The folder to create the zeroed out file in. This can be a drive root (c:\)  or a mounted folder (m:\mounteddisk). This must be the root of the mounted  volume, it cannot be an arbitrary folder within a volume.   .PARAMETER PercentFree  A float representing the percentage of total volume space to leave free. The  default is .05 (5%) .EXAMPLE  PS> Write-ZeroesToFreeSpace -Root "c:\"    This will create a file of all zeroes called c:\ThinSAN.tmp that will fill the  c drive up to 95% of its capacity.   .EXAMPLE  PS> Write-ZeroesToFreeSpace -Root "c:\MountPoints\Volume1" -PercentFree .1    This will create a file of all zeroes called  c:\MountPoints\Volume1\ThinSAN.tmp that will fill up the volume that is  mounted to c:\MountPoints\Volume1 to 90% of its capacity. .EXAMPLE  PS> Get-WmiObject Win32_Volume -filter "drivetype=3" | Write-ZeroesToFreeSpace    This will get a list of all local disks (type=3) and fill each one up to 95%  of their capacity with zeroes.   .NOTES  You must be running as a user that has permissions to write to the root of the  volume you are running this script against. This requires elevated privileges  using the default Windows permissions on the C drive. #> param(   [Parameter(Mandatory=$true,ValueFromPipelineByPropertyName=$true)]   [ValidateNotNullOrEmpty()]   [Alias("Name")]   $Root,   [Parameter(Mandatory=$false)]   [ValidateRange(0,1)]   $PercentFree =.05 ) process{   #Convert the $Root value to a valid WMI filter string   $FixedRoot = ($Root.Trim("\") -replace "\\","\\") + "\\"   $FileName = "ThinSAN.tmp"   $FilePath = Join-Path $Root $FileName      #Check and make sure the file doesn't already exist so we don't clobber someone's data   if( (Test-Path $FilePath) ) {     Write-Error -Message "The file $FilePath already exists, please delete the file and try again"   } else {     #Get a reference to the volume so we can calculate the desired file size later     $Volume = gwmi win32_volume -filter "name='$FixedRoot'"     if($Volume) {       #I have not tested for the optimum IO size ($ArraySize), 64kb is what sdelete.exe uses       $ArraySize = 64kb       #Calculate the amount of space to leave on the disk       $SpaceToLeave = $Volume.Capacity * $PercentFree       #Calculate the file size needed to leave the desired amount of space       $FileSize = $Volume.FreeSpace - $SpacetoLeave       #Create an array of zeroes to write to disk       $ZeroArray = new-object byte[]($ArraySize)              #Open a file stream to our file       $Stream = [io.File]::OpenWrite($FilePath)       #Start a try/finally block so we don't leak file handles if any exceptions occur       try {         #Keep track of how much data we've written to the file         $CurFileSize = 0         while($CurFileSize -lt $FileSize) {           #Write the entire zero array buffer out to the file stream           $Stream.Write($ZeroArray,0, $ZeroArray.Length)           #Increment our file size by the amount of data written to disk           $CurFileSize += $ZeroArray.Length         }       } finally {         #always close our file stream, even if an exception occurred         if($Stream) {           $Stream.Close()         }         #always delete the file if we created it, even if an exception occurred         if( (Test-Path $FilePath) ) {           del $FilePath         }       }     } else {       Write-Error "Unable to locate a volume mounted at $Root"     }   } }