Thursday, December 27, 2007

chkdsk disaster on W2k3 NTFS partition

In October of this year I ran Chkdsk on a volume on one of my W2k3 servers. I ran this utility because we had evidence of file or folder corruption on that particular user share. I was rewarded for my actions by chkdsk removing ALL the security permissions on every single folder. Needless to say I was a huge hit with the rest of the server support team who wound up helping me to replace all the security permissions on these folders so that folks could access their data. In four days we got this accomplished. We did our research and according to Microsoft knowledgebase articles this was something that happens to Windows 2000 servers. No where did we see where Windows 2003 servers with SP1 were supposed to be susceptible to this type of problem. We did find an obscure article that hinted that there was an NTFS related file that SP2 would update that would keep this from happening. Fast forward to December and this same server is now updated with SP2 and all of the Microsoft Updates that are available for Windows 2003 Standard Edition. We are in the midst of Christmas holidays and everyone other than us unlucky contractors are out doing other things than work. I figured this was a wonderful opportunity to let chkdsk do its magic and fix whatever data corruption that might be on the user share of this server. I run chkdsk with the /f switch so it dismounts the volume and off to the races it goes. The volume is close to 1.5 TB in size so it takes several hours to do. Phase 1 and 2 go along just fine with nothing being reported back to the screen as an error. Phase 3 checks the indexes and that is when I got the dreaded “replacing invalid security id with default security id for file (numbers). Needless to say I was not really excited about the prospect of having to manually replace the permissions on all these folders. I called my boss, told her the problem and she authorized me to call Microsoft and get paid support. I am hoping that they can identify the problem for us so that I can document it and make sure it never happens again on our hundreds of other servers and that they can provide me with a script from AD that will allow me to replace the permissions with the correct permissions from the command line. I will let you know what I find out when the dust settles and things are fixed.

No comments: