September 8, 2012; 6:00 PM
University IT Services has been working with Microsoft and NetApp, our data storage vendor, to determine the root cause of the problems the University has been having with our MS Exchange email system. At this time it appears a software glitch in an until recently unused feature of the NetApp system was causing the problem. This caused the NetApp Storage Area Network to report the size of the file systems incorrectly to MS Exchange. This in turn caused MS Exchange to fill the file systems, causing the recent system crashes and data corruption. Now that IT Services has determined the root cause of the problems, we have disabled this feature and stabilized the system.
I want to thank the University IT Systems team for their dedication throughout the last few days; they have put in many long hours to restore this vital service for the University. I also want to that the technical support teams from Microsoft and NetApp, whose assistance in resolving this problem have been invaluable. And I want to thank the University community for your patience and support as we’ve worked to resolve this problem.
A good deal of work remains to be done. We still have to restore Wednesday’s email for about 500 users. We’re also running on a temporary server, so we need to build and add additional servers to the Microsoft Exchange server cluster for redundancy to improve reliability and performance. And we may run in to additional snags as we complete repairs to this key communication tool. In the meantime, please be prepared for additional email outages with little or no warning until the system is fully repaired.
Steve Landry, CIO