All Aspect servers (2oo3) services restarted automatically, resulting in Operator Station hung up..
The operators on the platform reported that they were unable to operate any plant or view plant changes from any of the operator stations in the control room. I began by checking the System log and noticed that a number of services on all of the Aspect servers were Inoperative but returning to operation. After approx. 30 mins from the time the initial problem was observed all Aspect Server services were Operational and healthy and the operator stations functionality had returned to normal.
Can you please provide a possible explanation for the issue described above?
Notes: I have checked the following,
- disk space on all aspect servers and there are no issues there.
- all Aspect Servers for Crash Dumps and there were none relating to this issue (nothing for the past few months).
- Windows and Application And Services logs and can not see any issues reported.
The [Workplace Structure]Web System Workplace:System Event List + Operator Message List may contain evidence. Secure their content by copying to some persistent storage, ex MS Excel. Submit with support case.
System 800xA does not generate any crash dumps unless you beforehand turned process memory dumping on via the Configuration Wizard. However, any exception thrown by a malfunctioning process will always be logged to C:\ProgramData\ABB\Process Portal A\Log\... Make a ZIP archive of this folder on all nodes as input to the support case.
I have a couple of suspicions, but there is no reason throwing any guesses around until having examined the aforementioned data.