AfwDsOPCSurrogate fails to communicate between Server and third party client.
Hi,
I have a 800xa 5.1 rev e system . It has been Observed that over time , communication between 800XA server and 3rd Party client fails . Both has been communicated using dcom . It has been observed that memory usage of AfwDsOPCSurrogate.exe over time , exceeds 1.8 gb during communication loss and also physical memory reaches 77% . After restarting of servers , communication becomes healthy . This failure of communication happens on every 5th or 6th day.
System consist of 2 servers , 4 clients , 1 mis server . Its a domain environment.
I have attached applog viewer statistics of AfwDsOPCSurrogate during communication failure and after restart.
Please suggest some solution.
I have a 800xa 5.1 rev e system . It has been Observed that over time , communication between 800XA server and 3rd Party client fails . Both has been communicated using dcom . It has been observed that memory usage of AfwDsOPCSurrogate.exe over time , exceeds 1.8 gb during communication loss and also physical memory reaches 77% . After restarting of servers , communication becomes healthy . This failure of communication happens on every 5th or 6th day.
System consist of 2 servers , 4 clients , 1 mis server . Its a domain environment.
I have attached applog viewer statistics of AfwDsOPCSurrogate during communication failure and after restart.
Please suggest some solution.
Asked 4 months ago
Answers
0
Please file a support case with your regional ABB support center!
I will assist you with troubleshooting this once the case has been escalated to the SE L3 Operations Support Team.
In the meanwhile, enable the OPC Communicator Common log to level 3, click Get Messages and enable Save to file...
The log file will grow with the 3rd party client's activity - eventually filling the disk, but if the surrogate runs out of RAM, I deem it less likely to occur. Just keep a watch on the output. The logging can be initiated and output collected remotely from an arbitrary node of the system.
And additionally, periodically (at least two to three times during the "lifetime") call the OPC DA Connector Statistics operation on the DA Connector part of the Connection Service Group with this GUID {2D34134E-F973-4AF4-B411-5EBED2CEC124} where the surrogate has connected (if the service group is redundant, be sure to select the one having the surrogate connected; it will be listed among the clients in the top section of the output of said operation).
We also need more executions of the already submitted OPC Communicator Statistics operation, run periodically as well; throughout the lifetime of the surrogate.
Depending on the 3rd party client's modus operandi, the surrogate may leak memory on behalf of the client. E.g.:
- starting an advise based subscription but failing to accept OnDataChange callbacks when data arrives from the source OPC server (the surrogate server may buffer until it runs out of RAM).
- calling asynchronous operations without taking care of the OnWriteComplete, etc. callbacks.
- indefinitely disconnecting "unclean" and then shortly reconnect again
- indefinitely adding OPC groups, items
Its not possible to tell based only on the information from the files you submitted with this post. Unfortunately, there are many possibilities for a memory leak.
The results from the suggested logging and output from said operations will likely bring the details we need to figure this one out.
Do you have Audit Trail's AuditEvent_OperatorAction category enabled? If yes, consider exhonerating the surrogate (and 3rd party clients behind) from audit by setting the registry key [HKLM\Software\SYSWOW6432Node\ABB\AFW\SystemModules\OPCToolKit\AfwDsOPCSurrogat
e\1.0-0\private]InhibitAudit to 1. A 32-bit machine have a slightly different path in registry (remove the SYSWOW6432Node part). I might be slightly off typing the registry key, however, you will be able to find it, I'm sure.
I will assist you with troubleshooting this once the case has been escalated to the SE L3 Operations Support Team.
In the meanwhile, enable the OPC Communicator Common log to level 3, click Get Messages and enable Save to file...
The log file will grow with the 3rd party client's activity - eventually filling the disk, but if the surrogate runs out of RAM, I deem it less likely to occur. Just keep a watch on the output. The logging can be initiated and output collected remotely from an arbitrary node of the system.
And additionally, periodically (at least two to three times during the "lifetime") call the OPC DA Connector Statistics operation on the DA Connector part of the Connection Service Group with this GUID {2D34134E-F973-4AF4-B411-5EBED2CEC124} where the surrogate has connected (if the service group is redundant, be sure to select the one having the surrogate connected; it will be listed among the clients in the top section of the output of said operation).
We also need more executions of the already submitted OPC Communicator Statistics operation, run periodically as well; throughout the lifetime of the surrogate.
Depending on the 3rd party client's modus operandi, the surrogate may leak memory on behalf of the client. E.g.:
- starting an advise based subscription but failing to accept OnDataChange callbacks when data arrives from the source OPC server (the surrogate server may buffer until it runs out of RAM).
- calling asynchronous operations without taking care of the OnWriteComplete, etc. callbacks.
- indefinitely disconnecting "unclean" and then shortly reconnect again
- indefinitely adding OPC groups, items
Its not possible to tell based only on the information from the files you submitted with this post. Unfortunately, there are many possibilities for a memory leak.
The results from the suggested logging and output from said operations will likely bring the details we need to figure this one out.
Do you have Audit Trail's AuditEvent_OperatorAction category enabled? If yes, consider exhonerating the surrogate (and 3rd party clients behind) from audit by setting the registry key [HKLM\Software\SYSWOW6432Node\ABB\AFW\SystemModules\OPCToolKit\AfwDsOPCSurrogat
e\1.0-0\private]InhibitAudit to 1. A 32-bit machine have a slightly different path in registry (remove the SYSWOW6432Node part). I might be slightly off typing the registry key, however, you will be able to find it, I'm sure.
Add new comment