Improving performance for a virtual Aspect Server
I am trying to improve te performance of our PG2 Graphics when they are called up for the first time on a client. We have noticed that in our case the Aspect Server is the bottleneck. Every time we are calling up a new display, which isn't already cached on the client, the CPU load of the Aspect Server is increasing to 50%. This is done by the AFWADServer.exe process. The Aspect server is a virtual server with 2 CPU cores. When we give the Aspect Server not 2 but 4 CPU cores than the CPU load goes to 25% instead of 50%. It looks like the AFWADServer.exe Process can only handle 1 CPU Core. Are there any other options to give the AFWADServer.exe Process more CPU performance?
We are running with 800xA Version 5.1 FP4.
Firstly, have you gone through all the other methods of fixing slow display callups ...
- Reduce the number of subscriptions on a display to < around 2000
- Ensure the network is running optimally and at 1 Gig minimum all the way from the server to the client.
- Remove late binding from the graphics
- Check the diagnostics for the grahics and investigate where the timing delays are actually occuring.
You are correct, many of the afw Services appear to be single threaded and dont seem to be able to use more than one core at a time. However, I'm surprised that you see CPU load as being the bottleneck as this isnt usually the case on the Aspect Servers. Remember, the AfWAdServer process is only actually the bottleneck if every other resource in the system ( network, disks, memory) etc has spare capacity.
- What type of physical machine are you using to host the virtual machines ?
- What is the clock speed of the CPU ?
- Are you using vSphere and how many shares of the physical server have you given to the Aspect Server VM ? If the VM doesnt have access to enough of the physical machine's resources then it will show as using 100% resources in the VM
- How much memory ?
- Windows 32 bit or x64 ?
I know of one customer using SSD's ( solid state disks ) and I believe that the results of this are pretty good, but I havent seen the system myself.
I fully agree with Rob's comments and suspect that the slow displays are built some special way or with very large number of items.
- How many OPC items are contained in the displays?
- Is performance equally bad in displays stretching from "small number of items" to "large number of items"?
- There is a *huge* performance drop when using late bound properties* compared to early bound.
- If possible, use divide & conquer technique to isolate the slowest components of a slow display. Begin with copying the source display and edit the copy. Remove parts of the display and check performance. Continue with this until you have found out what gives biggest contribution to callup time.
- Please submit screen captures from the Diagnostic window (right click in graphic to call the menu; use <SHIFT>-right click in operator mode/user to force presentation of the Diagnostic item that otherwise is hidden from pure operator accounts).
*) **Only** use late binding when there is absolutely no other alternative, and even then use it with care. If you in the graphics builder can navigate to the property and select it, use that in favor of writing fancy "LateBoundPropertyRef(...)" functions. All late bound functions claim a lot of CPU, not only property refs so LateBoundViewRef(....) is equally bad.
For official product support, file a case with your regional ABB support team.
As such, we have not faced any issues in the proformance of 800xA system just because it is running on a virtual machine. As long as the configuration guidelines outlined by virtualization manual are met, there shouldnt be a problem.
To start with, first check if this is a common problem on all the graphics or just few of the graphics and then try to zero-in on the issue. If this is a generic issue of all graphics, you need to suspect the network bandwidth. Else, As said in above replies, It could be the way graphics are built that has an issue. Late bound properties, Animations ( moving conveyors, rotating things etc), graphic wrappers etc have a huge impact on performance. You can also attach a screenshot of the graphics that you have issues with.
First thanks a lot for the comprehensive answers. We have done some checks on our system after the answers but we cannot find any suspicious things. As I said, we are looking to improve the performance of the system when we call up a graphic display on a client for the first time. The system is working fine when the Graphic is already in the clients’ cache. I know it is one of the characteristics of the 800xA system that it can take a long time before a graphic is initially loaded on a client but we want to get it as quick as possible.
Some answers on your questions:
-Number of subscriptions: between 100 and 1000, so far under 2000
-Performance is equal for graphics with a small number of items and with a big number of items
-Network is running optimally, everything is 1 Ghz. I did a filecopy test, as subscribed in one of the manuals, with a big file of 600+mb and this copy took only a few seconds
-We don’t have any late bindings on graphics
-We don’t have any aspect view wrappers on graphics
-We are running with Windows Server 2008 R2 SP1 64-bit (servers) and Windows 7 Professional SP1 64-bit (clients)
-Every server and client has 8000+ mb memory
-As far as I can see there are no DNS errors
The system I am taking about contains three aspect servers (redundant), three connectivity servers(redundant) and two domain controllers (redundant). We have 20+ clients attached to the system and we are running a big 800xA application with almost 20 PLC’s and over 10.000 license tags.
More information about our configuration is in the attached file. I am open to further suggestions regarding our system setup!