CN2 blocked and not getting unblocked
Hope you are doing good.
Sir, I am facing this port disabled network storm detected error on my CN2 conituously since 19 nov 2016 as can be seen in attached image ( Ethernet 2 ). I tried a lot of methods of port unblocking from fault tracer like clearing logs and unblocking ports. But this ports disabled is not going away. I also replaced the secondary controller with new controller and when I switched over it was giving this same error of the same previous date. and when I also got successfull ping(on blocked port) from the replaced controller outside the network when i powered it up separately.
Now something has happened and whenver connect CN2 cable to controller its used memory starts increasing and after some time it crashes and switches over to secondary, this happened 3 times yesterday as shown in attached image ( Used memory)
I also remove all cables from the switch and connected only connectivity server and CN2 of the affected controller but still no affect
I also attached controller logs. Please suggest any other method to remove port blocking and diagnosing this issue.
Thanks and Best Regards,
Voted best answer
Before controller firmware 5.1.1-3 a *dual* storm/loop risk putting one of the Ethernet ports in a *permanent* blocked state.
A controller reset may be required to recover the blocked port (remember, a forced failover or reset of the secondary is not the same as a reset).
As mentioned, the RNRP Utility can only unblock ports in Windows, but with this dual storm bug, the RNRP service in Windows needs to be restarted. A rollup with a patch for RNRP in Windows is available on ABB Library.
The RCU link ensure memory is copied from primary to secondary before it light up the Dual LED. Resetting/replacing the secondary is futile if fault is in primary's state/memory. The faulty state is copied to the secondary (which holds a true memory copy of the primary).
A dual storm followed by a permanently blocked port may be resolved by a warmstart of RNRP in the controller. Eg make a dummy change, eg increase Max No Of Remote Areas, download. The blocked port may recover by this. Then revert setting and download again. IMPORTANT: this workaround MUST be made using local download from an engineering station ON THE SAME RNRP Area as the controller.
As little as 13 looped telegrams may trigger loop detection/protection.
Ensure your control networks are designed to prevent loops. Be extra careful if opting for ring or mesh structure requiring a L2 redundancy protocol like STP, RSTP, etc.
In 5.1.1-3 and later, loop protection is disabled leaving only loop detection and storm detection + protection active. As of then only storms (=packet/second regardless looped or not) will result in a blocked port. The limit is 800 packets/second except for PM891 that will wait with blocking until 1600 packets/second reaches any controller port. All Ethernet ports are storm protected, i.e. also including CEX communication units, eg CI867.
The controller log output is distinct: NSP = Network Storm Protection
RNRP can not handle node number exceeding 500. 172.16.80.200/255.255.252 is OK, but 172.17.15.35 (255.255.252.0) is too high (512+256+35=803). RNRP will not go live on an Ethernet interface with that second address.
Ensure that both CN1 and CN2 are not connected to same VLAN in logical switch OR same physical switch.
could you share the network configuration used for the project?
As observed in controllerlog:
I 1979-12-31 00:00:00.548 Network Interface 172.16.80.200 (255.255.252.0 ) : 00-00-23-1f-ca-66
I 1979-12-31 00:00:00.568 Network Interface 172.17.15.35 (255.255.252.0 ) : 00-00-23-1f-ca-67
I 1979-12-31 00:00:00.582 Backup Interface 172.16.82.200 (255.255.252.0 ) (default rule)
I 1979-12-31 00:00:00.584 Backup Interface 172.17.17.35 (255.255.252.0 ) (default rule)
Node IP address is not set properly to behave as well, CN2 configured to use explicit IP (node address=803)where in CN1 configured to use implicit (node=200)
node address should be same for both interface of both primary and secondary controllers
So set the IP address properly.
Always while setting IP address in controller be sure that first to perform restore it to factory settings so if only MAC ID already used in the network then it may cause problems
. Make wure you have the same Firmware installed on both CPU's with Control Builder. One at a time.
. Use IP Config tool to do a Restore Factory Setting on both CPU's.
. Check for proper Hardware, and RCU Link cables, and CPU possitions according to the manufacturer recommendation.
. Power off the Secundary CPU.
. Again, with IP Config tool, Re-configure the Primary Interface IP on the Primary CPU (172.16.80.200) and use the advanced mode to Set Backup IP Addresses, use the option "Obtain an IP address by using default rule" for both Ethernet 1 and 2.
. Restart Primary CPU.
. Re-connect power to Secundary CPU and wait for CPU sinchronization until DUAL led lights.
. Check network configuration in Control Builder for Ethernet 1 (172.16.80.200) and Ethernet 2 (172.17.80.200) configuation.
. Download your project and check again.
. Test with RNRP Utility Option 5 to Check response of every node IP Address of your System.