AC 800F redundancy failure
We've got a redundant AC 800F station (Freelance 2016 RU3) which lost its redundancy a few days ago after working without issues for years.
IP1 is primary, IP2 was secondary and is now in quite a strange state: On diagnostic page, IP1 shows
CPU Type: PM 803F (16 MB) as Primary
Current CPU Load: 46%
Redundancy State: Not Sync (0%)
... which is to be expected. IP2 lacks the redundancy line:
CPU Type: PM 803F (16 MB), No redundancy
Current CPU Load: 1%
On the web interface under boot config, IP2 still showed the correct project name and configuration, and no errors. I then tried rebooting (cold start), without any substantial change. LED-wise IP1 is normal (Power and Run/Stop green, prim/sec amber), while IP2 only has Power and Run/Stop green, others off.
Afterwards, I tried reflashing the firmware of IP2 using Settings. The boot configuration after this is the following.
Main IP-Addr: 172.16.1.1 [255.255.240.0]
2nd IP-Addr: 172.16.129.1 [255.255.240.0]
Red IP-Addr: 172.16.1.2
Station No: 2
Project Name: "<our project>", Version 8
ObjDir Size: 6000
GlobVar Size: 64 KByte
RedMem Size: 100 KByte
Main IP-Addr: 172.16.1.2 [255.255.240.0]
2nd IP-Addr: 172.16.129.0 [255.255.240.0]
Station No: 0
Project Name: "none", Version 0
ObjDir Size: 1000
GlobVar Size: 32 KByte
What you do see however is constant data traffic on E2/Red-Link.
I connected to the diagnostic interface in the hope of seeing any suspicious messages, but to no avail. IP2 just boots normally as if it didn't know what redundancy was. IP1 has no messages whatsoever.
The only hint I found are event log entries concerning REDBOOT, but I cannot make sense of them. They seem to appear once after IP2 boots.
74 REDT : Event 0x0000503D in REDBOOT. Line 380 20.12.19 13:45:06.012
69 REDT : Event 0xFFFF0026 in REDBOOT. Line 374 20.12.19 13:37:48.806
I've tried exchanging IP2's E2 ethernet module, but there's no change.
I'd be grateful about any hints on where to look.
Best regards and happy holidays