Missing Alarm & Events in IM from AC400C
Hi,
We are missingalarm/events from AC400C in our IM Server. We can see that collection of system alarm and events, audit events are working but we are missing the process events from AC400C.
Note! the alarm and events are working at the connectivity servers/ op clients / alarm lists/ event lists and trend data also works.
Basic history works but the link to long term is not working correctly and
when we connect to our IM db with datadirect and query the IM database we can't detect any process events(alarm and events from AC400C).
Anyone have any tip where we should search to solve the problem?
best regards
Mattias Lundqvist
Höganäs Sweden AB
Answers
Even if its not likely, you could suffer from a filter preventing storage of AC400 A&E.
Please search the IM Configuration User's Guide on filtering (basically you create an alarm and event list configuration aspect on a distinct object with a distinct name, the settings made there control what event categories the IM will be logging - does filter exist, but without 'Process(MB300)' category selected?).
Without filtering, all events should be logged in IM.
Its not that easy to fault trace; but I would start with:
1) Check IM Oracle Instance - any table space at or near 100%?
2) Check %HS_LOG%\hsMesssage-log-some-thing.log for abnormal messages
Number 2) is not that easy to perform, but will tell if IM event collection engine has run into a problem, e.g. an IM message log not too keen on attribute changes, e.g. if someone has added custom event attributes and subsequently changed their data types from one to another. This might require the category to be adjusted.
Also, you may see evidence of duplicate event categories causing issues. ABB has published product bulletins on this. Search ABB Library/SolutionsBank.
I believe you should file an official support case incl a zip of the %HS_LOG% folder and the alert file from Oracle (can't remember the path, but C:\Oracle\...\*alert*.log is the start and end of it). Google can help with any errors you may find therein.
1)We have this table space in our IM server, and there are room left to expand for every table to expand. se picture above
2) We don't find the logs you specify in your comment Stefan?
The log files are there.
Open a Windows Explorer and type %HS_LOG% in the address bar.
The explorer will jump to "C:\ProgramData\ABB\IM\History\Log" something.
This folder has numerous of log files with many versions. Many end with a number from 0 to 3. 0 is first. 1 is second as 0 becomes with certain size. When 3 is full, 1 will be overwritten (keeping 0 - reflecting the output after reboot)
There will be one log covering the IM's message logging activity; hsEventLogCollector.log
Try looking for errors, especially ones containing MB300 as category names are echoed as they are discovered, e.g. "Process(MB300)" - the process alarm category for MB300.
AEClient.cpp @ 1620 UTC 2014/09/22 14:03:50 PID=10288 | Type = Condition CatID = 792944717 CatName = Process(MB300)
...
I've shortened the output to make it better fit here on the thread:
Type = Condition CatID = 792944717 CatName = Process(MB300)
AttrId = 2300324373 AttrName = ResponsibleNode DataType = 8
AttrId = 2360567163 AttrName = SourceName DataType = 8
AttrId = 2421555061 AttrName = SourceProperty DataType = 8
AttrId = 2512072767 AttrName = SimpleCondition DataType = 11
AttrId = 2518782741 AttrName = Flags DataType = 19
AttrId = 2913184950 AttrName = HidingMaskCondition DataType = 8
AttrId = 2943588505 AttrName = Class DataType = 3
AttrId = 2950174305 AttrName = SourceGUID DataType = 8
AttrId = 3141503966 AttrName = ResponsibilitySection DataType = 8
AttrId = 3289886969 AttrName = Shelved DataType = 11
AttrId = 3365828977 AttrName = ServiceGroupId DataType = 8
AttrId = 3421200744 AttrName = AlarmState DataType = 8
AttrId = 3448271509 AttrName = HidingRuleId DataType = 8
AttrId = 3576953285 AttrName = ShelvingTime DataType = 7
AttrId = 3719197140 AttrName = ProcessSection DataType = 3
AttrId = 3927042709 AttrName = UncertainTimeTag DataType = 11
AttrId = 4013300096 AttrName = ObjectName DataType = 8
AttrId = 4041431376 AttrName = HidingRuleCondition DataType = 8
AttrId = 4144796539 AttrName = AlarmChange DataType = 8
AttrId = 47211389 AttrName = HidingMaskId DataType = 8
AttrId = 297910293 AttrName = Comment DataType = 8
AttrId = 376987265 AttrName = HidingMaskName DataType = 8
AttrId = 496402258 AttrName = ShelvingMode DataType = 8
AttrId = 565394243 AttrName = AckHighTime DataType = 3
AttrId = 911758175 AttrName = PrintoutBlocked DataType = 11
AttrId = 923458247 AttrName = ShelvingReason DataType = 8
AttrId = 1003727619 AttrName = AckLowTime DataType = 3
AttrId = 1019691562 AttrName = BlockedRepetitive DataType = 11
AttrId = 1107836271 AttrName = ShelfExpireTime DataType = 7
AttrId = 1141578067 AttrName = ObjectDescription DataType = 8
AttrId = 1161313119 AttrName = Responsibility DataType = 8
AttrId = 1342868174 AttrName = Hidden DataType = 11
AttrId = 1451181918 AttrName = AudibleAlarm DataType = 11
AttrId = 1638445637 AttrName = AutoDisabled DataType = 11
AttrId = 1642473083 AttrName = ResponsibleUser DataType = 8
AttrId = 1728002418 AttrName = PriorityLevel DataType = 3
AttrId = 1823505001 AttrName = ResponsibilityId DataType = 8
AttrId = 1957068387 AttrName = ResponsibleUserId DataType = 8
AttrId = 2139858547 AttrName = ActiveChangeTime DataType = 7
SetMsgTypeEx2 (Verify) was successfull.
I find the same type of message from hsEventLogCollector.log on our server like the example you showed.
I can attach the logs if you want to take a look at it.
AEClient.cpp @ 1620 UTC 2016/06/29 07:31:26 PID=10032 | Type = Simple CatID = 869895306 CatName = PcEvent(MB300)
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2357063470 AttrName = LotID DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2360567163 AttrName = SourceName DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2421555061 AttrName = SourceProperty DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2512072767 AttrName = SimpleCondition DataType = 11
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2879430574 AttrName = UnitInfo DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2943588505 AttrName = Class DataType = 3
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 2950174305 AttrName = SourceGUID DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 3719197140 AttrName = ProcessSection DataType = 3
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 3927042709 AttrName = UncertainTimeTag DataType = 11
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 4013300096 AttrName = ObjectName DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 4126406855 AttrName = BatchID DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 18726298 AttrName = ProcedurePath DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 911758175 AttrName = PrintoutBlocked DataType = 11
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 1019691562 AttrName = BlockedRepetitive DataType = 11
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 1027454529 AttrName = UnitID DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 1134895512 AttrName = CampaignID DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 1141578067 AttrName = ObjectDescription DataType = 8
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 1451181918 AttrName = AudibleAlarm DataType = 11
AEClient.cpp @ 1670 UTC 2016/06/29 07:31:26 PID=10032 | AttrId = 1728002418 AttrName = PriorityLevel DataType = 3
AEClient.cpp @ 1881 UTC 2016/06/29 07:31:26 PID=10032 | SetMsgTypeEx2 (Verify) was successfull.
I a
lso changed the Alarm list to look at IM Event list:Alarm and Event List Configuration in Library structure and I see this, see picture below
Hmm what does this mean...? something fails...
Spoke with mutual friend (GF) and agree with him about the Temp tablespace looking odd
Ensure the Next clause fit within the auto-extend maximum.
The C:\oracle\diag\rdbms\adva\adva\trace\alert_adva.log should be clean from errors.
About sync issues; ensure all Log Configurations properly reference a Log Template in aspect system. Use Find Tool to search for Aspect Category = Log Configuration, drag the results into the Consistency Checker and have them all checked + possibly corrected. A non-existent template could cause IM sync issues.
Or use SQL:
SQL> select tablespace_name, bytes, autoextensible, increment_by, maxbytes from dba_data_files;
TABLESPACE_NAME BYTES AUT INCREMENT_BY MAXBYTES
------------------------------ ---------- --- ------------ ----------
SYSTEM 603979776 YES 8192 805306368
SYSAUX 536870912 YES 8192 805306368
UNDO 469762048 YES 8192 603979776
USERS 5242880 YES 128 104857600
HS_CONFIG 67108864 YES 8192 335544320
HS_ICONFIG 67108864 YES 8192 335544320
HS_REPORTS 67108864 YES 8192 335544320
HS_RESTORED 67108864 YES 8192 335544320
INFORM_HS_RUNTIME 1610612736 YES 8192 1.0486E+10
HS_INDEXES 2415919104 YES 8192 1.0486E+10
HS_PDL 67108864 YES 8192 335544320
TABLESPACE_NAME BYTES AUT INCREMENT_BY MAXBYTES
HS_IPDL 67108864 YES 8192 335544320
12 rows selected.
SQL> select tablespace_name, bytes, autoextensible, increment_by, maxbytes from dba_temp_files;
TABLESPACE_NAME BYTES AUT INCREMENT_BY MAXBYTES
------------------------------ ---------- --- ------------ ----------
TEMP 1207959552 YES 8192 1476395008
Add new comment