Intrusion Prevention

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.



Expand all | Collapse all

IDP-200: "nic link is down" after Policy Push

Jump to Best Answer
  • 1.  IDP-200: "nic link is down" after Policy Push

    Posted 01-29-2010 03:30

    Hi all,

     

    I have a problem with our idp-200 system with version 4.1r4:

     

    We do an "Scheduled Attack an Device Update" via NSM 2007.3r5 every day at 10am.

     

    But before the new policy is loaded on the sensor, we get an error, that the nic link of our interfaces is going down and the nicBypass becomes active. After a second the nic link comes up again an nicBypass is deactivated.

     

    This link flapping before the policy load results in a frequently failover of our upstream firewall. This failover is a disruptive behavior and I search for a solution.

     

    I already tried KB10865, but it doesn't affect this behavior.

     

    Have anyone any idea?

     

    Thanks, Markus

     

     

     



  • 2.  RE: IDP-200: "nic link is down" after Policy Push
    Best Answer

    Posted 03-09-2010 16:40

    Markus:

     

    The way the system is designed, the kernel sends a reset signal to the NIC watchdog timer every second. If the watchdog timer doesn't get a reset from the kernel in three seconds, it will fail open, resulting in NIC bypass. This scenario may be happening when you are pushing the policy, if it is a big policy relative to the IDP-200, resulting in high CPU utilization.

     

    This is addressed with the 5.0+ version of the IDP code, as the user and data planes have been seperated. However, for single core IDP's like the IDP-200, this does not completely resolve the issue, although it does make it less likely to happen. A workaround (with any IDP version) for single core systems is to increase the watchdog timer to a longer interval than the default of one second. Ten seconds may be good.

     

    For information on setting the watchdog timer, please reference KB11077.

     

    cheers



  • 3.  RE: IDP-200: "nic link is down" after Policy Push

    Posted 04-04-2010 16:10

    Thanks

     



  • 4.  RE: IDP-200: "nic link is down" after Policy Push

    Posted 03-13-2010 04:10

    This KB  is not found



  • 5.  RE: IDP-200: "nic link is down" after Policy Push

    Posted 04-05-2010 14:07

    No problem! Glad I could help.



  • 6.  RE: IDP-200: "nic link is down" after Policy Push

    Posted 03-25-2010 13:20

    Ok, I see why you cannot see it, it is marked as internal only. I am working on permission to make it available to you.

     

    regards



  • 7.  RE: IDP-200: "nic link is down" after Policy Push

    Posted 03-25-2010 17:29

    Ok, here is what is in the KB:

     

    The Internal NICBypass helps prevent network outage due to IDP entering a hang state or experiencing high CPU utilization when the IDP device is in inline mode (transparent mode only). The NICBypass employs WatchDog Timers to achieve the functionality.

    The nicBypass script which is located in /usr/idp/device/bin/ prevents the interfaces from going into bypass mode when the IDP is working normally by resetting the watchdog timer. It also restores the interfaces to normal mode if they had gone into Bypass mode. The nicBypass script performs this check and takes the action (if required) every "loopInterval" seconds (the script 'sleeps' for "loopInterval" seconds).

    These two parameters can be configured via the idp.cfg file on the IDP Sensor as shown below:

    nicBypass.watchdogInterval 10 (secs)
    nicBypass.loopInterval 3 (secs)


    The duration specified against watchdogInterval controls how long the IDP will stay in the hang state before the activation of Bypass on the interface pairs or before the interfaces are forced into bypass mode. Practically, watchdogInterval specifies the value from which the timer will count down to zero. The command "bypassStatus" lets us know the status of the interface pairs as shown below:
    BYPASS STATUS: Tue Mar 4 09:32:18 PST 2008

    Status for nicBypass daemon      : on
    Watchdog timer setting(sec)      : 10
    Watchdog loop reset interval(sec): 3

    NIC         Setting   Current State   WD Time Left(ms)
    ---------------------------------------------------------------
    eth2,eth3   ENABLED     Normal          9800
    eth4,eth5   disabled    Normal          (wdt inactive)
    eth6,eth7   disabled    Normal          (wdt inactive)
    eth8,eth9   disabled    Normal          (wdt inactive)



    The watchdog time is decremented every few ms, as can be seen under the column "WD Time Left(ms)". The moment the "watchdog time left" becomes 0, the interface pair is forced to go into Bypass mode. In normal working conditions the nicBypass script would reset the value of watchdog to the configured value after every "loopInterval" seconds. This is how it prevents the interface pairs from going into bypass mode. However, when the IDP is hung which happens due to high CPU utilization, the nicBypass script will not be able to run and the watchdog time keeps decrementing without being reset and eventually becomes 0 causing the Bypass to be activated on interface pairs configured to go to bypass mode.