SRX

 View Only

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.



Expand all | Collapse all

Security logs to GUI/local/NSM/STRM/SYSLOG server

Jump to Best Answer
  • 1.  Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-18-2011 23:03

    Hi All

     

    I am very confused regarding the security logs on SRX. Could any one post the working example for the sending the security logs to GUI, LOCAL, NSM, STRM and syslog server.

     

    Looking forward for the response

     

    Thanks


    #logging


  • 2.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server
    Best Answer

    Posted 07-19-2011 02:23

    Explanation and examples attached for your reference


    Control Plane
       •    Logs generated by
                - User Process in particular flowd logs session info
                - Interactive commands, logs user typed cli commands
                - System daemons (like kmd, mgd, snmpd, etc)
        •    NSM can receive logs only from control plane / RE via DMI
        •    Support for forwarding to STRM

    Data Plane
       •    Traffic Logs including
        •                 - Session, IDP, UTM
        •    High-End SRX can generate up to 40K logs / Sec / SPU
        •    Limited Log forwarding Support for NSM
                - 10.0 (and above) High-End SRX and 9.6 (and above) Branch SRX:
                - Data logs can be forwarded to NSM via the control plane.
        •    Forwarded to 3’rd Party Syslog servers including STRM

    STRM Logging:
    1. STRM expects SRX logs in specific format:
        •    Control Plane Logs in Unstructured Syslog
        •    Data Plan Logs in Structured Syslog
    2. JUNOS 9.6 Data Logs generated by branch SRX’s cannot be parsed by STRM 
        •    Data Logs not formatted in expected Structured Syslog Format

    New User Configurable Option under [security log mode] for data plane logs
    1. Event
       - Send all traffic logs to RE
          set security log mode event
       - Recommended for forwarding data logs to NSM (High-end and Low-end SRX)
           Rate-limiting for high-end SRX to prevent flooding RE
           set security log event-rate <logs/s up to 1.5K>
    2. Stream
       - Data plane logs forwarded to third party syslog server / STRM
          set security log mode stream
       -  Logs forwarded in structured format
           set security log format sd-syslog
       - Recommended for forwarding data logs to STRM
         No Rate limiting
         Structured Format

    SRX LOG CONFIGURATION (STRM)

    CONTROL LOGS
    syslog {
      user * {
            any emergency
      }
      host 10.0.100.140 {
            any any;
            change-log none;
            interactive-commands none;
      }
    }

    DATA LOGS
    security {
      log {
         mode stream
      format sd-syslog;
      source-address 192.168.252.192
      stream security log {
      category all
      host {
          192.168.252.5
           port 514 
      }
      }
    }
    }

    1. Control Plane logs  can be forwarded with filtering capabilities
    2. Data Plane logs forwarded in
      - Structured Syslog format
      - Up to three streams supported on SRX

    SRX LOG CONFIGURATION (NSM)

    CONTROL LOGS
    syslog {
      file default-log-messages {
          any any;
          structured-data;
      }
    }

    DATA LOGS
    security {
      log {
         mode event
      format sd-syslog;
      event-rate <logs up to 1.5K >
    }
    }

    1. Control Plane logs sent via DMI
    2. Data logs are forwarded to the active RE
     - High End SRX (10.0 and above)
     - Branch SRX (9.6 and above)

    Caveats:

    1. Data logs cannot be filtered on device prior to forwarding to syslog server / STRM
    2. Data logs can be filtered prior to sending to NSM
      - Command to be used
         set system syslog file default-log-messages match
         Possible completions:
              <match>              Regular expression for lines to be logged


    thanks

    Raheel



  • 3.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-19-2011 11:59

    Hi Raheel

     

    Thanks for the great explaination. Just I need few clarification below:

     

    1- The configuraiton for logs send to NSM in control plane is just to create the below file but how to send the logs in this file to NSM?

     

    CONTROL LOGS
    syslog {
      file default-log-messages {
          any any;
          structured-data;
      }
    }

     

    2- What is the difference between structured and unstructure logs? Also control plan logs are always unstructured by default and data plan logs are strucured?

     

    3- For the local logs, what is the configuration in control and data plane?

     

    Looking forward for your response

     

    Thanks



  • 4.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-19-2011 14:55

    for (1)-

    • The is an xml-only command that streams a file to NSM, (<get-syslog-events>).  This  command when received by the RE sends the contents of the file to the requestor
    • there is one more stanza that is needed to setup NSM connection, [system service outbound-ssh]

    for (2)-

    This is an unstructured (traditional syslog) message:


    Apr 24 12:30:05  cs-loki3 RT_IDP: IDP_ATTACK_LOG_EVENT: IDP: at 1303673404, ANOMALY Attack log <64.1.2.1/48397->198.87.233.110/80> for TCP protocol and service HTTP application NONE by rule 3 of rulebase IPS in policy Recommended. attack: repeat=0, action=DROP, threat-severity=HIGH, name=HTTP:INVALID:MSNG-HTTP-VER, NAT <46.0.3.254:55870->0.0.0.0:0>, time-elapsed=0, inbytes=0, outbytes=0, inpackets=0, outpackets=0, intf:trust:fe-0/0/2.0->untrust:fe-0/0/3.0, packet-log-id: 0 and misc-message -

    This is an example of a structured syslog file.  Note how inside the “[]” brackets there is a key value pairing, which makes it easier for an automation system to parse out the attributes of the logs.


    <28>1 2011-07-19T21:51:03.624Z elza utmd 33838 WEBFILTER_URL_BLOCKED [junos@2636.1.1.1.2.36" target="_blank" rel="nofollow noopener noreferrer">junos@2636.1.1.1.2.36 source-address="192.168.1.109" source-port="39945" destination-address="192.168.2.3" destination-port="80" name="N/A" error-message="by other category" profile-name="UTM-WFCPA" object-name="192.168.2.3" pathname="/ss-eicar.com"] WebFilter: ACTION="URL Blocked" 192.168.1.109(39945)->192.168.2.3(80) CATEGORY="N/A" REASON="by other category" PROFILE="UTM-WFCPA"URL=192.168.2.3 OBJ=/ss-eicar.com

    for(3)-

     To setup configuration for local logs, you need to set:
    [security log mode event]  (dataplane logs sent to re)

    [system syslog file .....] (syslog setup to save logs to local file

     

    thanks

    Raheel

     

     

     



  • 5.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-20-2011 07:17

    I want to conquer: I don't think stream mode logging is only recommended for logging towards STRM. Please correct me if I am wrong.

     

    It makes a lot of sense to use this for NSM logging as well (supported since NSM 2011.1). It will remove a lot of burden from your CPU. There are some things to keep in mind though when doing this. Like it can't be done over fxp0.

     

    The logging situation on SRX is a big bad mess. It needs to be cleaned up by Juniper ASAP.

     

     



  • 6.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-21-2011 16:53

    replies enclosed with inital [RA]

     

    ------

    cryptochrome wrote:

     

    I want to conquer: I don't think stream mode logging is only recommended for logging towards STRM. Please correct me if I am wrong.

     

    [RA] stream mode is available and recommended for any long term log collection solutions. This can be STRM or any other SIEM solution or a windows or linux syslog collector.

     

    It makes a lot of sense to use this for NSM logging as well (supported since NSM 2011.1). It will remove a lot of burden from your CPU. There are some things to keep in mind though when doing this. Like it can't be done over fxp0.

     

    [RA] NSM logging will not reduce the CPU burden as it still saves files on the local filesystem.  NSM does offer a solution for capturing, viewing logs.

    The logging situation on SRX is a big bad mess. It needs to be cleaned up by Juniper ASAP.

    [RA] Could you please be more specific like what is not working?  Is it because we are not offering something, or is it because there is confusion about what options to use.

     

    ------

     

    thanks,

    Raheel



  • 7.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-27-2011 05:35

    Rahel, it's a mess because no one really knows how to use it and what to use. I have had several tickets open with JTAC and even phone conversations with Juniper SEs, and everybody seems to have a different opinion.

     

    Also, it is complete nonsense to have to configure separate interfaces just for logging because fxp0 can't handle the logs if you have to use stream mode logging.

     

    To sum it all up: I should just be able to tell the machine to log to a log destination and not have to worry about anything. It should just work. But it doesn't.

     

    As for high CPU load: If this has still not been fixed, then that's another point towards "it's a mess".

     

     



  • 8.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-28-2011 03:26

    Thanks for the valuable feedback – I sense the real issue maybe:


    (1) documents/notes do not have clear instructions with how to use the logging system and why there are recommendations (e.g. not using the fxp0 interface).
      - could you please share the logging document which you are currently using it?

    (2) also, It seems like that customers are not setting up the system correctly if they are having high CPU and using stream mode. This implies they are trying to route the logs out to some destination that only reachable from the fxp0.  Or they mistakenly have setup event mode.

    I agree with your point about just configuring a destination and the system should be able to handle the rest, but also not sure if that is the best practice for majority vendors to fulfill their needs.  I would pass this feedback to the right folks in Juniper to do more thinking in this regard. 

    Do you have details of your setup so I can see what can be done for your need? also do share the JTAC case-id etc. details.

    thanks,

    raheel 



  • 9.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 07-30-2011 03:50

    Hi Raheel

     

    Thanks a lot for the valuable information. Its really missing in the Juniper documents and KB. I just want to know:

     

    1- If we want to send the security logs to any file OR syslog server under [system syslog ...] then we have to make mode event? Am I right in understanding?

    set security log mode event

     

    2- By default the mode is event or stream on srx?

     

    3- To send the security logs to STRM/NSM (2011)/Syslog Server in stream mode (through data plane), we have to make two things. Am I right in understanding?

     

    a- The mode to stream

    b- STRM/NSM/Syslog should be defined under [securty log...]

     

    4- You replied above, For sending the logs from RE to NSM we just need to define the file under the [system syslog] and mode event. We dont need to define NSM as syslog server. But then Why in event mode we need to define STRM as syslog server?

     

    Thanks



  • 10.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 08-03-2011 16:56
    @aeroplane:
    please find enclosed my replies with initials.
    --------------
    aeroplane wrote:
    Hi Raheel

    Thanks a lot for the valuable information. Its really missing in the Juniper documents and KB. I just want to know:

    1- If we want to send the security logs to any file OR syslog server under [system syslog ...] then we have to make mode event? Am I right in understanding?

    set security log mode event

    [RA] Yes, if you want to use the [system syslog] features, you need to set [security log mode event]


    2- By default the mode is event or stream on srx?
    [RA] The default is event on SRX100, SRX210, SRX240, SRX650
    The default is stream on SRX1400, SRX3000 and SRX5000


    3- To send the security logs to STRM/NSM (2011)/Syslog Server in stream mode (through data plane), we have to make two things. Am I right in understanding?

    a- The mode to stream

    b- STRM/NSM/Syslog should be defined under [securty log...]
    [RA] The 2 settings as identified are required


    4- You replied above, For sending the logs from RE to NSM we just need to define the file under the [system syslog] and mode event. We dont need to define NSM as syslog server. But then Why in event mode we need to define STRM as syslog server?

    [RA] NSM has 2 log collection capabilities.  If the SRX is being managed by NSM, then it should take care of all the settings required to setup NSM DMI based logging.  

    NSM uses a different log transfer mechanism which is an xml data transfer of the logs.  This is why NSM does not need to be configured as a collector.

    Thanks
    -------------
    hope this helps
    thanks,
    Raheel



  • 11.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 08-04-2011 13:34

    Thanks Raheel.

     

     



  • 12.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 11-30-2011 11:47

    Thanks for a good thread. Im getting my hands dirty with the SRX's for the first time and I have to agree with a few things said out here:

     

    - The management interface thing is a real problem. As it stands I have to send logging traffic down the internal reth interface to me that's a fail.

     

    - To leave users without a real log viewer is not fair. Once the FW is setup I'm in the logs all day.

     

     



  • 13.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 12-02-2011 07:57

    @Jickfoo wrote:

    Thanks for a good thread. Im getting my hands dirty with the SRX's for the first time and I have to agree with a few things said out here:

     

    - The management interface thing is a real problem. As it stands I have to send logging traffic down the internal reth interface to me that's a fail.

     

    - To leave users without a real log viewer is not fair. Once the FW is setup I'm in the logs all day.

     

     


     

    +1



  • 14.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 08-01-2011 08:27

    Rahel,

     

    I appreciate your help. 

     

    I have used many different documentations, all available from the Juniper support website and KB. And that's the point. There is not ONE document that describes the logging in it's entirety, there are many documents. And they don't reference each other, so things get confusing quickly. 

     

    And as you can see by the questions others ask, there is confusion about this topic. Just search the forum for SRX logging, and you will find countless threads, all about the same topic and same questions.

     

    You are saying that maybe customers are not setting up their systems correctly. And exactly that is the point. Why are people setting it up incorrectly? Because they don't know it any better and are confused. 

     

    There should be ONE document that describes the logging mechanisms, explains the differences between event mode and stream mode, gives advise an which method to use under which circumstances and how to do it correctly. And it should go deeper than just providing config examples, the document should make people understand.

     

    If Juniper offers such a vast array of logging options AND makes changes to them with basically every new Junos version, then Juniper should make it clear to customers how these work. Right now, as a customer you only have scattered documentation, spread across multiple documents and multiple versions, and they end up coming to the forums, just to find out that people here ask the same questions.

     

    Hence: This is a mess.

     

     



  • 15.  RE: Security logs to GUI/local/NSM/STRM/SYSLOG server

    Posted 08-03-2011 13:08

    Hi Raheel

     

    Could you please reply on this?