SRX

Hi All

I am very confused regarding the security logs on SRX. Could any one post the working example for the sending the security logs to GUI, LOCAL, NSM, STRM and syslog server.

Looking forward for the response

Thanks

Explanation and examples attached for your reference

Control Plane
   •    Logs generated by
            - User Process in particular flowd logs session info
            - Interactive commands, logs user typed cli commands
            - System daemons (like kmd, mgd, snmpd, etc)
    •    NSM can receive logs only from control plane / RE via DMI
    •    Support for forwarding to STRM

Data Plane
   •    Traffic Logs including
    •                 - Session, IDP, UTM
    •    High-End SRX can generate up to 40K logs / Sec / SPU
    •    Limited Log forwarding Support for NSM
            - 10.0 (and above) High-End SRX and 9.6 (and above) Branch SRX:
            - Data logs can be forwarded to NSM via the control plane.
    •    Forwarded to 3’rd Party Syslog servers including STRM

STRM Logging:
1. STRM expects SRX logs in specific format:
    •    Control Plane Logs in Unstructured Syslog
    •    Data Plan Logs in Structured Syslog
2. JUNOS 9.6 Data Logs generated by branch SRX’s cannot be parsed by STRM 
    •    Data Logs not formatted in expected Structured Syslog Format

New User Configurable Option under [security log mode] for data plane logs
1. Event
   - Send all traffic logs to RE
      set security log mode event
   - Recommended for forwarding data logs to NSM (High-end and Low-end SRX)
       Rate-limiting for high-end SRX to prevent flooding RE
       set security log event-rate <logs/s up to 1.5K>
2. Stream
   - Data plane logs forwarded to third party syslog server / STRM
      set security log mode stream
   -  Logs forwarded in structured format
       set security log format sd-syslog
   - Recommended for forwarding data logs to STRM
     No Rate limiting
     Structured Format

SRX LOG CONFIGURATION (STRM)

CONTROL LOGS
syslog {
  user * {
        any emergency
  }
  host 10.0.100.140 {
        any any;
        change-log none;
        interactive-commands none;
  }
}

DATA LOGS
security {
  log {
     mode stream
  format sd-syslog;
  source-address 192.168.252.192
  stream security log {
  category all
  host {
      192.168.252.5
       port 514 
  }
  }
}
}

1. Control Plane logs  can be forwarded with filtering capabilities
2. Data Plane logs forwarded in
  - Structured Syslog format
  - Up to three streams supported on SRX

SRX LOG CONFIGURATION (NSM)

CONTROL LOGS
syslog {
  file default-log-messages {
      any any;
      structured-data;
  }
}

DATA LOGS
security {
  log {
     mode event
  format sd-syslog;
  event-rate <logs up to 1.5K >
}
}

1. Control Plane logs sent via DMI
2. Data logs are forwarded to the active RE
 - High End SRX (10.0 and above)
 - Branch SRX (9.6 and above)

Caveats:

1. Data logs cannot be filtered on device prior to forwarding to syslog server / STRM
2. Data logs can be filtered prior to sending to NSM
  - Command to be used
     set system syslog file default-log-messages match
     Possible completions:
          <match>              Regular expression for lines to be logged

thanks

Raheel

Hi Raheel

Thanks for the great explaination. Just I need few clarification below:

1- The configuraiton for logs send to NSM in control plane is just to create the below file but how to send the logs in this file to NSM?

CONTROL LOGS
syslog {
  file default-log-messages {
      any any;
      structured-data;
  }
}

2- What is the difference between structured and unstructure logs? Also control plan logs are always unstructured by default and data plan logs are strucured?

3- For the local logs, what is the configuration in control and data plane?

Looking forward for your response

Thanks

for (1)-

• The is an xml-only command that streams a file to NSM, (<get-syslog-events>).  This  command when received by the RE sends the contents of the file to the requestor
• there is one more stanza that is needed to setup NSM connection, [system service outbound-ssh]

for (2)-

This is an unstructured (traditional syslog) message:

Apr 24 12:30:05  cs-loki3 RT_IDP: IDP_ATTACK_LOG_EVENT: IDP: at 1303673404, ANOMALY Attack log <64.1.2.1/48397->198.87.233.110/80> for TCP protocol and service HTTP application NONE by rule 3 of rulebase IPS in policy Recommended. attack: repeat=0, action=DROP, threat-severity=HIGH, name=HTTP:INVALID:MSNG-HTTP-VER, NAT <46.0.3.254:55870->0.0.0.0:0>, time-elapsed=0, inbytes=0, outbytes=0, inpackets=0, outpackets=0, intf:trust:fe-0/0/2.0->untrust:fe-0/0/3.0, packet-log-id: 0 and misc-message -

This is an example of a structured syslog file.  Note how inside the “[]” brackets there is a key value pairing, which makes it easier for an automation system to parse out the attributes of the logs.

<28>1 2011-07-19T21:51:03.624Z elza utmd 33838 WEBFILTER_URL_BLOCKED [junos@2636.1.1.1.2.36" target="_blank" rel="nofollow noopener noreferrer">junos@2636.1.1.1.2.36 source-address="192.168.1.109" source-port="39945" destination-address="192.168.2.3" destination-port="80" name="N/A" error-message="by other category" profile-name="UTM-WFCPA" object-name="192.168.2.3" pathname="/ss-eicar.com"] WebFilter: ACTION="URL Blocked" 192.168.1.109(39945)->192.168.2.3(80) CATEGORY="N/A" REASON="by other category" PROFILE="UTM-WFCPA"URL=192.168.2.3 OBJ=/ss-eicar.com

for(3)-

 To setup configuration for local logs, you need to set:
[security log mode event]  (dataplane logs sent to re)

[system syslog file .....] (syslog setup to save logs to local file

thanks

Raheel

I want to conquer: I don't think stream mode logging is only recommended for logging towards STRM. Please correct me if I am wrong.

It makes a lot of sense to use this for NSM logging as well (supported since NSM 2011.1). It will remove a lot of burden from your CPU. There are some things to keep in mind though when doing this. Like it can't be done over fxp0.

The logging situation on SRX is a big bad mess. It needs to be cleaned up by Juniper ASAP.

replies enclosed with inital [RA]

------

cryptochrome wrote:

I want to conquer: I don't think stream mode logging is only recommended for logging towards STRM. Please correct me if I am wrong.

[RA] stream mode is available and recommended for any long term log collection solutions. This can be STRM or any other SIEM solution or a windows or linux syslog collector.

It makes a lot of sense to use this for NSM logging as well (supported since NSM 2011.1). It will remove a lot of burden from your CPU. There are some things to keep in mind though when doing this. Like it can't be done over fxp0.

[RA] NSM logging will not reduce the CPU burden as it still saves files on the local filesystem.  NSM does offer a solution for capturing, viewing logs.

The logging situation on SRX is a big bad mess. It needs to be cleaned up by Juniper ASAP.

[RA] Could you please be more specific like what is not working?  Is it because we are not offering something, or is it because there is confusion about what options to use.

------

thanks,

Raheel

Rahel, it's a mess because no one really knows how to use it and what to use. I have had several tickets open with JTAC and even phone conversations with Juniper SEs, and everybody seems to have a different opinion.

Also, it is complete nonsense to have to configure separate interfaces just for logging because fxp0 can't handle the logs if you have to use stream mode logging.

To sum it all up: I should just be able to tell the machine to log to a log destination and not have to worry about anything. It should just work. But it doesn't.

As for high CPU load: If this has still not been fixed, then that's another point towards "it's a mess".

Hi Raheel

Thanks a lot for the valuable information. Its really missing in the Juniper documents and KB. I just want to know:

1- If we want to send the security logs to any file OR syslog server under [system syslog ...] then we have to make mode event? Am I right in understanding?

set security log mode event

2- By default the mode is event or stream on srx?

3- To send the security logs to STRM/NSM (2011)/Syslog Server in stream mode (through data plane), we have to make two things. Am I right in understanding?

a- The mode to stream

b- STRM/NSM/Syslog should be defined under [securty log...]

4- You replied above, For sending the logs from RE to NSM we just need to define the file under the [system syslog] and mode event. We dont need to define NSM as syslog server. But then Why in event mode we need to define STRM as syslog server?

Thanks

@aeroplane:

please find enclosed my replies with initials.

--------------

aeroplane wrote:

Hi Raheel

Thanks a lot for the valuable information. Its really missing in the Juniper documents and KB. I just want to know:

1- If we want to send the security logs to any file OR syslog server under [system syslog ...] then we have to make mode event? Am I right in understanding?

set security log mode event

[RA] Yes, if you want to use the [system syslog] features, you need to set [security log mode event]


2- By default the mode is event or stream on srx?

[RA] The default is event on SRX100, SRX210, SRX240, SRX650
The default is stream on SRX1400, SRX3000 and SRX5000


3- To send the security logs to STRM/NSM (2011)/Syslog Server in stream mode (through data plane), we have to make two things. Am I right in understanding?

a- The mode to stream

b- STRM/NSM/Syslog should be defined under [securty log...]
[RA] The 2 settings as identified are required


4- You replied above, For sending the logs from RE to NSM we just need to define the file under the [system syslog] and mode event. We dont need to define NSM as syslog server. But then Why in event mode we need to define STRM as syslog server?

[RA] NSM has 2 log collection capabilities.  If the SRX is being managed by NSM, then it should take care of all the settings required to setup NSM DMI based logging.  

NSM uses a different log transfer mechanism which is an xml data transfer of the logs.  This is why NSM does not need to be configured as a collector.

Thanks

-------------

hope this helps

thanks,

Raheel

Thanks Raheel.

Thanks for a good thread. Im getting my hands dirty with the SRX's for the first time and I have to agree with a few things said out here:

- The management interface thing is a real problem. As it stands I have to send logging traffic down the internal reth interface to me that's a fail.

- To leave users without a real log viewer is not fair. Once the FW is setup I'm in the logs all day.

---

@Jickfoo wrote:

Thanks for a good thread. Im getting my hands dirty with the SRX's for the first time and I have to agree with a few things said out here:

 

- The management interface thing is a real problem. As it stands I have to send logging traffic down the internal reth interface to me that's a fail.

 

- To leave users without a real log viewer is not fair. Once the FW is setup I'm in the logs all day.

 

 

---

+1

Rahel,

I appreciate your help. 

I have used many different documentations, all available from the Juniper support website and KB. And that's the point. There is not ONE document that describes the logging in it's entirety, there are many documents. And they don't reference each other, so things get confusing quickly. 

And as you can see by the questions others ask, there is confusion about this topic. Just search the forum for SRX logging, and you will find countless threads, all about the same topic and same questions.

You are saying that maybe customers are not setting up their systems correctly. And exactly that is the point. Why are people setting it up incorrectly? Because they don't know it any better and are confused. 

There should be ONE document that describes the logging mechanisms, explains the differences between event mode and stream mode, gives advise an which method to use under which circumstances and how to do it correctly. And it should go deeper than just providing config examples, the document should make people understand.

If Juniper offers such a vast array of logging options AND makes changes to them with basically every new Junos version, then Juniper should make it clear to customers how these work. Right now, as a customer you only have scattered documentation, spread across multiple documents and multiple versions, and they end up coming to the forums, just to find out that people here ask the same questions.

Hence: This is a mess.

Hi Raheel

Could you please reply on this?