Starting in Junos OS Release 16.1R3 one can request streaming telemetry data using the gRPC framework with vendor-neutral data models based on OpenConfig. Great, but what configuration is required and is there a simple client available to test and troubleshoot it? I recently faced this question myself and found some great resources, which I documented in this post. For an in-depth discusssion about the various streaming telemetry option from Juniper devices, read thru the 3 parts on Tech Mocha, starting with USING OPENNTI AS A COLLECTOR FOR STREAMING TELEMETRY FROM JUNIPER DEVICES: PART 1.
Ingredients
Juniper vMX
You need a Junos network device to test against. I’m using the latest Junos vMX running 17.3R1 on KVM. Any MX, PTX or QFX with the same version should work equally well.
My lab vMX device uses the fxp0 IP address 172.17.0.4, which will be used to connect the JTI client to.
Before we can use any telemetry client, we must install two software packages on top of Junos:
- Junos Openconfig specifies an RPC model to enable the Junos Telemetry Interface plus configuration models based on OpenConfig
- Network Agent Software provides a framework to support OpenConfig and gRPC for the Junos Telemetry Interface. It functions as a gRPC server that terminates the OpenConfig remote procedure call (RPC) interface and streams the telemetry data according to the OpenConfig specification.
The latest versions downloaded are:
junos-openconfig-x86-32-0.0.0.8.tgz
network-agent-x86-32-17.3R1.10-C1.tgz
Install both packages on Junos with the following commands:
request system software add junos-openconfig-x86-32-0.0.0.8.tgz no-validate
request system software add network-agent-x86-32-17.3R1.10-C1.tgz no-validate
The CLI will ask for permition to relaunch its Daemon, which shall be granted. There is no need to restart Junos.
Junos Configuration
We need to enable gRPC and allow clients to connect from any source via
set system services extension-service request-response grpc clear-text port 50051
set system services extension-service request-response grpc skip-authentication
This opens up TCP port 50051 on the vMX and accepts gRPC requests in clear-text. Server-based SSL authentication is supported, but not used in this exercise. Earlier version of this blog post had notifications enabled, which is unrelated to gRPC and hence not needed.
JTI client
Install language Go and required libraries, then download and compile the client from github:
sudo apt install golang-go
go get github.com/golang/protobuf/proto
go get github.com/gorilla/mux
go get github.com/influxdata/influxdb/client/v2
go get github.com/prometheus/client_golang/prometheus/promhttp
go get github.com/spf13/pflag
go get golang.org/x/net/context
go get google.golang.org/grpc
go get github.com/nileshsimaria/jtimon/telemetry
go get github.com/nileshsimaria/jtimon/authentication
Clone and compile the client:
git clone https://github.com/nileshsimaria/jtimon.git
cd jtimon
go build
The build process takes no time at all. Display the tools help desk with:
$ ./jtimon -h
Usage of ./jtimon:
--compression string Enable HTTP/2 compression (gzip, deflate)
--config string Config file name
--csv-stats Capture size of each telemetry packet
--drop-check Check for packet drops
--gtrace Collect GRPC traces
--latency-check Check for latency
--log string Log file name
--max-kv uint Max kv
--max-run int Max run time in seconds
--prefix-check Report missing __prefix__ in telemetry packet
--print Print Telemetry data
--prometheus Stats for prometheus monitoring system
--sleep int Sleep after each read (ms)
--stats int Print collected stats periodically
--time-diff Time Diff for sensor analysis using InfluxDB
pflag: help requested
Action!
All the pieces are now in place and ready to have the JTI client connect to Junos and collect data.
Copy the sample config file sample-config/1.json to a filename of your choice and adjust the username/password and sensors to match your device. My example config looks like this:
$ cat wdm.json
{
"host": "172.17.0.4",
"port": 50051,
"user": "mwiget",
"password": "hb9rwm",
"cid": "my-client-id",
"grpc" : {
"ws" : 524288
},
"paths": [{
"path": "/interfaces",
"freq": 2000
}, {
"path": "/junos/system/linecard/cpu/memory/",
"freq": 1000
}, {
"path": "/bgp",
"freq": 10000
}, {
"path": "/components",
"freq": 10000
}]
}
Launch the client with the config file argument and –print:
$ ./jtimon --config wdm.json --stats 10 --drop-check --print
2017/11/28 09:46:44 LoginCheck failed
Well, if you get the LoginCheck failed message as I did initially, you got the username or password wrong, but got proof of a working gRPC connection between the client and your device. Once I enabled the lab account on Junos, the client succeeds and prints out all requested telemetry data:
$ ./jtimon --config wdm.json --print
gRPC headers from Junos:
init-response: [response { subscription_id: 1 } path_list { path: "/interfaces/" sample_frequency: 2000 } path_list { path: "/junos/system/linecard/cpu/memory/" sample_frequency: 1000 } path_list { path: "/bgp/" sample_frequency: 10000 } path_list { path: "/components/" sample_frequency: 10000 } ]
grpc-accept-encoding: [identity,deflate,gzip]
2017/11/28 09:50:43 system_id: wdm
2017/11/28 09:50:43 component_id: 0
2017/11/28 09:50:43 sub_component_id: 0
2017/11/28 09:50:43 path: sensor_1000_1_1:/junos/system/linecard/interface/:/junos/system/linecard/interface/:PFE
2017/11/28 09:50:43 sequence_number: 1
2017/11/28 09:50:43 timestamp: 1511859043721
2017/11/28 09:50:43 sync_response: %!d(bool=false)
2017/11/28 09:50:43 key: __timestamp__
2017/11/28 09:50:43 uint_value: 1511859043960
2017/11/28 09:50:43 key: __prefix__
2017/11/28 09:50:43 str_value: /interfaces/interface[name='xe-0/0/0']/
2017/11/28 09:50:43 key: init_time
2017/11/28 09:50:43 uint_value: 1511854410
2017/11/28 09:50:43 key: state/parent_ae_name
2017/11/28 09:50:43 str_value:
2017/11/28 09:50:43 key: state/high-speed
2017/11/28 09:50:43 int_value: 10000
2017/11/28 09:50:43 key: state/oper-status
2017/11/28 09:50:43 str_value: UP
2017/11/28 09:50:43 key: state/counters/in-pkts
2017/11/28 09:50:43 uint_value: 163
2017/11/28 09:50:43 key: state/counters/in-octets
2017/11/28 09:50:43 uint_value: 48226
2017/11/28 09:50:43 key: state/counters/in-unicast-pkts
2017/11/28 09:50:43 uint_value: 163
2017/11/28 09:50:43 key: state/counters/out-pkts
2017/11/28 09:50:43 uint_value: 166
2017/11/28 09:50:43 key: state/counters/out-octets
2017/11/28 09:50:43 uint_value: 49800
2017/11/28 09:50:43 key: state/counters/carrier-transitions
2017/11/28 09:50:43 uint_value: 1
2017/11/28 09:50:43 key: __prefix__
2017/11/28 09:50:43 str_value: /interfaces/interface[name='xe-0/0/1']/
2017/11/28 09:50:43 key: init_time
2017/11/28 09:50:43 uint_value: 1511854410
2017/11/28 09:50:43 key: state/parent_ae_name
...
Pretty cool! The shown output also answers a question I had and never dared to ask: Do I need an actual proto file to understand the content of the streamed data? Turns out the answer is ‘no’ and here is why: The output shows each value with a timestamp plus key and value pair. The key is text and self explenatory: e.g. ‘/interfaces/interface[name=’xe–0/0/1’]/’, which clearly describes the source of the data and ‘state/counters/out-octets’ its nature.
Similar, the CPU sensors reports, amongst other data:
2017/11/28 09:50:48 key: /components/component[name='FPC0:CPU0']/properties/property[name='mem-util-kernel-inline-ka-bytes-allocated']/value
2017/11/28 09:50:48 uint_value: 56
Again, no additional explanation needed !
The JTI tool has a lot more to offer, e.g. to report how much data is received every 10 second plus a summary once the client is terminated (using Ctrl-C), use:
$ ./jtimon --config wdm.json --stats 10 --drop-check
gRPC headers from Junos:
init-response: [response { subscription_id: 1 } path_list { path: "/interfaces/" sample_frequency: 2000 } path_list { path: "/junos/system/linecard/cpu/memory/" sample_frequency: 1000 } path_list { path: "/bgp/" sample_frequency: 10000 } path_list { path: "/components/" sample_frequency: 10000 } ]
grpc-accept-encoding: [identity,deflate,gzip]
+------------------------------+--------------------+--------------------+--------------------+--------------------+
| Timestamp | KV | Packets | Bytes | Bytes(wire) |
+------------------------------+--------------------+--------------------+--------------------+--------------------+
| Tue Nov 28 09:56:40 CET 2017 | 4525 | 20 | 189338 | 189338 |
| Tue Nov 28 09:56:50 CET 2017 | 11818 | 49 | 481065 | 481065 |
| Tue Nov 28 09:57:00 CET 2017 | 19285 | 80 | 793379 | 793379 |
| Tue Nov 28 09:57:10 CET 2017 | 26816 | 112 | 1108421 | 1108421 |
^C
Drops Distribution
+----+-----+-------+----------+-------------------------------------------------------------------------------------------------------------------------+
| CID |SCID| Drops | Received | Sensor Path |
+----+-----+-------+----------+-------------------------------------------------------------------------------------------------------------------------+
|65535| 0| 0| 16 | sensor_1003:/components/:/components/:chassisd |
|65535| 0| 0| 36 | sensor_1000_4_1:/interfaces/:/interfaces/:mib2d |
|65535| 0| 0| 9 | sensor_1000_6_1:/interfaces/:/interfaces/:xmlproxyd |
| 0| 0| 0| 17 | sensor_1001:/junos/system/linecard/cpu/memory/:/junos/system/linecard/cpu/memory/:PFE |
| 0| 0| 0| 17 | sensor_1000_1_1:/junos/system/linecard/interface/:/junos/system/linecard/interface/:PFE |
| 0| 0| 0| 17 | sensor_1000_1_2:/junos/system/linecard/interface/logical/usage/:/junos/system/linecard/interface/logical/usage/:PFE |
+----+-----+-------+----------+-------------------------------------------------------------------------------------------------------------------------+
Collector Stats (Run time : 40.257567985s)
112 : in-packets
26816 : data points (KV pairs)
415 : in-header wirelength (bytes)
1108421 : in-payload length (bytes)
1108421 : in-payload wirelength (bytes)
27710 : throughput (bytes per seconds)
0 : total packet drops
While the client is running, you can check the sensors on Junos:
lab@wdm> show agent sensors
Sensor Information :
Name : __default_fabric_sensor__
Resource : /junos/system/linecard/fabric/
Version : 1.0
Sensor-id : 129585707
Subscription-ID : 562952230490667
Parent-Sensor-Name : Not applicable
Component(s) : PFE
Server Information :
Name : __default__snmp_server__
Scope-id : 0
Remote-Address : 0.0.0.0
Remote-port : 0
Transport-protocol : UDP
Profile Information :
Name : __default_snmp_export_profile__
Reporting-interval : 30
Payload-size : 5000
Address : 0.0.0.0
Port : 0
Timestamp : 0
Format : GPB
DSCP : 0
Forwarding-class : 0
Loss-priority : low
Sensor Information :
Name : sensor_1000
Resource : /interfaces/
Version : 1.0
Sensor-id : 2657203
Subscription-ID : 1000
Parent-Sensor-Name : Not applicable
Component(s) : PFE,PFE,mib2d,xmlproxyd
Profile Information :
Name : export_1000
Reporting-interval : 2
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1000_1_1
Resource : /junos/system/linecard/interface/
Version : 1.1
Sensor-id : 186469721
Subscription-ID : 1000
Parent-Sensor-Name : sensor_1000
Component(s) : PFE
Profile Information :
Name : export_1000
Reporting-interval : 2
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1000_1_2
Resource : /junos/system/linecard/interface/logical/usage/
Version : 1.1
Sensor-id : 186469722
Subscription-ID : 1000
Parent-Sensor-Name : sensor_1000
Component(s) : PFE
Profile Information :
Name : export_1000
Reporting-interval : 2
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1000_4_1
Resource : /interfaces/
Version : 1.0
Sensor-id : 186472793
Subscription-ID : 1000
Parent-Sensor-Name : sensor_1000
Component(s) : mib2d
Profile Information :
Name : export_1000
Reporting-interval : 2
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1000_6_1
Resource : /interfaces/
Version : 1.0
Sensor-id : 186470745
Subscription-ID : 1000
Parent-Sensor-Name : sensor_1000
Component(s) : xmlproxyd
Profile Information :
Name : export_1000
Reporting-interval : 2
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1001
Resource : /junos/system/linecard/cpu/memory/
Version : 1.0
Sensor-id : 2657202
Subscription-ID : 1001
Parent-Sensor-Name : Not applicable
Component(s) : PFE
Profile Information :
Name : export_1001
Reporting-interval : 1
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1002
Resource : /bgp/
Version : 1.0
Sensor-id : 2657201
Subscription-ID : 1002
Parent-Sensor-Name : Not applicable
Component(s) : rpd
Profile Information :
Name : export_1002
Reporting-interval : 10
Payload-size : 5000
Format : GPB
Sensor Information :
Name : sensor_1003
Resource : /components/
Version : 1.0
Sensor-id : 2657200
Subscription-ID : 1003
Parent-Sensor-Name : Not applicable
Component(s) : chassisd
Profile Information :
Name : export_1003
Reporting-interval : 10
Payload-size : 5000
Format : GPB
Summary
The JTI client tool is a nifty open-source too to analyze, troubleshoot and learn about the streaming telemetry capabilities via gRPC. For more information, check out the following resources: