Wednesday, January 2, 2019

Parsing and Displaying Cisco ISE Data in Splunk

By Tony Lee

If you are reading this page chances are good that you have both Splunk and Cisco Identity Services Engine (ISE). Chances are also pretty good that you have seen what a challenge it can be to parse these logs and make them into a useful dashboard. Granted, there is probably one app and TA combination out of the 50+ Cisco Apps and TAs on Splunkbase that will work for you, but if you strike out there, you can always try the solution and dashboard provided in the article below.

To get started, you should at least install the TA found here to parse the fields: http://splunkbase.splunk.com/app/1915 and give your incoming Cisco ISE syslog stream a sourcetype of "cisco:ise:syslog" per the documentation here:  http://docs.splunk.com/Documentation/AddOns/released/CiscoISE/Datatypes.  If you have data flowing and the fields are parsed out, we are in business.

Now, hold on to your hats, we are about to dive into the world of Cisco ISE logs and figure out just how to create the following dashboard.


Figure 1:  A useful Cisco ISE dashboard with all necessary data.

Caveat:  This article assumes that you called your Cisco ISE index "cisco-ise".  If you did not, just change the commands and dashboard to fit your index name.

The Problem

As mentioned in the introduction, the logs are a bit messy. The upside is that they are data rich. There is so much that you can extract from the logs, but first you need to piece them back together -- literally. The logs are sent over in chunks as shown below:

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 3  ....

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 2  ....

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 1  ....

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 0  ....

Here is the kicker, those four events are related (as indicated by the first large number which we are calling an event_id and then the next two numbers, the last of which increments). When combined into one event, it contains a ton of data. So, how do we combine the events? 


The Solution

Fortunately, Splunk has a transaction function that we can use to indicate that the events are related and should be combined into one event.  But we have a problem, that field is not parsed by the Splunk TA mentioned in the introduction, so we will need to parse it.

We can parse it with the following gnarly regex:
^(?:[^ \n]* ){3}(?P<event_id>\d+)\s+

Figure 2:  event_id parsed using a Splunk field extraction

With the event_id parsed, we can now use the transaction statement to combine the four events into one event which can be seen with the following search command:

index=cisco-ise | transaction event_id

Now, let's take it a bit farther and table the most interesting fields (feel free to leave a comment if you feel that we left out an interesting field):

index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID

Conclusion

We hope that this article has been helpful in understanding Cisco ISE logs and how to combine them to extract feature rich data from single events. As always, happy Splunking!

Dashboard Code

The dashboard code below assumes the index is cisco-ise and the Cisco TA is properly parsing the data. Please adjust as necessary.

<form>
  <label>Cisco ISE</label>
  <description>Populated by syslog data</description>
  <fieldset submitButton="true" autoRun="false">
    <input type="time" token="time" searchWhenChanged="true">
      <label>Time Range</label>
      <default>
        <earliest>-8h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Top Network Device Name</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 NetworkDeviceName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Location</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 Location</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top ISE Policy Set Name</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 ISEPolicySetName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top User Name</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 UserName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Details</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>

3 comments:

  1. Tony,

    Is there any scenario where the events are logged out of order?

    Here is an example for message ID: 0398385034 with 35 and 36 in between:

    0398385034 2 1

    0398385035 2 1
    0398385036 2 1

    0398385034 2 0

    ReplyDelete
    Replies
    1. Yes, I believe this is a possibility, but the transaction command should make the order irrelevant.

      Delete
  2. Hello-thanks for putting this together! Would you know where I can find a resource that maps the fields found in the syslog to the fields found in the Cisco ISE console?

    ReplyDelete