Wednesday, January 9, 2019

Parsing and Displaying Infoblox DHCP Data in Splunk

By Tony Lee

This article builds on our Infoblox DNS article available at:  http://securitysynapse.com/2019/01/parsing-and-displaying-infoblox-dns-in-splunk.html

If you are reading this page chances are good that you have both Splunk and Infoblox DHCP. While there is a pre-built TA (https://splunkbase.splunk.com/app/2934/) to help with the parsing, we needed some visualizations, so we wrote them and figured we would share what we created.


Figure 1:  At the time of writing this article, only a TA existed for Infoblox DHCP.

If you have this same situation, hopefully we can help you too. As a bonus, we will include the dashboard code at the end of the article.

Figure 2:  Dashboard that we include at the end of the article

Raw Log

This is what an Infoblox raw log might look like:

Sep 4 09:23:44 10.34.6.28 dhcpd[20310]: DHCPACK on 70.1.20.250 to fc:5c:fc:5f:10:85 via eth1 relay 10.120.20.66 lease-duration 600

Source:  https://docs.infoblox.com/display/NAG8/Using+a+Syslog+Server


Fields to Parse

Fortunately, our job is taken care of by the Infoblox TA (https://splunkbase.splunk.com/app/2934/)!  Just use the sourcetype of infoblox:dhcp to ensure it is properly parsed.

Search String

Now that the data is parsed, we can use the following to table the data:

index=infoblox sourcetype="infoblox:dhcp" | table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay

Combine a few panels together and we will have a dashboard similar to the one in the dashboard code section at the bottom of the article.

Conclusion

Even though we only had a Splunk TA (and not an app to go with it), we used the flexibility provided within Splunk to gain insight into Infoblox DHCP logs. We hope this article helps other save time. Feel free to leave comments in the section below. Happy Splunking!

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of infoblox. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Infoblox DHCP</label>
  <description>This is a high volume data feed - Be mindful of your time range</description>
  <fieldset submitButton="true">
    <input type="time" token="time" searchWhenChanged="true">
      <label>Time Range</label>
      <default>
        <earliest>-4h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild" searchWhenChanged="true">
      <label>Wildcard Search</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Total DHCP Traffic by Infoblox Host</title>
        <search>
          <query>| tstats count where index=infoblox, sourcetype="infoblox:dhcp" by host</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Action</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dhcp" $wild$ | table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay | top limit=0 action</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top signature</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dhcp" $wild$ | table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay | top limit=0 signature</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Servicing Host</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dhcp" $wild$ | table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay | top limit=0 src_hostname</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top src_ip</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dhcp" $wild$ | table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay | top limit=0 src_ip</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top dest_ip</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dhcp" $wild$ |  table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay | top limit=0 dest_ip</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Raw Logs</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dhcp" $wild$ | table _time, host, action, signature, src_category, src_hostname, src_ip, src_mac, dest_category, dest_hostname, dest_ip, relay</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>




Friday, January 4, 2019

Parsing and Displaying Infoblox DNS Data in Splunk

By Tony Lee

If you are reading this page chances are good that you have both Splunk and Infoblox DNS. While there is a pre-built TA (https://splunkbase.splunk.com/app/2934/) to help with the parsing, we needed some visualizations, so we wrote them and figured we would share what we created.


Figure 1:  At the time of writing this article, only a TA existed for Infoblox DNS.

If you have this same situation, hopefully we can help you too. As a bonus, we will include the dashboard code at the end of the article.

Figure 2:  Dashboard that we include at the end of the article

Raw Log

This is what an Infoblox raw log might look like:

30-Apr-2013 13:35:02.187 client 10.120.20.32#42386: query: foo.com IN A + (100.90.80.102)

Source:  https://docs.infoblox.com/display/NAG8/Capturing+DNS+Queries+and+Responses


Fields to Parse

Fortunately, our job is taken care of by the Infoblox TA (https://splunkbase.splunk.com/app/2934/)!  Just use the sourcetype of infoblox:dns to ensure it is properly parsed.

Search String

Now that the data is parsed, we can use the following to table the data:

index=infoblox sourcetype="infoblox:dns" | table _time, host, message_type, record_type, query, dns_request_client_ip, dns_request_client_port,  dns_request_name_serverIP, named_message | top limit=0 dns_request_client_ip

Combine a few panels together and we will have a dashboard similar to the one in the dashboard code section at the bottom of the article.

Conclusion

Even though we only had a Splunk TA (and not an app to go with it), we used the flexibility provided within Splunk to gain insight into Infoblox DNS logs. We hope this article helps other save time. Feel free to leave comments in the section below. Happy Splunking!

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of infoblox and a sourcetype of infoblox:dns. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Infoblox DNS</label>
  <description>This is a high volume data feed - Be mindful of your time range</description>
  <fieldset submitButton="true">
    <input type="time" token="time" searchWhenChanged="true">
      <label>Time Range</label>
      <default>
        <earliest>-15m</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild" searchWhenChanged="true">
      <label>Wildcard Search</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Total DNS Traffic by Infoblox Host</title>
        <search>
          <query>| tstats count where index=infoblox, sourcetype="infoblox:dns" by host</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top dns_request_client_ip</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dns" $wild$ | table _time, host, message_type, record_type, query, dns_request_client_ip, dns_request_client_port,  dns_request_name_serverIP, named_message | top limit=0 dns_request_client_ip</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top message_type</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dns" $wild$ | table _time, host, message_type, record_type, query, dns_request_client_ip, dns_request_client_port,  dns_request_name_serverIP, named_message | top limit=0 message_type</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top record_type</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dns" $wild$ | table _time, host, message_type, record_type, query, dns_request_client_ip, dns_request_client_port,  dns_request_name_serverIP, named_message | top limit=0 record_type</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top query</title>
        <search>
          <query>index=infoblox sourcetype="infoblox:dns" $wild$ | table _time, host, message_type, record_type, query, dns_request_client_ip, dns_request_client_port,  dns_request_name_serverIP, named_message | top limit=0 query</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <search>
          <query>index=infoblox sourcetype="infoblox:dns" $wild$ | table _time, host, message_type, record_type, query, dns_request_client_ip, dns_request_client_port,  dns_request_name_serverIP, named_message</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>



Wednesday, January 2, 2019

Parsing and Displaying Cisco ISE Data in Splunk

By Tony Lee

If you are reading this page chances are good that you have both Splunk and Cisco Identity Services Engine (ISE). Chances are also pretty good that you have seen what a challenge it can be to parse these logs and make them into a useful dashboard. Granted, there is probably one app and TA combination out of the 50+ Cisco Apps and TAs on Splunkbase that will work for you, but if you strike out there, you can always try the solution and dashboard provided in the article below.

To get started, you should at least install the TA found here to parse the fields: http://splunkbase.splunk.com/app/1915 and give your incoming Cisco ISE syslog stream a sourcetype of "cisco:ise:syslog" per the documentation here:  http://docs.splunk.com/Documentation/AddOns/released/CiscoISE/Datatypes.  If you have data flowing and the fields are parsed out, we are in business.

Now, hold on to your hats, we are about to dive into the world of Cisco ISE logs and figure out just how to create the following dashboard.


Figure 1:  A useful Cisco ISE dashboard with all necessary data.

Caveat:  This article assumes that you called your Cisco ISE index "cisco-ise".  If you did not, just change the commands and dashboard to fit your index name.

The Problem

As mentioned in the introduction, the logs are a bit messy. The upside is that they are data rich. There is so much that you can extract from the logs, but first you need to piece them back together -- literally. The logs are sent over in chunks as shown below:

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 3  ....

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 2  ....

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 1  ....

2019-08-06T16:33:06+00:00 HOST CISE_Passed_Authentications 0000649495 4 0  ....

Here is the kicker, those four events are related (as indicated by the first large number which we are calling an event_id and then the next two numbers, the last of which increments). When combined into one event, it contains a ton of data. So, how do we combine the events? 


The Solution

Fortunately, Splunk has a transaction function that we can use to indicate that the events are related and should be combined into one event.  But we have a problem, that field is not parsed by the Splunk TA mentioned in the introduction, so we will need to parse it.

We can parse it with the following gnarly regex:
^(?:[^ \n]* ){3}(?P<event_id>\d+)\s+

Figure 2:  event_id parsed using a Splunk field extraction

With the event_id parsed, we can now use the transaction statement to combine the four events into one event which can be seen with the following search command:

index=cisco-ise | transaction event_id

Now, let's take it a bit farther and table the most interesting fields (feel free to leave a comment if you feel that we left out an interesting field):

index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID

Conclusion

We hope that this article has been helpful in understanding Cisco ISE logs and how to combine them to extract feature rich data from single events. As always, happy Splunking!

Dashboard Code

The dashboard code below assumes the index is cisco-ise and the Cisco TA is properly parsing the data. Please adjust as necessary.

<form>
  <label>Cisco ISE</label>
  <description>Populated by syslog data</description>
  <fieldset submitButton="true" autoRun="false">
    <input type="time" token="time" searchWhenChanged="true">
      <label>Time Range</label>
      <default>
        <earliest>-8h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Top Network Device Name</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 NetworkDeviceName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Location</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 Location</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top ISE Policy Set Name</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 ISEPolicySetName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top User Name</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$ | top limit=0 UserName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Details</title>
        <search>
          <query>index=cisco-ise | transaction event_id | table _time, host, event_id, NetworkDeviceName, NAS_IP_Address, NAS_Port, Location, SelectedAuthenticationIdentityStores, SelectedAuthorizationProfiles, SSID, ISEPolicySetName, UserName, EndPointMACAddress, Calling_Station_ID, Called_Station_ID | search $wild$</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>

Friday, December 21, 2018

Parsing and Displaying Airwatch Data in Splunk

By Tony Lee

Have you ever searched for a Splunk app or TA and came up empty? We have too...  Not to worry though, with a little parsing and some dashboarding we should be able to create visibility where there may not be much previously. This was exactly the case when we tried to parse AirWatch logs (https://www.air-watch.com/).


Figure 1:  At the time of writing this article, no app or TA existed for airwatch.

If you have this same situation, hopefully we can help you too. This is the process we followed along with the regex used and the final dashboard produced. As a bonus, not only will we give you our regex that we used, but also the dashboard code at the end of the article.


Figure 2:  Final dashboard to display airwatch data


Raw Log

Mar 15 07:43:45 airwatchhost Mar 15 13:43:45 AirWatch AirWatch Syslog Details are as follows Event Type: Device
Event: SecurityInformationConfirmed
User: sysadmin
Enrollment User: TLEE
Event Source: Device
Event Module: Devices
Event Category: Command
Event Data: 
Device Friendly Name: TLEE iPhone iOS 12.1.0 GRY9


Fields we need to parse


  • Event Type
  • Event
  • User
  • Enrollment User
  • Event Source
  • Event Module
  • Event Category
  • Event Data
  • Device Friendly Name

Regular Expression Needed

There may be more graceful ways to parse these logs, but this seemed to work for us.  Go to Settings > Fields > Field Extractions > New Field Extraction.  For the fields use the following:

  • Select the app
  • Name:  All-Airwatch-Fields
  • Select the sourcetype for airwatch data
  • Inline
  • Extraction:  Copy and paste what we have below


Event\sType:\s(?P<EventType>.*?)\sEvent:\s(?P<Event>.*?)\sUser:\s(?P<User>.*?)\sEnrollment\sUser:\s(?P<EnrollmentUser>.*?)\sEvent\sSource:\s(?P<EventSource>.*?)\sEvent\sModule:\s(?P<EventModule>.*?)\sEvent\sCategory:\s(?P<EventCategory>.*?)\sEvent\sData:\s(?P<EventData>.*?)\sDevice\sFriendly\sName:\s(?P<DeviceFriendlyName>.*)

You should not need to restart Splunk, but give it 5 minutes and search with your index and sourcetype again in Verbose mode and the fields should now be parsed.

Conclusion

Even though we did not have a Splunk TA or App to help create visibility, we did this ourselves using the flexibility provided within Splunk. We hope this article helped other save time. If it helped or even if it did not work, feel free to leave a comment below. Happy Splunking!

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of airwatch. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Airwatch</label>
  <fieldset submitButton="true" autoRun="true">
    <input type="time" token="time">
      <label>Time Range</label>
      <default>
        <earliest>-60m@m</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard Search</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
  </fieldset>
  <row>
    <panel>
      <single>
        <title>Event Count</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | stats count</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">all</option>
      </single>
    </panel>
    <panel>
      <table>
        <title>Top Event</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | top limit=0 Event</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top EventModule</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | top limit=0 EventModule</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Enrollment User</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | top limit=0 EnrollmentUser</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Device Friendly Name</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | top limit=0 DeviceFriendlyName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <chart>
        <title>Top Event over Time</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | timechart count by Event</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="charting.chart">line</option>
        <option name="charting.drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </chart>
    </panel>
    <panel>
      <chart>
        <title>Top Enrollment User over Time</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName | timechart count by EnrollmentUser</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="charting.chart">line</option>
        <option name="charting.drilldown">none</option>
      </chart>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Details</title>
        <search>
          <query>index=airwatch $wild$ | table _time, EventType, Event, User, EnrollmentUser, EventSource, EventModule, EventCategory, EventData, DeviceFriendlyName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>


Monday, November 12, 2018

Monitoring USB Storage Activity with Splunk – Part II (Read/Write/Delete/Modify events)

By Tony Lee

Welcome to Part two in our series on Monitoring USB Storage Activity. In the first article (http://www.securitysynapse.com/2018/11/monitoring-usb-storage-activity-part-1.html), we examined what is required to monitor USB Storage connect and disconnect events. But how about activity that happens after the drives are connected? The good news is that this is also possible using Microsoft Windows Event logs and a bit of data crunching effort. In this article we will again use Splunk to aggregate, process, and display the logs. As a bonus, we will not only outline the steps to accomplish this task, but we will also provide working dashboard code at the end of the article.

Note: The Audit Removable Storage policy is only available in Windows 8 / 2008 and above—It is not available in Windows 7 / 2003.  ☹

Figure 1:  Dashboard provided at the end of the article


High-level steps

There are two main steps needed to accomplish this task. We need to generate and collect the Windows event logs and then we need to process and display the logs within Splunk. Each is outlined below.

Windows Event Generation
For Windows 8 / 2008 hosts and above, Microsoft USB activity logs can be enabled manually one machine at a time or via Group Policy (see references section below for instructions). For this demo, we will show how to enable it on one machine using Local Security Policy:  Advanced Audit Policy Configuration > System Audit Policies > Local Group > Object Access > Audit Removable Storage

Figure 2:  Enabling Audit of Removable Storage

Double click and audit for Success and Failure. After enabling auditing, we rebooted for good measure, because hey, this is Windows.

Activity Event IDs
Now that Audit Removable Storage is enabled, open Event Viewer > Windows Logs > Security.  Select Filter Current Log on the right-hand side and type in 4663 for event ID and click OK.  Insert a USB device and click the Refresh button on the right-hand side. If all is well, there should be multiple 4663 success events. Note that Event ID 4656 is used for failures.


Figure 3:  Testing 4663 and 4656 event visibility

Feel free to explore the data within each event but take note that for USB auditing the events that we care about have a Task Category of “Removable Storage”. For convenience we provide a file delete event below:

XX/XX/XXXX 05:54:43 PM
LogName=Security
SourceName=Microsoft Windows security auditing.
EventCode=4663
EventType=0
Type=Information
ComputerName=DESKTOP-8HSPO8Q
TaskCategory=Removable Storage
OpCode=Info
RecordNumber=1211
Keywords=Audit Success
Message=An attempt was made to access an object.

Subject:
Security ID: S-1-5-21-XXXXXXX-XXXXXXXXX-XXXXXXXXXX-XXXX
Account Name: User
Account Domain: DESKTOP-8HSPO8Q
Logon ID: 0x229E9

Object:
Object Server: Security
Object Type: File
Object Name: \Device\HarddiskVolume7\New Microsoft Word Document.docx
Handle ID: 0x1404
Resource Attributes:

Process Information:
Process ID: 0x17b4
Process Name: C:\Windows\explorer.exe

Access Request Information:
Accesses: DELETE

Access Mask: 0x10000



Windows Event Collection
Now that the logs are being generated, they need to be forwarded from the endpoints to a central location—in this case Splunk. This task could be accomplished using a number of methods such as Windows Event Collector (WEC), a Splunk Universal Forwarder agent, or some other forwarding method. For this demo, we will use a Splunk Universal Forwarder shown in next section.

Splunk

While we are assuming a functional Splunk Enterprise installation exists, we still need to collect the logs. We provide a sample Splunk Universal Forwarder configuration file below to help those using the Splunk Universal Forwarder. Note: we will be placing the events into an index called wineventlog. If this index does not already exist, you will first need to create it.

inputs.conf 
Located on the Windows endpoint (Usually found here:  C:\Program Files\SplunkUniversalForwarder\etc\apps\SplunkUniversalForwarder\local\inputs.conf)

[WinEventLog://Security]
index = wineventlog
checkpointInterval = 5
current_only = 0
disabled = 0
start_from = oldest
whitelist = 4663, 4656


Once the inputs.conf file is properly configured (and the universal forwarder restarted) to collect these logs from the endpoint, we need to verify that the logs are reaching Splunk. Try running the following Splunk search:

index=wineventlog 


If you see results, try something more specific, such as either of the following:

index=wineventlog EventCode=4663
index=wineventlog EventCode=4656


Conclusion

Now that we have the proper event IDs flowing into Splunk, we created a Removable Storage Activity dashboard. The dashboard provides statistical analysis for top accounts, hostname, actions, and processes. It even includes events over time by hostname and action along with the details needed to investigate USB connections. Because there may be applications within an environment that scan or interact with removable storage, it may be necessary to add some filters to reduce noise which can be customized for each environment. For your convenience, we included the dashboard code below.

Acknowledgement and References

https://www.eventtracker.com/tech-articles/tracking-removable-storage-windows-security-log/
https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/jj574128(v=ws.11)


Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of wineventlog. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Removable Storage Activity</label>
  <description>index=wineventlog EventCode=4663 TaskCategory="Removable Storage"</description>
  <fieldset autoRun="true" submitButton="true">
    <input type="time" token="time">
      <label>Time Range</label>
      <default>
        <earliest>0</earliest>
        <latest></latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard Search</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="multiselect" token="Accesses">
      <label>Actions (Accesses)</label>
      <choice value="*">All</choice>
      <choice value="ReadData (or ListDirectory)">ReadData (or ListDirectory)</choice>
      <choice value="WriteData (or AddFile)">WriteData (or AddFile)</choice>
      <choice value="AppendData (or AddSubdirectory or CreatePipeInstance)">AppendData (or AddSubdirectory or CreatePipeInstance)</choice>
      <choice value="DELETE">DELETE</choice>
      <default>*</default>
      <initialValue>*</initialValue>
      <valuePrefix>Accesses="</valuePrefix>
      <valueSuffix>"</valueSuffix>
      <delimiter> OR </delimiter>
    </input>
  </fieldset>
  <row>
    <panel>
      <single>
        <title>Total events</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | stats count</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">all</option>
        <option name="refresh.display">progressbar</option>
      </single>
    </panel>
    <panel>
      <table>
        <title>Top Account_Domain</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | top limit=0 Account_Domain</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top ComputerName</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | top limit=0 ComputerName</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Account_Name</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | top limit=0 Account_Name</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Top Accesses</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | top limit=0 Accesses</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Process_Name</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | top limit=0 Process_Name</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <chart>
        <title>Activity Over Time</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | eval ComputerAction = ComputerName + ":" + Accesses | timechart count(ComputerAction) by ComputerAction</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="charting.chart">column</option>
        <option name="charting.drilldown">none</option>
        <option name="refresh.display">progressbar</option>
      </chart>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Details</title>
        <search>
          <query>index=wineventlog EventCode=4663 TaskCategory="Removable Storage" $wild$ $Accesses$ | dedup _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name | table _time, Account_Domain, ComputerName, Account_Name, Accesses, Process_Name, Object_Name</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>