Friday, June 21, 2019

Microsoft Bug? Shift + Delete does not generate 4660 Event

By Tony Lee

If you have ever searched for anything along the lines of:  "How do I discover who deleted a file", you will probably find a dozen articles or more telling you to check for Windows Event ID 4660.

Some examples of excellent search results and resources can be found below (lmgtfy):

Then the article will most likely mention that Event ID 4660 lacks the object name (son of a biscuit!) and you will need to map the event using the handle ID to Event ID 4656 or Event ID 4663 (with an Accesses=DELETE).  No problem, we've got this.  But what do you do if 4660 is not always created? This can happen!  (Dun... dun... duuuuunn.....)

Discovery

There are obviously multiple ways to delete a file such as the following:
  • Delete key
  • Right Click > Delete
  • "del" from a command prompt
  • Shift + Delete
Every single method above generates a 4660 (and 4663) except the last one, Shift + Delete, which happens to be my personal favorite way to to delete a file.  :-(  Delete it like you mean it...

Test Methodology

The discovery was frustrating and quite accidental. While deleting files, we noticed that no 4660 (or even 4663) logs were being created when we used Shift + Delete. In utter disbelief, we set up the following to prove our sanity:
  • Enable all necessary auditing (lots of articles on this)
  • Open Event Viewer > Windows Logs > Security > Filter Current Log > 4660 in the filter box
  • Create 4 text files in which you will delete using the methods above
  • Delete one file at a time and wait for Event Viewer to notify you of a new log
  • Notice that Shift + Delete DOES NOT GENERATE A 4660!

Figure 1:  Test methodology shown above with Event Viewer, filters, notifications, and four files to delete


Conclusion

We all know that Windows logging is horrible, but this one takes the cake. It just seems scary that holding down shift while pressing delete will omit the log whose entry starts with: "An object was deleted." One possible work around is enabling the ever noisy Event ID 4656 and filtering that down--which still has its own pitfalls. Anyway, we hope this article helped debunk the myth that using Event ID 4660 for detecting file deletes is reliable (regardless of the name of the log entry).


Sample Logs

Some sample logs from our friend Randy Franklin Smith:

https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4663#examples

https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4660#examples

https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4656#examples

Wednesday, June 19, 2019

Parsing and Displaying Windows Firewall data in Splunk

By Tony Lee

Have you ever wondered what it would be like to harness all of the Windows firewall data in your environment and crunch that activity using big data analytics?  What you find might shock you.

Fortunately for us Splunk heads, Andreas Roth has already completed most of the work to send the logs to Splunk and even parse them.  https://splunkbase.splunk.com/app/3300/#/details

Figure 1:  TA to collect and parse the logs already exists!

Big shout out to Andreas for the jump start.  However, we added some parsing to get the layer 4 transport protocol and then created a dashboard (shown below) that we are going to share here.

Figure 2:  Dashboard provided in this article

Prerequisites

There are some things you will need to do before we can make use of the Windows Firewall logs:
1)  Enable Windows Firewall Logging 
   Tip:  Use the link in the Log Location section below to enable and configure the firewall via GPO

2)  Forward the logs (written to disk) to Splunk via a Splunk UF, beats agent, etc.
   Tip:  This is made "easy" by installing the TA mentioned above on your forwarders

3)  Parse the logs
   Tip:  This is made easy by installing the TA mentioned above on your indexers

4)  Display the logs
   Tip:  This is made easy by using our dashboard code found at the end of this article

Log Location

The first thing we need to do is discover where those logs are located. After a bit of research, you will find that by default they should be located here:

%systemroot%\Windows\system32\LogFiles\Firewall\pfirewall.log

However, when I tried searching my machine for any sign of the logs, I discovered that they were turned off.  In fact, Windows does not log these to disk by default.

Figure 1:  Windows Firewall Logs off by default

To enable Windows Firewall logging, see the following article:  https://docs.microsoft.com/en-us/windows/security/threat-protection/windows-firewall/configure-the-windows-firewall-log#to-configure-the-windows-firewall-log


Raw Log

What do these logs look like once they are written to disk? Well, they are short sweet and to the point.  See the example log below:

2018-07-03 14:19:55 DROP UDP 192.168.2.1 224.0.0.252 50859 5355 56 - - - - - - - RECEIVE

There are a lot of fields there, but Microsoft kindly places a header in the log file to indicate the field names:

date time action protocol src-ip dst-ip src-port dst-port size tcpflags tcpsym tcpack tcpwin icmptype icmpcode info path

As mentioned before, if you installed the TA-winfw on your search head and universal forwarders most of the parsing will be performed for you.  Although, there was one field which did not seem to be parsed for us.  No biggie, we don't have to do the real heavy lifting.

Parsing

Using our example event from earlier, the transport field did not seem to be parsed for us which typically indicates if the packet was UDP or TCP, etc..  Instead the protocol field just displayed "ip".

2018-07-03 14:19:55 DROP UDP 192.168.2.1 224.0.0.252 50859 5355 56 - - - - - - - RECEIVE


To correct this, we added the following regex generated from the Splunk field extractor for our sourcetype of winfw:

^(?:[^ \n]* ){3}(?P<transport>\w+)

Search String

After parsing out the "transport" field, we can now form our search string:

index=winfw | table _time, dvc, direction, action, transport, src_ip, src_port, dest_ip, dest_port

Taking this a step farther, we created a dashboard which is provided at the bottom of the article.

Conclusion

Using the dashboard code below I bet you can find some interesting events in your network. Even if you don't find something malicious, you can probably find a misconfiguration or two. Correcting these issues will not only save on host performance and network performance, but now Splunk performance too.  Happy Splunking!

Great  resource:
https://www.howtogeek.com/220204/how-to-track-firewall-activity-with-the-windows-firewall-log/

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of winfw and a sourcetype of winfw. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Windows Firewall</label>
  <fieldset submitButton="true" autoRun="true">
    <input type="time" searchWhenChanged="false" token="time">
      <label>Time Range</label>
      <default>
        <earliest>-15m</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" searchWhenChanged="false" token="wild">
      <label>Wildcard Search</label>
      <default>*</default>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Event Count</title>
        <search>
          <query>| tstats count where index=winfw by host</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <chart>
        <title>Top Action</title>
        <search>
          <query>index=winfw $wild$ | table _time, action | top action</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="charting.chart">pie</option>
        <option name="refresh.display">progressbar</option>
      </chart>
    </panel>
    <panel>
      <table>
        <title>Top Source IP</title>
        <search>
          <query>index=winfw $wild$ | table _time, src_ip | top limit=0 src_ip</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Dest IP</title>
        <search>
          <query>index=winfw  $wild$ | table _time, dest_ip | top limit=0 dest_ip</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
    <panel>
      <table>
        <title>Top Dest Port</title>
        <search>
          <query>index=winfw $wild$ | table _time, dest_port | top limit=0 dest_port</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="count">10</option>
        <option name="drilldown">cell</option>
        <option name="refresh.display">progressbar</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Details</title>
        <search>
          <query>index=winfw $wild$ | table _time, dvc, direction, action, transport, src_ip, src_port, dest_ip, dest_port</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">20</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>

Monday, June 17, 2019

Parsing and Displaying Okta Data in Splunk - Part III - App Lookup Tool

By Tony Lee

If you are reading this page chances are good that you have both Splunk and Okta. The good news is that there is a pre-built TA (https://splunkbase.splunk.com/app/2806/) to help with the data ingest and parsing, plus an app (https://splunkbase.splunk.com/app/2821/) to help with the visualizations. However, there is always room to improve and thus we created and are sharing some additional lookup dashboards to make the data more actionable.

Figure 1:  At the time of this article, an Okta TA and App exists

The first two articles of this series covered two useful lookup tools:
1)  User Lookup - http://securitysynapse.blogspot.com/2019/06/parsing-and-displaying-okta-data-part-i-user-lookup.html
2)  Group Lookup - http://securitysynapse.blogspot.com/2019/06/parsing-and-displaying-okta-data-part-ii-group-lookup.html

In this third article, we will show how to create an app lookup tool (with group and user drilldown!) using the information contained within the Okta logs. Since Okta has quite a bit of user and group information, the existing data makes a useful Rolodex that is available to Splunk. This is especially useful to a SOC analyst who might be tracking down user or group access based on application name.


Figure 2:  App Lookup Tool created using Okta data!


Data Categorization

Okta data brought in via the TA is easily distinguishable via the source field.  For example:
  • okta:user
  • okta:event
  • okta:group
  • okta:app
Thus, for app data, we will use source=okta:app 


Raw Log

A sample app event is shown below (this should be close to what you see using the TA). Our event also contained two multi-value fields called assigned_users{} which contains the user IDs for the users assigned to that app and assigned_groups{} which contains the group IDs for the groups assigned to that app:

[
  {
    "id": "0oa1gjh63g214q0Hq0g4",
    "name": "testorgone_customsaml20app_1",
    "label": "Custom Saml 2.0 App",
    "status": "ACTIVE",
    "lastUpdated": "2016-08-09T20:12:19.000Z",
    "created": "2016-08-09T20:12:19.000Z",
    "accessibility": {
      "selfService": false,
      "errorRedirectUrl": null,
      "loginRedirectUrl": null
    },
    "visibility": {
      "autoSubmitToolbar": false,
      "hide": {
        "iOS": false,
        "web": false
      },
      "appLinks": {
        "testorgone_customsaml20app_1_link": true
      }
    },
    "features": [],
    "signOnMode": "SAML_2_0",
    "credentials": {
      "userNameTemplate": {
        "template": "${fn:substringBefore(source.login, \"@\")}",
        "type": "BUILT_IN"
      },
      "signing": {}
    },
    "settings": {
      "app": {},
      "notifications": {
        "vpn": {
          "network": {
            "connection": "DISABLED"
          },
          "message": null,
          "helpUrl": null
        }
      },
      "signOn": {
        "defaultRelayState": "",
        "ssoAcsUrl": "https://{yourOktaDomain}",
        "idpIssuer": "http://www.okta.com/${org.externalKey}",
        "audience": "https://example.com/tenant/123",
        "recipient": "http://recipient.okta.com",
        "destination": "http://destination.okta.com",
        "subjectNameIdTemplate": "${user.userName}",
        "subjectNameIdFormat": "urn:oasis:names:tc:SAML:1.1:nameid-format:emailAddress",
        "responseSigned": true,
        "assertionSigned": true,
        "signatureAlgorithm": "RSA_SHA256",
        "digestAlgorithm": "SHA256",
        "honorForceAuthn": true,
        "authnContextClassRef": "urn:oasis:names:tc:SAML:2.0:ac:classes:PasswordProtectedTransport",
        "spIssuer": null,
        "requestCompressed": false,
        "attributeStatements": []
      }
    },
    "_links": {
      "logo": [
        {
          "name": "medium",
          "href": "http://testorgone.okta.com/assets/img/logos/default.6770228fb0dab49a1695ef440a5279bb.png",
          "type": "image/png"
        }
      ],
      "appLinks": [
        {
          "name": "testorgone_customsaml20app_1_link",
          "href": "http://testorgone.okta.com/home/testorgone_customsaml20app_1/0oa1gjh63g214q0Hq0g4/aln1gofChJaerOVfY0g4",
          "type": "text/html"
        }
      ],
      "help": {
        "href": "http://testorgone-admin.okta.com/app/testorgone_customsaml20app_1/0oa1gjh63g214q0Hq0g4/setup/help/SAML_2_0/instructions",
        "type": "text/html"
      },
      "users": {
        "href": "http://testorgone.okta.com/api/v1/apps/0oa1gjh63g214q0Hq0g4/users"
      },
      "deactivate": {
        "href": "http://testorgone.okta.com/api/v1/apps/0oa1gjh63g214q0Hq0g4/lifecycle/deactivate"
      },
      "groups": {
        "href": "http://testorgone.okta.com/api/v1/apps/0oa1gjh63g214q0Hq0g4/groups"
      },
      "metadata": {
        "href": "http://testorgone.okta.com:/api/v1/apps/0oa1gjh63g214q0Hq0g4/sso/saml/metadata",
        "type": "application/xml"
      }
    }
  },
  {
    "id": "0oabkvBLDEKCNXBGYUAS",
    "name": "template_swa",
    "label": "Sample Plugin App",
    "status": "ACTIVE",
    "lastUpdated": "2013-09-11T17:58:54.000Z",
    "created": "2013-09-11T17:46:08.000Z",
    "accessibility": {
      "selfService": false,
      "errorRedirectUrl": null
    },
    "visibility": {
      "autoSubmitToolbar": false,
      "hide": {
        "iOS": false,
        "web": false
      },
      "appLinks": {
        "login": true
      }
    },
    "features": [],
    "signOnMode": "BROWSER_PLUGIN",
    "credentials": {
      "scheme": "EDIT_USERNAME_AND_PASSWORD",
      "userNameTemplate": {
        "template": "${source.login}",
        "type": "BUILT_IN"
      }
    },
    "settings": {
      "app": {
        "buttonField": "btn-login",
        "passwordField": "txtbox-password",
        "usernameField": "txtbox-username",
        "url": "https://example.com/login.html"
      }
    },
    "_links": {
      "logo": [
        {
          "href": "https:/example.okta.com/img/logos/logo_1.png",
          "name": "medium",
          "type": "image/png"
        }
      ],
      "users": {
        "href": "https://{yourOktaDomain}/api/v1/apps/0oabkvBLDEKCNXBGYUAS/users"
      },
      "groups": {
        "href": "https://{yourOktaDomain}/api/v1/apps/0oabkvBLDEKCNXBGYUAS/groups"
      },
      "self": {
        "href": "https://{yourOktaDomain}/api/v1/apps/0oabkvBLDEKCNXBGYUAS"
      },
      "deactivate": {
        "href": "https://{yourOktaDomain}/api/v1/apps/0oabkvBLDEKCNXBGYUAS/lifecycle/deactivate"
      }
    }
  }
]

Source:  https://developer.okta.com/docs/api/resources/apps#list-applications

Fields we need to parse

Fortunately, the available TA already parses the data for us, but the fields that we are most interested in for this lookup dashboard are the following:
  • dest
  • name
  • label
  • signOnMode
  • created
  • lastUpdated
  • status
  • assigned_users{}
  • assigned_groups{}
Feel free to modify the search and replace fields as needed.

Search String

A simple search string that gets us the table needed is shown below. We deduplicated the results by app id since it is a unique field. We also added a count of the number of users (assigned_users) and groups (assigned_groups) in each app. Now just add filters such as the ones we provided in our dashboard code at the end of the article and you are in business! 

index=okta source=okta:app | dedup id | eval assigned_users=mvcount('assigned_users{}') | eval assigned_groups=mvcount('assigned_groups{}') | fillnull value=0 assigned_users, assigned_groups | table dest, name, label, signOnMode, assigned_groups, assigned_users, created, lastUpdated, status, assigned_users{}, assigned_groups{}

The dashboard code we included below also contains a user and group drilldown to reveal the users and groups assigned to selected app. Simply click the row of the application you are interested in and it will show you the users and groups assigned to that app. This interactive drilldown pulls the assigned_users{} and assigned_groups{} multi-value fields and performs a user lookup (source=okta:user and source=okta:group) as seen in the previous article. Note, we also use a clever <fields> trick to hide the assigned_users{} and assigned_groups{} columns, while still making that data usable in the drilldown.

Conclusion

Even though we had a Splunk TA and App to perform the parsing and help create visibility, we extended the usefulness of the data to build an app lookup tool with a user and group drilldown. We hope this article helps others gain additional insight into their user and group data via Okta logs. Happy Splunking!

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of okta. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Okta App Lookup</label>
  <description>index=okta source=okta:app   (First try last 6 hours, then try a longer time range)</description>
  <fieldset autoRun="true" submitButton="true">
    <input type="time" token="time">
      <label>Time Range</label>
      <default>
        <earliest>-6h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard Search</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="name">
      <label>Name (Exact match)</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="label">
      <label>Label (Exact Match)</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="dropdown" token="status">
      <label>Status</label>
      <choice value="*">ALL</choice>
      <choice value="ACTIVE">ACTIVE</choice>
      <choice value="INACTIVE">INACTIVE</choice>
      <default>ACTIVE</default>
      <initialValue>ACTIVE</initialValue>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>App Details</title>
        <search>
          <query>index=okta source=okta:app $wild$ name=$name$ label=$label$ status=$status$ | dedup id | eval assigned_users=mvcount('assigned_users{}') | eval assigned_groups=mvcount('assigned_groups{}') | fillnull value=0 assigned_users, assigned_groups | table dest, name, label, signOnMode, assigned_groups, assigned_users, created, lastUpdated, status, assigned_users{}, assigned_groups{}</query>
          <earliest>-6h@h</earliest>
          <latest>now</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
        <fields>["dest","name","label","signOnMode","assigned_groups","assigned_users","created","lastUpdated","status"]</fields>
        <drilldown>
          <set token="users">$row.assigned_users{}$</set>
          <set token="groups">$row.assigned_groups{}$</set>
        </drilldown>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Assigned Groups (Click a row above to fetch the groups assigned to the app)</title>
        <search>
          <query>| stats count as id | eval id=split("$groups$", ",") | mvexpand id | join type=left id [search index=okta source=okta:group id IN ($groups$)] | eval num_members=mvcount('members{}') | fillnull value=0 num_members | table dest, type, profile.groupScope, profile.windowsDomainQualifiedName, profile.name, profile.description, created, lastUpdated, lastMembershipUpdated, num_members</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="rowNumbers">true</option>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Assigned Users (Click a row above to fetch the users assigned to the app)</title>
        <search>
          <query>| stats count as id | eval id=split("$users$", ",") | mvexpand id | join type=left id [search index=okta source=okta:user id IN ($users$) | table id, credentials.provider.type, profile.title, profile.firstName, profile.middleName, profile.lastName, profile.email, profile.primaryPhone, created, passwordChanged, lastLogin, status]</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="rowNumbers">true</option>
      </table>
    </panel>
  </row>
</form>

Friday, June 14, 2019

Parsing and Displaying Okta Data in Splunk - Part II - Group Lookup Tool

By Tony Lee

If you are reading this page chances are good that you have both Splunk and Okta. The good news is that there is a pre-built TA (https://splunkbase.splunk.com/app/2806/) to help with the data ingest and parsing, plus an app (https://splunkbase.splunk.com/app/2821/) to help with the visualizations. However, there is always room to improve and thus we created and are sharing some additional lookup dashboards to make the data more actionable.

Figure 1:  At the time of this article, an Okta TA and App exists

We shared a user lookup tool in part I of the series: http://securitysynapse.blogspot.com/2019/06/parsing-and-displaying-okta-data-part-i-user-lookup.html

In this second article, we will show how to create a group lookup tool (with member drilldown!) using the information contained within the Okta logs. Since Okta has quite a bit of user and group information, the existing data makes a useful Rolodex (again, ask your parents) that is available to Splunk. This is especially useful to a SOC analyst who might be tracking down a user or group.

Figure 2:  Group Lookup Tool created using Okta data!


Data Categorization

Okta data brought in via the TA is easily distinguishable via the source field.  For example:
  • okta:user
  • okta:event
  • okta:group
  • okta:app
Thus, for group data, we will use source=okta:group 


Raw Log

A sample group event is shown below. Our event also contained a multi-value field called members{} which contains the user ID for the members of that group:

{
  "id": "00g1emaKYZTWRYYRRTSK",
  "created": "2015-02-06T10:11:28.000Z",
  "lastUpdated": "2015-10-05T19:16:43.000Z",
  "lastMembershipUpdated": "2015-11-28T19:15:32.000Z",
  "objectClass": [
    "okta:user_group"
  ],
  "type": "OKTA_GROUP",
  "profile": {
    "name": "West Coast Users",
    "description": "All Users West of The Rockies"
  },
  "_links": {
    "logo": [
      {
        "name": "medium",
        "href": "https://{yourOktaDomain}/img/logos/groups/okta-medium.png",
        "type": "image/png"
      },
      {
        "name": "large",
        "href": "https://{yourOktaDomain}/img/logos/groups/okta-large.png",
        "type": "image/png"
      }
    ],
    "users": {
      "href": "https://{yourOktaDomain}/api/v1/groups/00g1emaKYZTWRYYRRTSK/users"
    },
    "apps": {
      "href": "https://{yourOktaDomain}/api/v1/groups/00g1emaKYZTWRYYRRTSK/apps"
    }
  }
}

Source:  https://developer.okta.com/docs/api/resources/groups#get-group

Fields we need to parse

Fortunately, the available TA already parses the data for us, but the fields that we are most interested in for this lookup dashboard are the following:
  • type
  • groupScope
  • windowsDomainQualifiedName
  • name
  • description
  • created
  • lastUpdated
  • lastMembershipUpdated
  • members{}
Feel free to modify the search and replace fields as needed.

Search String

A simple search string that gets us the table needed is shown below. We deduplicated the results by id since it is a unique field. We also added a count of the number of members (num_members) in each group. Now just add filters such as the ones we provided in our dashboard code at the end of the article and you are in business! 

index=okta source=okta:group | dedup id | eval num_members=mvcount('members{}') | fillnull value=0 num_members | table dest, type, profile.groupScope, profile.windowsDomainQualifiedName, profile.name, profile.description, created, lastUpdated, lastMembershipUpdated, num_members

The dashboard code we included below also contains a group drilldown to reveal the users in the group. Simply click the row of the group you are interested in and it will show you the users in that group. This clever drilldown pulls the members{} multi-value field and performs a user lookup (source=okta:user) as seen in the previous article. Note, we also use a clever <fields> trick to hide the members{} column, but still make it usable in the drilldown.

Conclusion

Even though we had a Splunk TA and App to perform the parsing and help create visibility, we extended the usefulness of the data to build a group lookup tool with member drilldown. We hope this article helps others gain additional insight into their user and group data via Okta logs. Happy Splunking!

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of okta. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Okta Group Lookup</label>
  <description>index=okta source=okta:group   (First try last 6 hours, then try a longer time range)</description>
  <fieldset autoRun="true" submitButton="true">
    <input type="time" token="time">
      <label>Time Range</label>
      <default>
        <earliest>-6h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard Search</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="name">
      <label>Group Name (Exact match)</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="description">
      <label>Group Description</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="dropdown" token="exclude_empty_groups">
      <label>Exclude Empty Groups</label>
      <choice value="members{}=*">Yes</choice>
      <choice value="">No</choice>
      <default>members{}=*</default>
      <initialValue>members{}=*</initialValue>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Group Details</title>
        <search>
          <query>index=okta source=okta:group $wild$ profile.name=$name$ profile.description=*$description$* $exclude_empty_groups$ | dedup id | eval num_members=mvcount('members{}') | fillnull value=0 num_members | table dest, type, profile.groupScope, profile.windowsDomainQualifiedName, profile.name, profile.description, created, lastUpdated, lastMembershipUpdated, num_members, members{}</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">cell</option>
        <option name="percentagesRow">false</option>
        <option name="refresh.display">progressbar</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
        <fields>["dest","type","profile.groupScope","profile.windowsDomainQualifiedName","profile.name","profile.description","created","lastUpdated","lastMembershipUpdated","num_members"]</fields>
        <drilldown>
          <set token="members">$row.members{}$</set>
        </drilldown>
      </table>
    </panel>
  </row>
  <row>
    <panel>
      <table>
        <title>Members (Click a row above to fetch the members of the group)</title>
        <search>
          <query>| stats count as id | eval id=split("$members$", ",") | mvexpand id | join type=left id [search index=okta source=okta:user id IN ($members$) | table id, credentials.provider.type, profile.title, profile.firstName, profile.middleName, profile.lastName, profile.email, profile.primaryPhone, created, passwordChanged, lastLogin, status]</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
        </search>
        <option name="drilldown">none</option>
        <option name="rowNumbers">true</option>
      </table>
    </panel>
  </row>
</form>

Wednesday, June 12, 2019

Parsing and Displaying Okta Data in Splunk - Part I - User Lookup Tool

By Tony Lee

If you are reading this page chances are good that you have both Splunk and Okta. The good news is that there is a pre-built TA (https://splunkbase.splunk.com/app/2806/) to help with the data ingest and parsing, plus an app (https://splunkbase.splunk.com/app/2821/) to help with the visualizations. However, there is always room to improve and thus we created and are sharing some additional lookup dashboards to make the data more actionable.

Figure 1:  At the time of this article, an Okta TA and App exists

In this first article, we will show how to create a user lookup tool using the information contained within the Okta logs. As a bonus, we will provide our dashboard board at the bottom. Since Okta has quite a bit of user information, the existing data makes a useful Rolodex (yes, that paper-based wheel of information--ask your parents if still unclear) that is available to Splunk. This is especially useful to a SOC analyst who might be tracking down a user or group.

Figure 2:  Okta User Lookup dashboard with useful filters!


Data Categorization

Okta data brought in via the TA is easily distinguishable via the source field.  For example:
  • okta:user
  • okta:event
  • okta:group
  • okta:app
Thus, for user data, we will use source=okta:user 


Raw Log

A sample user event is shown below. Notice how much useful data is contained within the event:

[
  {
    "id": "00ub0oNGTSWTBKOLGLNR",
    "status": "STAGED",
    "created": "2013-07-02T21:36:25.344Z",
    "activated": null,
    "statusChanged": null,
    "lastLogin": null,
    "lastUpdated": "2013-07-02T21:36:25.344Z",
    "passwordChanged": "2013-07-02T21:36:25.344Z",
    "profile": {
      "firstName": "Isaac",
      "lastName": "Brock",
      "email": "isaac.brock@example.com",
      "login": "isaac.brock@example.com",
      "mobilePhone": "555-415-1337"
    },
    "credentials": {
      "provider": {
        "type": "OKTA",
        "name": "OKTA"
      }
    },
    "_links": {
      "self": {
        "href": "https://{yourOktaDomain}/api/v1/users/00ub0oNGTSWTBKOLGLNR"
      }
    }
  },
  {
    "id": "00ub0oNGTSWTBKOLGLNR",
    "status": "ACTIVE",
    "created": "2013-06-24T16:39:18.000Z",
    "activated": "2013-06-24T16:39:19.000Z",
    "statusChanged": "2013-06-24T16:39:19.000Z",
    "lastLogin": "2013-06-24T17:39:19.000Z",
    "lastUpdated": "2013-07-02T21:36:25.344Z",
    "passwordChanged": "2013-07-02T21:36:25.344Z",
    "profile": {
      "firstName": "Eric",
      "lastName": "Judy",
      "email": "eric.judy@example.com",
      "secondEmail": "eric@example.org",
      "login": "eric.judy@example.com",
      "mobilePhone": "555-415-2011"
    },
    "credentials": {
      "password": {},
      "recovery_question": {
        "question": "The stars are projectors?"
      },
      "provider": {
        "type": "OKTA",
        "name": "OKTA"
      }
    },
    "_links": {
      "self": {
        "href": "https://{yourOktaDomain}/api/v1/users/00ub0oNGTSWTBKOLGLNR"
      }
    }
  }
]


Source:  https://developer.okta.com/docs/api/resources/users#list-users

Fields we need to parse

Fortunately, the available TA already parses the data for us, but the fields that we are most interested in for this lookup dashboard are the following:
  • title
  • firstName
  • middleName
  • lastName
  • email
  • primaryPhone
  • created
  • passwordChanged
  • lastLogin
  • status
Feel free to modify the search and replace fields as needed.

Search String

A simple search string that gets us the table needed is shown below. We deduplicated the results by id since it is a unique field. Now just add filters such as the ones we provided in our dashboard code at the end of the article and you are in business! 

index=okta source=okta:user | dedup id | table credentials.provider.type, profile.title, profile.firstName, profile.middleName, profile.lastName, profile.email, profile.primaryPhone, created, passwordChanged, lastLogin, status

Conclusion

Even though we had a Splunk TA and App to perform the parsing and help create visibility, we extended the usefulness of the data to build a user lookup tool. We hope this article helps others gain additional insight into their user data via Okta logs. Happy Splunking!

Dashboard Code

The following dashboard assumes that the appropriate logs are being collected and sent to Splunk. Additionally, the dashboard code assumes an index of okta. Feel free to adjust as necessary. Splunk dashboard code provided below:


<form>
  <label>Okta User Lookup</label>
  <description>index=okta source=okta:user   (First try last 24 hours, then try a longer time range)</description>
  <fieldset autoRun="true" submitButton="true">
    <input type="time" token="time">
      <label>Time Range</label>
      <default>
        <earliest>-24h@h</earliest>
        <latest>now</latest>
      </default>
    </input>
    <input type="text" token="wild">
      <label>Wildcard Search</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="first">
      <label>First Name</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="last">
      <label>Last Name</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
    <input type="text" token="email">
      <label>Email Address</label>
      <default>*</default>
      <initialValue>*</initialValue>
    </input>
  </fieldset>
  <row>
    <panel>
      <table>
        <title>Details</title>
        <search>
          <query>index=okta source=okta:user $wild$ profile.firstName=$first$ profile.lastName=$last$ profile.email=$email$ | dedup id | table credentials.provider.type, profile.title, profile.firstName, profile.middleName, profile.lastName, profile.email, profile.primaryPhone, created, passwordChanged, lastLogin, status</query>
          <earliest>$time.earliest$</earliest>
          <latest>$time.latest$</latest>
          <sampleRatio>1</sampleRatio>
        </search>
        <option name="count">10</option>
        <option name="dataOverlayMode">none</option>
        <option name="drilldown">none</option>
        <option name="percentagesRow">false</option>
        <option name="rowNumbers">false</option>
        <option name="totalsRow">false</option>
        <option name="wrap">true</option>
      </table>
    </panel>
  </row>
</form>