Tuesday, June 4, 2019

osquery - Part VI - Controlling fleetctl with fleet_helper

By Tony Lee and Matt Kemelhar

** Spoiler alert -- we are releasing fleet_helper.sh in this article **  The download is available here:
https://github.com/TonyLeeVT/fleet_helper/

This series on osquery has taken us on a journey from stand-alone agents, to managing multiple agents with Kolide Fleet, and then finally onto more advanced integrations and analysis.  We already covered the following topics:

Part I - Local Agent Interaction:  http://securitysynapse.blogspot.com/2019/05/osquery-part-i-local-agent-interaction.html
Part II - Kolide Centralized Management:  http://securitysynapse.blogspot.com/2019/05/osquery-part-ii-kolide-centralized.html
Part III - Queries and Packs:  http://securitysynapse.blogspot.com/2019/05/osquery-part-iii-queries-and-packs.html
Part IV - Fleet Control Using fleetctl - http://securitysynapse.blogspot.com/2019/05/osquery-part-iv-fleet-control-using-fleetctl.html
Part V - Integration - http://securitysynapse.blogspot.com/2019/05/osquery-part-v-integration.html


However, back in Part IV (Fleet Control Using fleetctl), we covered the fleetctl command line tool used to interface with the Fleet API.  We also mentioned the following issue:

Exporting All Queries and Packs (Previous Statement)

Unfortunately, we cannot find a single command or option within fleetctl to export all queries and packs to re-import them elsewhere.  The documentation states that fleetctl functions similarly to kubectl, but the -o yaml option does not appear to be implemented yet...

We also mentioned we would expand on this, so we are by releasing a helper tool called fleet_helper.

Exporting packs and queries is critical for sharing them and enabling faster integration with our Kolide Fleet App for Splunk - https://splunkbase.splunk.com/app/4518/#/details

We submitted a feature request (https://github.com/kolide/fleet/issues/2052), but figured we could prototype the desired functionality in bash and release it in the meantime.


fleet_helper Usage

The feet_helper tool (https://github.com/TonyLeeVT/fleet_helper/) is relatively simple and just over 200 lines (including comments).  Two of the functions are straight from the fleetctl tool, but the rest perform some sort of output manipulation or iteration over data to achieve the desired output.

Requirements:

  • fleetctl must be setup and authenticated (post fleetctl login)
  • sed, grep, tail, cut, echo commands


Help Menu:
Activate the help menu by providing no options, -h, or --help.
Usage:  ./fleet_helper.sh <function>

Note: All functions output to stdout.  Redirect to .yaml files as needed.

Possible functions:
 listpacks - Lists pack names only within the Kolide Fleet instance
 listqueries - Lists query names only within the Kolide Fleet instance
 exportpacks - Exports all packs in yaml format
 exportqueries - Exports all queries in yaml format
 exportpack <pack_name> - Exports a specified pack in yaml format
 exportquery <query_name> - Exports a specified query in yaml format
 exportpackquery <pack_name> - Exports a pack and all associated queries in yaml format
 exportall - Warning!  This exports all packs and all queries in yaml format


Listing Packs and Queries

The feet_helper can list packs and queries in simple text format using the following two functions:

 listpacks - Lists pack names only within the Kolide Fleet instance
 listqueries - Lists query names only within the Kolide Fleet instance


./fleet_helper.sh listpacks

users pack                   
osquery_info pack            
process_open_sockets pack    
programs pack                
network_connection_listening 


Exporting (All or select) Packs and Queries

The feet_helper can export all or select packs and queries using the following four functions:
 exportpacks - Exports all packs in yaml format
 exportqueries - Exports all queries in yaml format
 exportpack <pack_name> - Exports a specified pack in yaml format
 exportquery <query_name> - Exports a specified query in yaml format


./fleet_helper.sh exportqueries
apiVersion: v1
kind: query
spec:
  description: Query all users
  name: users query
  query: SELECT * FROM users
---
apiVersion: v1
kind: query
spec:
  description: Query the version of osquery
  name: osquery_info query
  query: SELECT * FROM osquery_info
--SNIP--


Exporting a Specific Pack and All Associated Queries

This feature is critical to sharing queries within a pack.  The feet_helper can export a pack and all associated queries as well using the following function:

 exportpackquery <pack_name> - Exports a pack and all associated queries in yaml format


./fleet_helper.sh exportpackquery "network_connection_listening"
apiVersion: v1
kind: pack
spec:
  id: 14
  name: network_connection_listening
  queries:
--SNIP--
---
Snaphost_Windows_Process_Listening_Port
apiVersion: v1
kind: query
spec:
  description: Returns the Listening port List - ATT&CK T1043,T1090,T1094,T1205,T1219,T1105,T1065,T1102
  name: Snaphost_Windows_Process_Listening_Port
  query: select p.name, p.path, lp.port, lp.address, lp.protocol  from listening_ports
    lp LEFT JOIN processes p ON lp.pid = p.pid WHERE lp.port != 0 AND p.name != '';


Exporting All Packs and Queries

This feature is useful for backing up packs and queries or sharing an entire environment with someone else.  The feet_helper can export all packs and queries to yaml using the following function:

 exportall - Warning!  This exports all packs and all queries in yaml format

./fleet_helper.sh exportall > exportall.yaml

We won't show the output on this function because the output is quite lengthy -- hence why we recommend redirecting the output to a file.


Conclusion

While fleetctl is a great tool, some features that assist in pack and query sharing are missing.  This quick prototype should help provide a stop gap and model new features.  Feel free to let us know about any bugs over at github.  Feel free to leave any comments below.  Thanks for reading!



No comments:

Post a Comment