osquery - Part IV - Fleet Control Using fleetctl

By Tony Lee and Matt Kemelhar

This series on osquery will take us on a journey from stand-alone agents, to managing multiple agents with Kolide Fleet, and then finally onto more advanced integrations and analysis.  We already covered the following topics:

Part I - Local Agent Interaction:  http://securitysynapse.blogspot.com/2019/05/osquery-part-i-local-agent-interaction.html
Part II - Kolide Centralized Management:  http://securitysynapse.blogspot.com/2019/05/osquery-part-ii-kolide-centralized.html
Part III - Queries and Packs:  http://securitysynapse.blogspot.com/2019/05/osquery-part-iii-queries-and-packs.html

Now that we have a centralized osquery management platform using Kolide Fleet, we need to learn how to manage the manager.  This may not be difficult with a single Kolide Fleet instance, but keep in mind you could also run multiple load balanced Kolide Fleet instances.  Plus exporting and importing queries and packs are helpful tasks if you want to share or consume queries and packs with/from others.

Managing Fleets using fleetctl

Fortunately, there is a command-line tool that interacts with the Fleet API called fleetctl.  While it isn't perfect, at least it is a start.  This article will hopefully provide some additional tips on usage and limitations.


Official documentation can be found at the link below, but we will give you a primer with examples:  

https://github.com/kolide/fleet/blob/master/docs/cli/README.md


In order to use fleetctl, we first need to set it up (replace <hostname> with the hostname of your server):

Setup
fleetctl config set --address https://<hostname>:443

fleetctl login
Log in using the standard Fleet credentials.
Email: <my email address>
Password: 
[+] Fleet login successful and context configured!

If you see the following error message while attempting to log in, it is probably because you are using a self-signed cert without a common name:
"error logging in: POST /api/v1/kolide/login: Post https://<hostname>:443/api/v1/kolide/login: x509: certificate is not valid for any names, but wanted to match localhost"

Please see our previous article for details, however hopefully this will correct this issue for a self-signed cert:
fleetctl config set --rootca /opt/fleet/ssl/fleetserver-cert.crt

Otherwise, now that you are logged in, try something easy (this value should be familiar from the GUI):

Quick Test
fleetctl get enroll-secret
6U**********rUc


To see other options, use -h for help.

Help Menu
fleetctl -h

NAME:
   fleetctl - CLI for operating Kolide Fleet

USAGE:
   fleetctl [global options] command [command options] [arguments...]

VERSION:
   2.1.0

COMMANDS:
     apply    Apply files to declaratively manage osquery configurations
     delete   Specify files to declaratively batch delete osquery configurations
     setup    Setup a Kolide Fleet instance
     login    Login to Kolide Fleet
     logout   Logout of Kolide Fleet
     query    Run a live query
     get      Get/list resources
     config   Modify how and which Fleet server to connect to
     convert  Convert osquery packs into decomposed fleet configs
     help, h  Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --help, -h     show help
   --version, -v  print the version


Get Command Syntax (Queries, Packs, Labels, Host, Enroll-Secret)
fleetctl get
NAME:
   fleetctl get - Get/list resources

USAGE:
   fleetctl get command [command options] [arguments...]

COMMANDS:
     queries, query, q  List information about one or more queries
     packs, pack, p     List information about one or more packs
     labels, label, l   List information about one or more labels
     options            Retrieve the osquery configuration
     hosts, host, h     List information about one or more hosts
     enroll-secret      Retrieve the osquery enroll secret

OPTIONS:
   --help, -h  show help

Get A List of All Packs -- NOTE:  fleetctl does not currently provide an option to change the format
fleetctl get p
+---------------------------+----------+-------------------------------+
|           NAME            | PLATFORM |          DESCRIPTION          |
+---------------------------+----------+-------------------------------+
| users pack                |          | Query all users               |
+---------------------------+----------+-------------------------------+
| osquery_info pack         |          | Query the version of osquery  |
+---------------------------+----------+-------------------------------+
| process_open_sockets pack |          | Pack for process_open_sockets |
+---------------------------+----------+-------------------------------+
| programs pack             |          | pack for programs             |
+---------------------------+----------+-------------------------------+

(Note:  For a list of the queries, change the p to a q)

Get a Pack as Config File

For information about the config file format, please see this link:
https://github.com/kolide/fleet/blob/master/docs/cli/file-format.md

By specifying the exact query or pack in the "get" command, we get the properly formatted output, but we cannot find a way to say all packs or all queries (*).

fleetctl get p "users pack"

apiVersion: v1
kind: pack
spec:
  description: Query all users
  id: 1
  name: users pack
  queries:
  - description: ""
    interval: 60
    name: Users Query
    platform: ""
    query: users query
    removed: false
    snapshot: true
    version: ""
  targets:
    labels: null

(Note:  This also works for query names too)

Exporting All Queries and Packs

Unfortunately, we cannot find a single command or option within fleetctl to export all queries and packs to re-import them elsewhere.  The documentation states that fleetctl functions similarly to kubectl, but the -o yaml option does not appear to be implemented yet... so we had to perform some Linux trickery to get a list of all packs and queries in a non-tabled format.

To get a list of all of the packs, we used:

fleetctl get p | grep -v '+' | tail -n +2 | cut -f 2 -d '|' | sed 's/^.//' | sed -r '/^\s*$/d'

users pack                
osquery_info pack         
process_open_sockets pack 
programs pack           

To get a list of all of the queries, we just changed the first p to a q:

fleetctl get q | grep -v '+' | tail -n +2 | cut -f 2 -d '|' | sed 's/^.//' | sed -r '/^\s*$/d'

users query                
osquery_info query         
process_open_sockets query 
programs query           

We will try to expand on this when we get more time...

Import a Simple Query

If you happen to have a simple query such as the one below, you can import it using the fleetctl apply command:

cat querytoimport.yaml 

apiVersion: v1
kind: query
spec:
  description: Query the version of osquery
  name: osquery_info query
  query: SELECT * FROM osquery_info

fleetctl apply -f querytoimport.yaml
[+] applied 1 queries

This should now show up in the Web UI.

Importing Third Party Query Packs

Now that we know how to create a query pack, it does not mean that we need to create them all by ourselves.  There is already a significant amount of content being freely shared that we can leverage, such as the following:

• Mapping MITRE Att&ck Framework in osquery - https://github.com/teoseller/osquery-attck
• Palantir Security - https://github.com/palantir/osquery-configuration 
• osquery packs - https://github.com/facebook/osquery/tree/experimental/packs

Note:  Some of the content above is provided as an osquery pack and not a fleet config that can be directly imported using fleetctl apply.  For these, use fleetctl convert first.  For example:

wget https://raw.githubusercontent.com/teoseller/osquery-attck/master/network_connection_listening.conf

fleetctl convert -f network_connection_listening.conf > network_connection_listening.yaml

fleetctl apply -f network_connection_listening.yaml 
[+] applied 4 queries
[+] applied 1 packs

 More on this later... but we wanted to introduce the concept here.

Conclusion

This article covered how to manage Kolide Fleet using a tool called fleetctl.  We covered how to setup fleetctl, how to get a list of packs, queries, labels, users, and even how to import third part query packs.  You may have noticed that each article lays the groundwork to build an integration which will really weaponize Kolide Fleet...  As such, our next article we will use all of the information from the first four articles to create an integration with Splunk!  :-)