Splunk and ELK – Impartial Comparison Part II - Differences

By Tony Lee

In the first part of our series (http://securitysynapse.blogspot.com/2019/02/splunk-and-elk-impartial-comparison-part-i.html), we discussed the similarities between Splunk and the ELK stack.  Part II will discuss some of the differences in terms of limitations.  Not all of these are deal breakers and they cannot necessarily be scored as one for one in terms of importance.  But it is good for folks to know the differences before implementing one platform vs. the other.  We welcome the reader to chime in with their own limitations (or corrections) as well.  We will start off with the Splunk limitations and then follow up with the ELK limitations.  Remember, these are not necessarily weighted equally in terms of importance (as that is determined by the end user), so we are not declaring a winner.

Splunk Limitations

-          ELK can easily create dynamically named indexes and keys, Splunk cannot

-          ELK can search on a wildcarded key…  For example:  search host.*=foo

-          ELK provides DevTools à Console:  a useful method for running commands against the ELK instance from the Kibana GUI

-          Splunk does not provide relevance weighting such as ELK’s _score field

ELK limitations

-          ELK does not allow piping of search commands to create more complex commands  ß This is one of the most difficult differences to overcome when transitioning from Splunk to ELK

-          Splunk is considered “Schema on read”, which means you can throw pretty much anything at it and it may autoparse or can be parsed later.  ELK requires more upfront parsing to make use of the data.

-          There is no central manager for beat agents, Splunk includes a deployment server for free which manages Universal Forwarders

-          discuss.elastic.co closes threads after 60 days of inactivity…  Splunk Answers never closes a thread and thus users can contribute at any time – this helps prevent duplicate entries and stale worthless data

-          Installation of Splunk can be completed in minutes, ELK takes much more time and is more dependent upon versions of each component since there is no unified installer

-          Kibana can only sort on numeric fields and not alphabetical fields

-          It appears that Splunk has more mathematical/statistical functions out of the box

-          ELK has a separate beat for collecting different sources/components of a system.  Splunk has a single Universal Forwarder that can collect different data sources by using a flexible configuration file.

-          ELK time range selector is missing a range for:  Quick à All time

-          ELK may introduce significant “breaking changes” on new version releases which can cause some customers to become stuck on a certain version of the platform.  Splunk seems to be very careful not to do this and it is rare and often not as limiting if it does occur.

Conclusion

This should serve as an initial list of limitations for both platforms.  Again, we will not declare a winner because some of those limitations may not matter to the end user, however it is good to get the list out in the open for discussion.  Both platforms are always looking for ways to innovate and improve the customer experience.  These lists are often a good start for that purpose and competition is definitely a good thing. If you have a correction, please keep it constructive and it will get posted in the comments section below.  Thanks for reading. 😉

Splunk and ELK – Impartial Comparison Part I - Similarities

By Tony Lee

This series is not intended to start a “Big Data” holy war, but instead hopefully offer some unbiased insight for those looking to implement Splunk, ELK or even both platforms.  After all both platforms are highly regarded in their abilities to collect, parse, analyze, and display log data.  In fact, the first article in this series will show how the two competing technologies are similar in the following areas:

• Purpose
• Architecture
• Cost

Caveat

Most articles on this subject seem to have some sort of agenda to push folks in one direction or another—so we will do our absolute best to keep it unbiased. We admit that we know Splunk better than we know the ELK stack, so we are banking on ELK (and even Splunk) colleagues and readers to help keep us honest. Lastly, our hope is to update this article as we learn or receive more information and the two products continue to mature.

Similar Purpose

Both Splunk and ELK stack are designed to be highly efficient in log collection and search while allowing users to create visualizations and dashboards.  The similar goal and purpose of the two platforms naturally means that many of the concepts are also similar.  One minor annoyance is that the concepts are referred to by different names.  Thus, the table below should help those that are familiar with one platform map ideas and concepts to the other.

Splunk	ELK Stack
Search Head	Kibana
Indexer	Elastic Search
Forwarder	Logstash
Universal Forwarder	Beats (Filebeat, Metricbeat, Packetbeat, Winlogbeat, Auditbeat, Heartbeat, etc.)
Search Processing Language (SPL)	Lucene query syntax
Panel	Panel
Index	Index

Similar Architecture

In many ways, even the architecture between Splunk and ELK are very similar.  The diagram below highlights the key components along with the names of each component in both platforms.

Figure 1:  Architectural similarities

Cost

This is also an area where there are more similarities than most would imagine due to a misconception that ELK (with comparable features to Splunk) is free.  While the core components may be free, the extensions that make ELK an enterprise-scalable log collection platform are not free—and this is by design.  According to Shay Banon, Founder, CEO and Director of Elasticsearch:

“We are a business. And part of being a business is the belief that those businesses who can pay us, should. And those who cannot, should not be paying us. In return, our responsibility is to ensure that we continue to add features valuable to all our users and ensure a commercial relationship with us is beneficial to our customers. This is the balance required to be a healthy company.”

Elastic does this by identifying “high-value features and to offer them as commercial extensions to the core software. This model, sometimes called ‘open core’, is what culminated in our creation of X-Pack. To build and integrate features and capabilities that we maintain the Intellectual Property (IP) of and offer either on a subscription or a free basis. Maintaining this control of our IP has been what has allowed us to invest the vast majority of our engineering time and resources in continuing to improve our core, open source offerings.”

Source:  https://www.elastic.co/blog/doubling-down-on-open

That said, which enterprise-critical features aren’t included in the open source or even basic free license?  The subscription comparison screenshot found below shows that one extension not included for free is Security (formerly Shields).  This includes Encrypted communications, Role-based Access Control (RBAC), and even authentication.  Most would argue that an enterprise needs a login page and the ability to control who can edit vs. view searches, visualizations, and dashboards, thus it is not a fair comparison to say that Splunk costs money while ELK is free.  There are alternatives to X-PACK, but we will leave that to another article since it is not officially developed and maintained as part of the ELK stack.

Figure 2:  Encryption, RBAC, and even authentication is not free

In terms of host much Splunk costs vs. ELK, there are also many arguments there--some of which include the cost of build time, maintenance, etc.  It mostly depends on your skills to negotiate with each vendor.

Conclusion

Splunk and ELK stack are similar in many ways.  In fact, knowing one platform can help a security practitioner learn the other because many of the concepts are close enough to transfer.  The reduction in the learning curve is a huge advantage for those that need to convert from one platform to the other.  That said, there are differences, however we will discuss those in the next article.  In the meantime, we hope that this article was useful for you and we are open to feedback and corrections, so feel free to leave your comments below.  Please note that any inappropriate comments will not be posted—thanks in advance.  😊