Splunk and ELK – Impartial Comparison Part I - Similarities

By Tony Lee

This series is not intended to start a “Big Data” holy war, but instead hopefully offer some unbiased insight for those looking to implement Splunk, ELK or even both platforms.  After all both platforms are highly regarded in their abilities to collect, parse, analyze, and display log data.  In fact, the first article in this series will show how the two competing technologies are similar in the following areas:

• Purpose
• Architecture
• Cost

Caveat

Most articles on this subject seem to have some sort of agenda to push folks in one direction or another—so we will do our absolute best to keep it unbiased. We admit that we know Splunk better than we know the ELK stack, so we are banking on ELK (and even Splunk) colleagues and readers to help keep us honest. Lastly, our hope is to update this article as we learn or receive more information and the two products continue to mature.

Similar Purpose

Both Splunk and ELK stack are designed to be highly efficient in log collection and search while allowing users to create visualizations and dashboards.  The similar goal and purpose of the two platforms naturally means that many of the concepts are also similar.  One minor annoyance is that the concepts are referred to by different names.  Thus, the table below should help those that are familiar with one platform map ideas and concepts to the other.

Splunk	ELK Stack
Search Head	Kibana
Indexer	Elastic Search
Forwarder	Logstash
Universal Forwarder	Beats (Filebeat, Metricbeat, Packetbeat, Winlogbeat, Auditbeat, Heartbeat, etc.)
Search Processing Language (SPL)	Lucene query syntax
Panel	Panel
Index	Index

Similar Architecture

In many ways, even the architecture between Splunk and ELK are very similar.  The diagram below highlights the key components along with the names of each component in both platforms.

Figure 1:  Architectural similarities

Cost

This is also an area where there are more similarities than most would imagine due to a misconception that ELK (with comparable features to Splunk) is free.  While the core components may be free, the extensions that make ELK an enterprise-scalable log collection platform are not free—and this is by design.  According to Shay Banon, Founder, CEO and Director of Elasticsearch:

“We are a business. And part of being a business is the belief that those businesses who can pay us, should. And those who cannot, should not be paying us. In return, our responsibility is to ensure that we continue to add features valuable to all our users and ensure a commercial relationship with us is beneficial to our customers. This is the balance required to be a healthy company.”

Elastic does this by identifying “high-value features and to offer them as commercial extensions to the core software. This model, sometimes called ‘open core’, is what culminated in our creation of X-Pack. To build and integrate features and capabilities that we maintain the Intellectual Property (IP) of and offer either on a subscription or a free basis. Maintaining this control of our IP has been what has allowed us to invest the vast majority of our engineering time and resources in continuing to improve our core, open source offerings.”

Source:  https://www.elastic.co/blog/doubling-down-on-open

That said, which enterprise-critical features aren’t included in the open source or even basic free license?  The subscription comparison screenshot found below shows that one extension not included for free is Security (formerly Shields).  This includes Encrypted communications, Role-based Access Control (RBAC), and even authentication.  Most would argue that an enterprise needs a login page and the ability to control who can edit vs. view searches, visualizations, and dashboards, thus it is not a fair comparison to say that Splunk costs money while ELK is free.  There are alternatives to X-PACK, but we will leave that to another article since it is not officially developed and maintained as part of the ELK stack.

Figure 2:  Encryption, RBAC, and even authentication is not free

In terms of host much Splunk costs vs. ELK, there are also many arguments there--some of which include the cost of build time, maintenance, etc.  It mostly depends on your skills to negotiate with each vendor.

Conclusion

Splunk and ELK stack are similar in many ways.  In fact, knowing one platform can help a security practitioner learn the other because many of the concepts are close enough to transfer.  The reduction in the learning curve is a huge advantage for those that need to convert from one platform to the other.  That said, there are differences, however we will discuss those in the next article.  In the meantime, we hope that this article was useful for you and we are open to feedback and corrections, so feel free to leave your comments below.  Please note that any inappropriate comments will not be posted—thanks in advance.  😊