Splunk and its Components
Its all about machine data (logs) – Collect, Index and Analyze.
Splunk is a software that indexes IT machine data from any Infrastructure components such as Application, Server (Physical & Virtual) , Network Devices, Web Servers etc . Splunk platform actually aggregates and analyzes the logs that is collected and indexed from various components. It’s powerful, versatile and fast search & analysis capability serves as a critical tool to investigate, troubleshoot, monitor, alert and report everything in a IT infrastructure. In other words, It is a Single point of source to view the real-time state of the entire IT infrastructure.
Splunk Captures, indexes and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards and visualizations (Source: Wiki)
Use Splunk to:
- Continually index all of your IT data in real time.
- Automatically discover useful information embedded in your data
- Search your physical and virtual IT infrastructure for literally anything of interest and get results in seconds.
- Save searches and tag useful information to make your system smarter.
- Set up alerts to automate the monitoring of your system for specific recurring events.
- Generate analytical reports with interactive charts, graphs, tables and share them with others.
- Share saved searches and reports with fellow Splunk users, and distribute their results to team members and relevant stakeholders
- Proactively review your IT systems to head off server downtimes and security incidents before they arise.
- Design specialized, information-rich views and dashboards that fit the wide-ranging needs of your enterprise.
Splunk Enterprise performs three key functions as the data is passed through the data pipeline. First, it consumes/collects data from files, servers, networks etc. Then it parses and indexes the data (Note: parsing can be considered to be part of the indexing process) . Finally, it runs interactive or scheduled searches on this indexed data.
Below are the basic components of Splunk Enterprise in a distributed environment.
- Search heads
- Deployment server
Indexers – A Splunk Enterprise instance that indexes data, transforming raw data into events and placing the results into an index. It also searches the indexed data in response to search requests.
The indexer also frequently performs the other fundamental Splunk Enterprise functions: data input and search management. In larger deployments, forwarders handle data input and forward the data to the indexer for indexing. Similarly, although indexers always perform searches across their own data, in larger deployments, a specialized Splunk Enterprise instance, called a search head, handles search management and coordinates searches across multiple indexers.
The indexer is sometimes referred to by more specific terms, according to its context.
search peer. An indexer in a distributed search topology.
peer node. An indexer in a indexer cluster.
Forwarders – A Splunk Enterprise instance that forwards data to another Splunk Enterprise instance, such as an indexer or another forwarder, or to a third-party system.
There are three types of forwarders:
Universal forwarder – is a dedicated, streamlined version of Splunk Enterprise that contains only the essential components needed to send data.
Heavy forwarder – is a full Splunk Enterprise instance, with some features disabled to achieve a smaller footprint.
Light forwarder – is a full Splunk Enterprise instance, with most features disabled to achieve a small footprint. The universal forwarder supersedes the light forwarder for nearly all purposes.
[Note : The light forwarder has been deprecated as of Splunk Enterprise version 6.0.0.]
The universal forwarder is the best tool for forwarding data to indexers. Its main limitation is that it forwards only unparsed data. To send event-based data to indexers, you must use a heavy forwarder.
Search Head – In a distributed search environment, a Splunk Enterprise instance that handles search management functions, directing search requests to a set of search peers and then merging the results back to the user.
A Splunk Enterprise instance can function as both a search head and a search peer. A search head that performs only searching, and not any indexing, is referred to as a dedicated search head.
Search head clusters are groups of search heads that coordinate their activities.
Search heads are also required components of indexer clusters.
Deployment Server – A Splunk Enterprise instance that acts as a centralized configuration manager, grouping together and collectively managing any number of Splunk Enterprise instances. Instances that are remotely configured by deployment servers are called deployment clients. The deployment server downloads updated content, such as configuration files and apps, to deployment clients. Units of such content are known as deployment apps.