The exponential growth of data produced by the multiplication of infrastructures, applications, and network traffic has left behind the ability of most security solutions to effectively leverage this data to detect, alert, and prevent security events. As data speeds exceed an organization\'s ability to index, store, and analyze data, it is estimated that more than half of the security-related data that is collected, is not being used for analysis in the field, because it simply cannot be loaded or processed on time. This means that there are huge blind spots in operations and computer security.
The problem is that the combination of an older infrastructure that is not optimized for Splunk and the lack of training in Kubernetes results in the underutilization of oversized systems. Even for optimal solutions, indexing is measured in the range of 100 to 300 GB per day per host and still operate with very low CPU utilization, where massive additional infrastructure deployment is required to keep pace with data growth.
This excessive dimension of infrastructure means that data centers are getting as close to their capacity as IT support teams strive to scale out data ingestion, processing, storage, and analysis. Augmented infrastructure, coupled with the increasing delay in data knowledge and security, is forcing organizations to find ways to optimize their Splunk analytics delivery and consumption.
INNOVATION FROM HPE, INTEL AND SPLUNK
Solving the blind spot problem is crucial for security, but requires modernization of the application model, architecture, and consumption. Separately, these changes can be risky, but combined, they become an overwhelming task. That\'s why Hewlett Packard Enterprise (HPE), Intel, and Splunk have collaborated to solve the ingestion problem with a unique platform-as-a-service (PaaS) solution that independently scales indexers and search heads; up, down and out. It enables organizations to leverage critical data to get a complete view of the IT security landscape and multiply the value of existing investments in Splunk with efficient, correct-sized deployments.
Workload-optimized infrastructure that helps eliminate bottlenecks at scale
It helps eliminate the necessary ingestion bottleneck by facilitating optimization of HPE ProLiant DL380 Gen10 servers for concurrent indexing. HPE and Intel found the ideal configuration of Intel Xeon Scalable Processors and RAM and paired it with SSD Intel NVMe NAND local units as a cache, expandable up to 122 TB per host, for massive ingestion performance. The solution is complemented by an object store that uses HPE Scalable Object Storage with Scality RING on HPE Apollo 4500 Gen10 and Splunk SmartStore storage servers to safely scale retention independently of the hot cache with over fourteen nines (14 9) of data durability.
Containerized software to scale up
HPE and Splunk have teamed up to harness the power of HPE Ezmeral Container Platform and Kubernetes open source to bring agility and scale to the new containerized Splunk operator. This has two immediate advantages: on the one hand, the solution implements new indexers and search heads in minutes, and on the other hand, it scales them up independently on a host to completely overwhelm the infrastructure and scale out across all available information resources.
HPE GreenLake Cloud Services
With HPE GreenLake complemented by HPE Pointnext Services, the solution is provided as a service locally or in a co-located solution, managed by HPE through container and storage layers. With the HPE GreenLake model, you pay for what you consume without initial disbursement, scaling capacity vertically and horizontally. No repairs, performance adjustments or maintenance required. There is also no need to look for the limited Kubernetes training currently available, as HPE takes care of everything. In addition, this configuration provides burst options, while the containerized solution is ready for multiple clouds/hybrid cloud, in anticipation of the time to expand from the endpoint to the cloud.
KEY RESULTS OF THE SOLUTION
The HPE GreenLake Service facilitates modern containerized Splunk delivery that can take full advantage of optimized Intel Xeon scalable processors and Intel NVMe NAND units to improve utilization and performance. The HPE, Splunk, and Intel team tested concurrent searches and data model acceleration, using Splunk Enterprise Security with Intel IT production data (actual high cardinality data, i.e. non-synthetic data) from seven different data sources. By running these searches, the solution was able to scale independently from 1 to 6, and up to 12 indexers per host, resulting in an astonishing 17-bit multiplication of indexer performance, while maintaining CPU saturation below 70%.
Close collaboration between HPE, Intel, and Splunk brings to light dark data by simplifying the collection, analysis, and use of the unused value of the big data generated by your technology infrastructure, security systems, and enterprise applications, so you gain the knowledge you need to drive operations performance and improve business outcomes.