# elk Published articles for elk. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Are you checking your top two transactions? DevFeed: [Are you checking your top two transactions?]() Original publisher: [Read original article]() Author: Pedro Gil Carvalho Published: 2023-07-28T13:19:09Z Content type: opinion Language: en Sources: [Pedro Gil Carvalho]() Topics: [Application Performance Management (APM)](), [Maintainers](), [Software]() Tags: [apm](), [customer-experience](), [datadog](), [elk](), [maintainers](), [self-hosted](), [software]() ### AI overview This short engineering tip recommends periodically reviewing the one or two transactions that consume most of a software service's time. Using available APM data, engineering leaders can discuss performance with maintainers and identify architecture improvements affecting costs or customer experience. ### Source excerpt Short edition today, because not everything needs to be a ten page essay. ## SAML SSO Authentication for Splunk with G Suite DevFeed: [SAML SSO Authentication for Splunk with G Suite]() Original publisher: [Read original article]() Author: kishlaya kumar Published: 2018-04-12T07:00:30Z Content type: tutorial Language: en Sources: [CleverTap]() Topics: [saml](), [Single sign-on (SSO)](), [Authentication](), [Google](), [log management](), [elasticsearch](), [kibana](), [logstash]() Tags: [analytics](), [authentication](), [authorization](), [browser](), [elasticsearch](), [elk](), [engineering](), [google](), [kibana](), [log-management](), [logstash](), [saml](), [security](), [sso](), [technology](), [technology-engineering]() ### AI overview A step-by-step guide to configuring SAML-based single sign-on for Splunk using Google (G Suite) as the identity provider. It explains the SAML authentication model and flow, following the authors' move from ELK to Splunk for on-premises log management and analytics. ### Source excerpt From early on, our team used ELK (Elasticsearch-Logstash-Kibana) for log management and analytics. ELK served us well, but as our The post SAML SSO Authentication for Splunk with G Suite first appeared on CleverTap. ## Target and Elasticsearch: Maintaining an ELK stack over Peak Season DevFeed: [Target and Elasticsearch: Maintaining an ELK stack over Peak Season]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2017-05-25T05:00:00Z Content type: article Language: en Sources: [Target]() Topics: [elasticsearch](), [Logging](), [Cloud](), [Monitoring](), [dashboards](), [Scalability](), [Kafka]() Tags: [apache](), [apache-kafka](), [cloud](), [clusters](), [consul](), [contribute](), [dashboards](), [elasticsearch](), [elk](), [hashicorp](), [infrastructure](), [kafka](), [logging](), [logging-and-metrics](), [logs](), [make](), [metrics](), [monitoring](), [open](), [open-source](), [scalability]() ### AI overview This article describes how Target operated and evolved an Elasticsearch-based ELK stack for large-scale log aggregation, search, analytics, and multi-tenant logging during peak production periods. It discusses cloud scalability, operational challenges, and the open-source tools used alongside Elasticsearch. ### Source excerpt One of the strongest benefits of launching an application into the cloud is the pure on-demand scalability that it provides. I've had the privilege of working with the ELK stack (Elasticsearch, Logstash, Kibana) for purposes of log aggregation for the past two years. When we started at that time, we were pleased with our performance on search and query times with 10's of gigabytes of data in the cluster in production. When Peak time hit, we reveled as our production clusters successfully managed half a terabyte of data(!). During peak, Target hosted 14 Elasticsearch clusters in the cloud containing more than 83 billion documents across nearly 100 terabytes in production environments alone. Consumers of these logs are able to get access to queries in blazing fast times with excellent reliability. It wasn't always that way though, and our team learned much about Elasticsearch in the process. What's The Use Case At Target? In a word, "vast." The many teams that use our platform for log aggregation and search are often times looking for a variety of things. Simple Search This one is easy, and the least resource intensive. Simply doing a match query and searching for fields within our data. Metrics / Analytics This one can be harder to accommodate at times, but some teams use our Elasticsearch clusters for near-realtime monitoring and Analytics using Kibana dashboards. Multi-tenant Logging Not necessarily consumer facing, but an interesting use for Elasticsearch is that we can aggregate many teams and applications into one cluster. In essence, this saves money over individual applications paying for infrastructure to log themselves. Simple search is the least of our concerns here. Queries add marginal load on the cluster, but often they are one-offs or otherwise infrequently used. However, the largest challenge faced here is multi-tenant demand. Different teams have very different needs for logging/metrics; designing a robust and reliable 'one-size-fits-all' platform is ## Distributed Troubleshooting DevFeed: [Distributed Troubleshooting]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2017-04-05T05:00:00Z Content type: article Language: en Sources: [Target]() Topics: [big-data](), [incident](), [SIEM, Security, Observability](), [systems](), [elasticsearch](), [Open Source](), [telemetry]() Tags: [big-data](), [data](), [distributed](), [elasticsearch](), [elk](), [incident](), [infrastructure](), [kibana](), [logs](), [logstash](), [metrics](), [open-source](), [open-source-components](), [troubleshooting]() ### AI overview The article describes Target's Distributed Troubleshooting Platform for investigating issues across a large open source big data platform. It aggregates service logs and metrics so engineers can inspect information from many machines and services in one place, using open-source components and the Elasticsearch, Logstash, and Kibana stack. ### Source excerpt Target's open source big data platform contains a vast array of clustered technologies or ecosystems working together. Troubleshooting an issue within a single ecosystem is a difficult task let alone an issue that spans several ecosystems. It is impractical for a single human to individually investigate ecosystems one at a time for potential problems. The house will burn to the ground long before an engineer can find the cause of an issue and resolve it without quick access to aggregated system metrics and logs. The Solution How to identify, troubleshoot and resolve a distributed issue? Fight fire with fire of course! Big data issues must be solved with big data solutions. At Target, we are constantly expanding our Distributed Troubleshooting Platform to encapsulate every log and metric from every service in every ecosystem of our big data platform. Aggregating this data into a single troubleshooting platform enables an engineer to view error logs and system metrics across hundreds of machines and services with a single click. A troubleshooting platform like the one described above is not a new idea. Systems like Splunk have been doing it for years. Splunk however, has restrictions on the amount of data that can be ingested without an enterprise license. The larger we scale; the more money we pay for systems like Splunk. We created our Distributed Troubleshooting Platform from open-source components and without enterprise licenses. This allows us to utilize it on every server in the big data platform without worrying about the volume of data it is processing and re-negotiating enterprise licenses. It becomes a given, not a variable. Our Distributed Troubleshooting Platform is similar to the black box recorder on an aircraft. A majority of the time, the contents are never viewed. When the plane crashes however, the contents of the black box are the only way to reconstruct what happened and learn from the incident. Running a big data platform without enterprise licens