# Logging and metrics Published articles for Logging and metrics. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Ariadne: building a custom observability UI for personalized search DevFeed: [Ariadne: building a custom observability UI for personalized search]() Original publisher: [Read original article]() Published: 2023-06-13T01:00:00Z Content type: article Language: en Sources: [Stitch Fix]() Topics: [observability](), [ui](), [React](), [d3.js](), [Microservice](), [Logging and metrics](), [debugging]() Tags: [d3-js](), [debugging](), [development](), [logging-and-metrics](), [microservices](), [observability](), [react](), [ui]() ### AI overview Stitch Fix describes Ariadne, a custom interactive observability UI for introspecting its personalized search pipeline. Built with React, D3.js, visx, and internal libraries, the tool uses production and experimental search APIs to help teams trace search behavior, diagnose bugs, and support experimentation. ### Source excerpt In June 2022, Stitch Fix launched the Freestyle search feature, transforming how our clients discover styles that are tailored to their taste. Under the hood, personalized search is fulfilled by a pipeline of modular microservices. We designed the search system to be composable, making it easy to swap in new components and speeding up experimentation and development. Composability also made the system more observable, making it simpler to make each event traceable and reproducible with built-in logging and metrics. However, even with extensive logging, debugging a complex system such as personalized search can be quite challenging. As a hypothetical example, if a client searches for running shoes and instead sees paisley sweaters, we need to quickly diagnose where in the search pipeline the problem occurred, so that the experts in that system can work on a resolution. Where did the sweaters come from, and where are the shoes we expected to see instead? Did the query get parsed into incorrect attributes? Did the attributes get poorly matched with items in our inventory? Did the result rankings disproportionately focus on the client's past preference for sweaters and disinterest in sportswear? Understanding which service is responsible for the problem is a crucial first step in addressing it. Ideally, we want to anticipate and prevent such problems through interrogating hypothetical search scenarios before clients encounter them. Early on, we knew that slogging through system logs to understand bugs and dependencies of search would not sustain our need for iterative experimentation and collaborative development. To reduce toil and enhance the impact of the Search team, we invested in building a dedicated tool. Ariadne, named after the labyrinth expert from Greek mythology, is a custom interactive UI designed for search introspection. Ariadne is built with React and custom visualization components based on D3.js, visx, and internal libraries, and is powered by producti ## Tracing High Volume Services DevFeed: [Tracing High Volume Services]() Original publisher: [Read original article]() Author: Anton Kropp was a Software Engineer Published: 2017-09-26T12:11:36Z Content type: article Language: en Sources: [Curalate]() Topics: [tracing](), [debugging](), [Logging and metrics](), [Finagle](), [HTTP]() Tags: [debugging](), [devops](), [finagle](), [http](), [logging](), [logging-and-metrics](), [okhttp](), [scala](), [services](), [soa](), [tracing]() ### AI overview The article explains how to trace requests across a high-volume service ecosystem by propagating a trace identifier through service calls and including it in logs. It describes Curalate's use of Finagle and custom OkHttp interceptors to place the identifier in HTTP headers, then reuse it in logging contexts and structured JSON logs. ### Source excerpt We like to think that building a service ecosystem is like stacking building blocks. You start with a function in your code. That function is hosted in a class. That class in a service. That service is hosted in a cluster. That cluster in a region. That region in a data center, etc. At each level there's a myriad of challenges. From the start, developers tend to use things like logging and metrics to debug their systems, but a certain class of problems crops up when you need to debug across services. From a debugging perspective, you'd like to have a higher projection of the view of the system: a linearized view of what requests are doing. I.e. You want to be able to see that service A called service B and service C called service D at the granularity of single requests. Cross Service Logging The simplest solution to this is to require that every call from service to service comes with some sort of trace identifier. Incoming requests into the system, either from public API's or client side requests, or even from async daemon invoked timers/schedules/etc generates a trace. This trace then gets propagated through the entire system. If you use this trace in all your log statements you can now correlate cross service calls. How is this accomplished at Curalate? For the most part we use Finagle based services and the Twitter ecosystem has done a good job of providing the concept of a thread local TraceId and automatically propagating it to all other twitter-* components (yet another reason we like Finatra!). All of our service clients automatically pull this thread local trace id out and populate a known HTTP header field that services then pick up and re-assume. For Finagle based clients this is auto-magick'd for you. For other clients that we use, like OkHttp, we had to add custom interceptors that pulled the trace from the thread local and set it on the request. Here is an example of the header being sent automatically as part of Zipkin based headers (which we re-use ## Target and Elasticsearch: Maintaining an ELK stack over Peak Season DevFeed: [Target and Elasticsearch: Maintaining an ELK stack over Peak Season]() Original publisher: [Read original article]() Author: Target Brands, Inc Published: 2017-05-25T05:00:00Z Content type: article Language: en Sources: [Target]() Topics: [elasticsearch](), [Logging](), [Cloud](), [Monitoring](), [dashboards](), [Scalability](), [Kafka]() Tags: [apache](), [apache-kafka](), [cloud](), [clusters](), [consul](), [contribute](), [dashboards](), [elasticsearch](), [elk](), [hashicorp](), [infrastructure](), [kafka](), [logging](), [logging-and-metrics](), [logs](), [make](), [metrics](), [monitoring](), [open](), [open-source](), [scalability]() ### AI overview This article describes how Target operated and evolved an Elasticsearch-based ELK stack for large-scale log aggregation, search, analytics, and multi-tenant logging during peak production periods. It discusses cloud scalability, operational challenges, and the open-source tools used alongside Elasticsearch. ### Source excerpt One of the strongest benefits of launching an application into the cloud is the pure on-demand scalability that it provides. I've had the privilege of working with the ELK stack (Elasticsearch, Logstash, Kibana) for purposes of log aggregation for the past two years. When we started at that time, we were pleased with our performance on search and query times with 10's of gigabytes of data in the cluster in production. When Peak time hit, we reveled as our production clusters successfully managed half a terabyte of data(!). During peak, Target hosted 14 Elasticsearch clusters in the cloud containing more than 83 billion documents across nearly 100 terabytes in production environments alone. Consumers of these logs are able to get access to queries in blazing fast times with excellent reliability. It wasn't always that way though, and our team learned much about Elasticsearch in the process. What's The Use Case At Target? In a word, "vast." The many teams that use our platform for log aggregation and search are often times looking for a variety of things. Simple Search This one is easy, and the least resource intensive. Simply doing a match query and searching for fields within our data. Metrics / Analytics This one can be harder to accommodate at times, but some teams use our Elasticsearch clusters for near-realtime monitoring and Analytics using Kibana dashboards. Multi-tenant Logging Not necessarily consumer facing, but an interesting use for Elasticsearch is that we can aggregate many teams and applications into one cluster. In essence, this saves money over individual applications paying for infrastructure to log themselves. Simple search is the least of our concerns here. Queries add marginal load on the cluster, but often they are one-offs or otherwise infrequently used. However, the largest challenge faced here is multi-tenant demand. Different teams have very different needs for logging/metrics; designing a robust and reliable 'one-size-fits-all' platform is