# Finagle An extensible, fault-tolerant RPC system for the JVM for building high-concurrency servers. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## The End of the Public API Strangler DevFeed: [The End of the Public API Strangler]() Original publisher: [Read original article]() Published: 2022-03-14T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [BFF](), [Microservice](), [Finagle](), [Scala](), [Architecture & Design](), [Remote Procedure Call (RPC)](), [Rails](), [Ruby]() Tags: [api](), [architecture](), [bff](), [engineering](), [finagle](), [microservices](), [rpc](), [ruby](), [scala]() ### AI overview SoundCloud describes an eight-year migration of its public API from a monolithic Ruby on Rails application to a BFF using the Strangler pattern. The approach introduced a Scala service built with Finagle to proxy and augment API responses while functionality moved across microservices. ### Source excerpt This is the story of how we used the Strangler pattern to migrate our public API from a monolithic codebase to a fully fledged BFF over the... ## Did I Break You? Reverse Dependency Verification DevFeed: [Did I Break You? Reverse Dependency Verification]() Original publisher: [Read original article]() Published: 2021-05-25T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Reverse Dependencies](), [Back end](), [Tooling](), [Finagle](), [Microservice](), [Rails](), [Ruby](), [Architecture & Design](), [Documentation](), [Data Science](), [Mobile](), [PWA]() Tags: [analysis](), [automation](), [backend](), [collaboration](), [configuration](), [dependency](), [development](), [engineering](), [framework](), [jvm](), [microservices](), [rails](), [reverse-dependencies](), [ruby](), [tech-stack](), [testing](), [tooling](), [verification](), [web-development]() ### AI overview SoundCloud describes how its evolving technology stack and team autonomy created fragmentation across languages, frameworks, processes, and documentation. It presents Golden Paths as opinionated, reusable recommendations that simplify development while preserving flexibility, including a JVM backend path built around Finagle and supporting tooling. ### Source excerpt SoundCloud was founded 13 years ago, and throughout its history, the company and much of its tech stack has changed. We started with a... ## Announcing Twinagle: Twirp and Protobuf for Finagle DevFeed: [Announcing Twinagle: Twirp and Protobuf for Finagle]() Original publisher: [Read original article]() Published: 2020-06-12T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Twirp](), [Finagle](), [Remote Procedure Call (RPC)](), [Go Language](), [Microservices](), [Go](), [HTTP](), [JSON](), [API](), [Back end](), [Architecture & Design]() Tags: [announcements](), [backend](), [engineering](), [finagle](), [go](), [http](), [json](), [microservices](), [open-source](), [performance](), [rest](), [rpc](), [scala](), [twirp]() ### AI overview SoundCloud announces Twinagle, an open source implementation of the Twirp protocol for Scala/Finagle. The article explains how Twirp and Protocol Buffers can replace more flexible HTTP and JSON conventions with standardized RPC contracts, reducing API inconsistencies and potentially improving performance and development speed. ### Source excerpt A previous post on this blog ended with the following paragraph: "We might also replace JSON with a more efficient serialization protocol... ## Time Travel with JVM DevFeed: [Time Travel with JVM]() Original publisher: [Read original article]() Author: Alimate Published: 2020-02-23T00:00:00Z Content type: tutorial Language: en Sources: [Ali Dehghan - Kemikit]() Topics: [Java](), [Finagle](), [Code]() Tags: [bytecode](), [constructor](), [java](), [jvm](), [object](), [process]() ### AI overview This article demonstrates how a Java object can be allocated without calling its constructor. It explains that the object receives default field values during JVM initialization, while constructor-based initialization is skipped, and cautions against using the demonstrated approach. ### Source excerpt Let's allocate a Java object without calling its constructor! ## Tracing High Volume Services DevFeed: [Tracing High Volume Services]() Original publisher: [Read original article]() Author: Anton Kropp was a Software Engineer Published: 2017-09-26T12:11:36Z Content type: article Language: en Sources: [Curalate]() Topics: [tracing](), [debugging](), [Logging and metrics](), [Finagle](), [HTTP]() Tags: [debugging](), [devops](), [finagle](), [http](), [logging](), [logging-and-metrics](), [okhttp](), [scala](), [services](), [soa](), [tracing]() ### AI overview The article explains how to trace requests across a high-volume service ecosystem by propagating a trace identifier through service calls and including it in logs. It describes Curalate's use of Finagle and custom OkHttp interceptors to place the identifier in HTTP headers, then reuse it in logging contexts and structured JSON logs. ### Source excerpt We like to think that building a service ecosystem is like stacking building blocks. You start with a function in your code. That function is hosted in a class. That class in a service. That service is hosted in a cluster. That cluster in a region. That region in a data center, etc. At each level there's a myriad of challenges. From the start, developers tend to use things like logging and metrics to debug their systems, but a certain class of problems crops up when you need to debug across services. From a debugging perspective, you'd like to have a higher projection of the view of the system: a linearized view of what requests are doing. I.e. You want to be able to see that service A called service B and service C called service D at the granularity of single requests. Cross Service Logging The simplest solution to this is to require that every call from service to service comes with some sort of trace identifier. Incoming requests into the system, either from public API's or client side requests, or even from async daemon invoked timers/schedules/etc generates a trace. This trace then gets propagated through the entire system. If you use this trace in all your log statements you can now correlate cross service calls. How is this accomplished at Curalate? For the most part we use Finagle based services and the Twitter ecosystem has done a good job of providing the concept of a thread local TraceId and automatically propagating it to all other twitter-* components (yet another reason we like Finatra!). All of our service clients automatically pull this thread local trace id out and populate a known HTTP header field that services then pick up and re-assume. For Finagle based clients this is auto-magick'd for you. For other clients that we use, like OkHttp, we had to add custom interceptors that pulled the trace from the thread local and set it on the request. Here is an example of the header being sent automatically as part of Zipkin based headers (which we re-use ## Inside a SoundCloud Microservice DevFeed: [Inside a SoundCloud Microservice]() Original publisher: [Read original article]() Published: 2017-07-28T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Microservice](), [Microservices](), [Architecture & Design](), [Back end](), [Finagle](), [Scala](), [API](), [JSON](), [Network](), [RSS Feed](), [telemetry](), [Messaging]() Tags: [api](), [architecture](), [backend](), [configuration](), [finagle](), [functional-programming](), [json](), [messaging](), [microservices](), [microservices-architecture](), [network](), [rss](), [scala](), [telemetry]() ### AI overview The article describes SoundCloud's transition from a monolithic backend to independently built and deployed microservices. It explains how the Creators Team implements services in Scala with Finagle, shared libraries, APIs, JSON or Thrift communication, configuration injection, telemetry, and service-to-service request aggregation. ### Source excerpt If you're a regular visitor to this blog, you might be aware that we have been transitioning to a microservices based architecture over the past four to five years, as we have shared insights into the process and the related challenges on multiple occasions. To recap, adopting a microservices architecture has allowed us to regain team autonomy by breaking up our monolithic backend into dozens of decoupled services, each encapsulating a well defined portion of our product domain. Every service is built and deployed individually, communicating with other services over the network via light-weight data interchange formats such as JSON or Thrift. What we haven't touched on so far is how a microservice at SoundCloud looks backstage. ## Microservices and the monolith DevFeed: [Microservices and the monolith]() Original publisher: [Read original article]() Published: 2016-08-26T00:00:00Z Content type: opinion Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Architecture & Design](), [Microservice](), [microservices architecture](), [HTTP](), [Prometheus](), [Docker](), [Kubernetes](), [Finagle](), [telemetry](), [Testing]() Tags: [architecture](), [docker](), [finagle](), [http](), [kubernetes](), [microservices](), [microservices-architecture](), [monolithic-architecture](), [prometheus](), [telemetry](), [testing]() ### AI overview SoundCloud reviews its migration from a monolithic architecture to microservices and evaluates the strategy needed to decommission the monolith. The article reports that internal tools and libraries improved service creation and engineering productivity, while extraction projects remained difficult because they took months and depended on multiple teams. ### Source excerpt In a previous series of blog posts, we covered our decision to move away from a monolithic architecture, replacing it with microservices, interacting synchronously with each other over HTTP, and asynchronously using events. In this post, we review our progress toward this goal, and talk about the conditions and strategy required to decommission our monolith. ## Synchronous communication for microservices: current status and learnings DevFeed: [Synchronous communication for microservices: current status and learnings]() Original publisher: [Read original article]() Published: 2016-07-27T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Microservice](), [Architecture & Design](), [Finagle](), [Remote Procedure Call (RPC)](), [Scala](), [HTTP](), [JSON](), [Netty](), [Scalability](), [Prometheus](), [MySQL](), [Routing (disambiguation)]() Tags: [architecture](), [blog-post](), [finagle](), [framework](), [http](), [json](), [library](), [microservices](), [microservices-architecture](), [mysql](), [programming](), [prometheus](), [rpc](), [scala](), [scalability]() ### AI overview This article describes SoundCloud's use of synchronous request-response communication in a microservices architecture. It discusses Finagle and Scala-based services, the internal jvmkit library, non-blocking I/O, service discovery, routing, logging, Prometheus integration, and the use of HTTP/1.1 with JSON. It also explains the interoperability and debugging benefits of HTTP and JSON, along with their greater overhead compared with more optimized protocols. ### Source excerpt Since we started breaking our monolith and introduced a microservices architecture we rely a lot on synchronous request-response style communication. In this blog post we'll go over our current status and some of the lessons we learned. ## Building Products at SoundCloud--Part III: Microservices in Scala and Finagle DevFeed: [Building Products at SoundCloud--Part III: Microservices in Scala and Finagle]() Original publisher: [Read original article]() Published: 2014-06-13T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [Finagle](), [Node.js]() Tags: [architecture](), [clojure](), [concurrency](), [finagle](), [http](), [microservices](), [node-js](), [resilience](), [rpc](), [ruby](), [scala]() ### AI overview SoundCloud describes its move from a Ruby on Rails monolith to microservices, consolidation around JVM languages, and evaluation of Netty, Netflix, and Finagle stacks for RPC, resilience, and concurrency. ### Source excerpt In the first two parts of this series, we talked about how SoundCloud started breaking away from a monolithic Ruby on Rails application into... ## 100,000 strong - CodeFactory server DevFeed: [100,000 strong - CodeFactory server]() Original publisher: [Read original article]() Published: 2013-03-12T00:00:00Z Content type: article Language: en Sources: [HackerEarth]() Topics: [C](), [C++](), [Java](), [Finagle](), [Remote Procedure Call (RPC)](), [Processes](), [servers](), [Development]() Tags: [c](), [c-plus-plus](), [development](), [distributed](), [distributed-system](), [ipc](), [java](), [process](), [server]() ### AI overview This article recounts the development of HackerEarth's CodeFactory server. It describes an early C-based code checker that compiled and ran C, C++, and Java programs, including process execution, inter-process communication, privilege limitations, and retry handling on a shared server. The article then explains the adoption of Apache Thrift and the implementation of a client-server architecture for distributed, cross-language services, followed by expansion of the evaluation engine to languages such as Perl, PHP, Python, Ruby, and Haskell. ### Source excerpt The Inception January 2012 was an idyllic time for us. Three of us had just teamed up to build something cool. There was no planning for the future, no sort of agreement - we were just three geeks sitting in the dorm room who wanted to build a product. We started working on MyCareerStack where there was supposed to be interview questions, tutorials etc. Soon, we realized that there was code editor needed, but that was easy. The harder part was that people don't only want to write code online, they want to run them. Now that I had never done before! What resulted then was a hacky few hundred lines of code in C, which could compile and run code in just C, C++ and Java. It might have been even the worst piece of code I had ever written, but it worked and it was sweet! /* create FIFO to be used here & later for IPC. */ char fifo_1[NAME_MAX]; createFilePath(fifo_1, id, dir, FIFO_FILE_1); umask(0); /* reset file creation mask. */ int readfd_1, writefd_1, readfd_2; if (mkfifo(fifo_1, S_IRWXU | S_IRWXG | S_IRWXO) < 0) err(1, NULL); /* Fork a child to exec the process: {id}_a.out */ pid_t pid; int status; signal(SIGALRM, handle_tle); /* change current working directory to mycareerstack */ chdir(RUNTIME_DIR); Wait, you haven't seen the ugly part yet! This will make you cringe, it made me too. See what the hell I am doing with re-tries, but it was required! int tries = 0; /* try the exec 100 times. */ while(tries < 100) { /* run the program. */ if(execl(executable, executable, (char*) 0) < 0) { // } ++tries; } printf("Couldn't run the program!"); _exit(0); There were some serious reasons for which this hack was done. We were hosted on a shared webfaction server, with no access to root. This meant I couldn't drop the user process privilege to any user with limited access permissions. I needed to create another user then with limited access which could be controlled by the original user. And again, I didn't know of any way to know if there is already a process running with privi