# nsq Published articles for nsq. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Z Proxy: Bitly's Independent Link-Redirect Fallback DevFeed: [Z Proxy: Bitly's Independent Link-Redirect Fallback]() Original publisher: [Read original article]() Author: Wordbitly Published: 2013-10-02T17:47:22Z Content type: article Language: en Sources: [Bitly]() Topics: [Availability](), [Amazon S3](), [Go Language](), [Amazon EC2](), [nsq]() Tags: [availability](), [ec2](), [go](), [nsq](), [s3]() ### AI overview Bitly describes Z Proxy, a Go application designed to keep short-link redirects available independently of its main infrastructure. It stores link mappings in S3, uses NSQ to populate them, runs on EC2, and caches S3 lookups locally with memcached. ### Source excerpt Here is a tale of how we leverage redundant datacenters, redundant code, and multi-tiered fallbacks in the quest for uptime. But Why? High availability is important for any site operating at scale, but at bitly it is particularly important; people expect bitly links to work, no matter what. We have enterprise customers who rely on them for key metrics, users who share them on social networks, and websites with custom short domains that trust us to serve requests with their name on them. A bitly link not working in any of these scenarios would make our users look bad, so it is something we take very seriously. No matter how redundant, distributed, and fault tolerant your main infrastructure is, things can always go wrong. Recently Google Apps and Search, probably the most distributed infrastructure in existence, went down. There are unknowns everywhere, and ultimately you have to plan for any part of your infrastructure breaking for unknown reasons. Under failure, a distributed system should degrade gracefully, not suddenly. This is why we created Z Proxy. So What is Z Proxy? Z Proxy is an application that serves decodes (this is what we call redirecting from a short bitly link to its long URL, and what happens every time you click on a bitly link) without relying on any other part of the bitly infrastructure. This means that it does not use our primary database of urls, or any of our other servers, to do lookups. So how does it work? How it Works Z Proxy is essentially a self contained wrapper around S3, written in Go. When all of bitly is running properly, every time a link is shortened, a message is put on NSQ, which a queuereader later grabs. A queuereader then writes the short and long urls into S3 so that Z Proxy can perform lookups against S3 by short url, get the long url, and serve a 301 or 302 redirect. To the browser, nothing different happened. There are multiple host running Z Proxy in EC2. This location provides proximity to S3, high availability, and m ## Building NSQ Client Libraries DevFeed: [Building NSQ Client Libraries]() Original publisher: [Read original article]() Author: Wordbitly Published: 2013-05-09T18:58:23Z Content type: tutorial Language: en Sources: [Bitly]() Topics: [Messaging](), [client](), [configuration](), [servers](), [HTTP]() Tags: [asynchronous](), [client-library](), [configuration](), [developers](), [distributed](), [guide](), [infrastructure](), [message-queue](), [messaging](), [nsq](), [tcp](), [thundering-herd]() ### AI overview A guide to the responsibilities and design expectations of NSQ client libraries, focusing on consumers. It covers configuration, discovery, TCP connections, message handling, pipelining, asynchronous processing, and techniques for maintaining cluster robustness and performance. ### Source excerpt Brace yourself, this is a long one. The following guide was originally intended for client library developers to describe in detail all the important features and functionality we expected in an NSQ client library. While writing it we began to realize that it had value beyond client library developers. It incorporates a comprehensive analysis of most of the capabilities of NSQ (both client and server) and is therefore interesting and useful for end-users as well (or anyone using or interested in infrastructure messaging platforms). If you need some background on NSQ please see our original blog post or its follow up, spray some NSQ on it. Intro NSQ's design pushes a lot of responsibility onto client libraries in order to maintain overall cluster robustness and performance. This guide attempts to outline the various responsibilities well-behaved client libraries need to fulfill. Because publishing to nsqd is trivial (just an HTTP POST to the /put endpoint), this document focuses on consumers. By setting these expectations we hope to provide a foundation for achieving consistency across languages for NSQ users. Overview Configuration Discovery (optional) Connection Handling Feature Negotiation Data Flow / Heartbeats Message Handling RDY State Backoff Configuration At a high level, our philosophy with respect to configuration is to design the system to have the flexibility to support different workloads, use sane defaults that run well "out of the box", and minimize the number of dials. A client subscribes to a topic on a channel over a TCP connection to nsqd instance(s). You can only subscribe to one topic per connection so multiple topic consumption needs to be structured accordingly. Using nsqlookupd for discovery is optional so client libraries should support a configuration where a client connects directly to one or more nsqd instances or where it is configured to poll one or more nsqlookupd instances. When a client is configured to poll nsqlookupd the polling int