# posts Published articles for posts. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## The oldest architecture in computing DevFeed: [The oldest architecture in computing]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-09-08T14:00:00Z Content type: opinion Language: en Sources: [All Things Distributed]() Topics: [Kiro](), [Artificial Intelligence](), [GPU](), [Open Source]() Tags: [agents](), [ai](), [gpu](), [kiro](), [memory](), [open-source](), [posts](), [research]() ### AI overview The article argues that AI may automate some work and change job roles, but people can remain relevant by adapting and learning to use new tools. It describes Kiro Crew, an open-source agent workspace used for research, schedule monitoring, cron-job management, and meeting preparation, with particular attention to how its independent agents manage memory and improve over time. The discussion connects these ideas to biological intelligence, including the energy efficiency of the human brain compared with a GPU and research on cortical columns. ### Source excerpt A GPU burns 700 watts and hundreds of thousands of tries to learn Pong. A human brain does it on 20 watts in a handful of attempts. So what can the oldest architecture in computing teach us about where agents and memory are headed? ## DuckDB and the changing physics of analytics DevFeed: [DuckDB and the changing physics of analytics]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-08-26T14:00:00Z Content type: opinion Language: en Sources: [All Things Distributed]() Topics: [DuckDB](), [data](), [Databases](), [Amazon S3](), [Amazon Web Services](), [Computer science]() Tags: [analytics](), [aws](), [data](), [databases](), [duckdb](), [posts](), [s3](), [software](), [systems]() ### AI overview The article describes how changing relative costs for compute, memory, and networking are enabling more data processing to remain within applications. It presents DuckDB as a database supporting this shift and discusses its relationship to AWS data offerings, including S3 Files, S3 Tables, and S3 Vectors, alongside DuckLabs joining AWS. ### Source excerpt In this post, Andy Warfield explains how databases like DuckDB are enabling a new way to build with data, why they matter right now, and how they complement the work we've been doing in S3 (e.g., S3 Files, S3 Tables, S3 Vectors). And most importantly, why DuckLabs, the team behind DuckDB, is joining AWS ## On building scalable control planes DevFeed: [On building scalable control planes]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-08-04T14:00:00Z Content type: article Language: en Sources: [All Things Distributed]() Topics: [distributed-systems](), [Amazon Web Services](), [Amazon EC2](), [DSQL](), [Cloud]() Tags: [aws](), [cloud](), [distributed-systems](), [dsql](), [infrastructure](), [kubernetes](), [posts](), [servers](), [ssh]() ### AI overview This article explores the engineering of scalable control planes through Zak van der Merwe's experience building them at AWS, first for EC2 and later for DSQL. It explains how control planes record intended state and reconcile it with actual state, bringing together difficult distributed-systems problems and influencing whether services can survive growth. ### Source excerpt Zak van der Merwe has spent his entire career at AWS building control planes. First for EC2 and now for DSQL. On the surface, the control plane looks quite boring: it records what should exist and reconciles that with what actually does. Nobody leaves school dreaming of building one, but Zak will be the first to tell you that if you like solving hard problems in distributed systems, there are few better places to be. It's where many of those hard problems converge, and where the decisions you make determine whether a service survives its own growth. ## Secret RSS Posts DevFeed: [Secret RSS Posts]() Original publisher: [Read original article]() Published: 2026-08-04T00:00:00Z Content type: opinion Language: en Sources: [Jonathan Snook]() Topics: [RSS Feed](), [Web]() Tags: [blog-post](), [posts](), [rss](), [technical]() ### AI overview The author introduces an RSS Club for posts that appear in the RSS feed without being linked from the site's home or index pages. The article explains how 11ty collections and tags control whether content appears in the RSS feed, on the site, or in both. ### Source excerpt I've seen Dave Rupert do posts that only appear in the RSS feed. He called it the RSS Club. In his words, It's like a newsletter delivered to your feed reader to celebrate the medium of RSS and breakaway from social media. I like that ethos and am choosing to do the same. What posts will qualify as part of the RSS Club remains to be seen. Maybe they'll be more personal. Maybe they'll cover topics that don't need to be directly linked from home pages and index pages. Maybe I'll get bored of the idea and we never hear of this again. And it's not like they'll be sooper-dooper secret because there will still be a web page that will be generated. You could still share and link the page, if you wanted. At least, for now. It's just that they won't be linked to from any of the index pages. Perhaps one day I'll look into creating content purely for the RSS feed that doesn't have a permalink generated for it. That'll require some tweaks to the RSS feed since it relies on the URL for things like the ID. Consider this an invitation to subscribe to the RSS feed--if you haven't already--and perhaps some interesting content will find it's way to you. Technical Bits This site is built with 11ty. Every file--okay, not every file; I'm simplifying here--gets added to a collection. Files can be added to a specific collection using tags. When I first built this site, every blog post was tagged as a post. Then, I told the home page to grab the most recent posts. And the index view to grab all of the posts. And then the RSS feed should grab the ten most recent posts. What I've done is added a new collection called rss and I changed the RSS feed to only grab items from the rss collection. tags: - post - rss This allows me to create content that only appears in the RSS feed or only appears on the site or appears in both. This system will be used for any new thing that I add. For example, if I add photography, I can create a photo tag and add the photos to the RSS feed or choose not to. I can ad ## Now Go Build CTO Fellowship: Season 2 DevFeed: [Now Go Build CTO Fellowship: Season 2]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-07-08T09:00:00Z Content type: release Language: en Sources: [All Things Distributed]() Topics: [Resilience](), [Security](), [Open Source](), [Artificial Intelligence]() Tags: [amazon](), [building](), [digital](), [open-source](), [posts](), [resilience](), [security](), [series](), [technology]() ### AI overview Amazon is releasing the second season of the Now Go Build CTO Fellowship documentary series. The five-episode season features technology leaders building projects addressing healthcare, education, food security, and other social challenges. ### Source excerpt Today, we're releasing the second season of the Now Go Build documentary series. Five episodes featuring technology leaders from around the world solving the hardest problems in healthcare and education. ## Chat SDK adds Dial support DevFeed: [Chat SDK adds Dial support]() Original publisher: [Read original article]() Author: Ben Sabic Published: 2026-07-08T00:00:00Z Content type: release Language: en Sources: [Vercel News]() Topics: [SDKs](), [Bots]() Tags: [apis](), [bots](), [documentation](), [media](), [posts](), [sdk](), [subscriptions](), [thread](), [voice]() ### AI overview Chat SDK adds a vendor-official Dial adapter for building bots that communicate through SMS, MMS, and iMessage on real phone numbers. Conversations are represented as Chat SDK threads, with support for media, inbound voice-call transcripts, standard message APIs, HMAC-verified webhooks, subscriptions, handlers, posts, and per-thread state. ### Source excerpt Chat SDK now supports Dial with the new vendor-official adapter. Build bots that send and receive SMS, MMS, and iMessage on a real phone number, with bidirectional media and inbound voice-call transcripts. Replies use the standard Chat SDK thread and message APIs, with HMAC-verified webhooks and stable per-conversation threading. Each phone conversation becomes a Chat SDK thread. Texts and media arrive as messages on that thread, and when a voice call ends, its transcript appears there too. Subscriptions, handlers, posts, and per-thread state work the same as with any other Chat SDK adapter. Read the Dial documentation to get started. Read more ## Digital Autonomy and Europe's Dependence on American Technology DevFeed: [Digital Autonomy and Europe's Dependence on American Technology]() Original publisher: [Read original article]() Published: 2026-07-02T11:37:00Z Content type: opinion Language: nl Sources: [Bert Hubert's writings]() Topics: [Security](), [Microsoft](), [Cloud](), [servers]() Tags: [autonomie](), [cloud](), [microsoft](), [posts](), [privacy](), [security](), [servers]() ### AI overview This article, adapted from a conference opening talk, argues that Europe's digital resilience and autonomy are undermined by dependence on American technology, especially Microsoft and cloud services. It contends that European organizations should consider locally operated servers and European products as alternatives. ### Source excerpt Afgelopen 25 juni deed ik het openingspraatje van de Surf Privacy en Security Conferentie. Nou heb ik vaker over digitale autonomie gesproken, maar deze keer heb ik het nadrukkelijk meer over wat er nu moet gebeuren. Of zoals de titel zegt "en nu echt". Ondanks dat ik nu letterlijk meer dan 50 praatjes over dit onderwerp heb gegeven had ik nog niet eerder de moeite genomen een goed transcript te maken. ## A return to two-pizza culture DevFeed: [A return to two-pizza culture]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-06-30T14:00:00Z Content type: article Language: en Sources: [All Things Distributed]() Topics: [amazon](), [Code]() Tags: [amazon](), [organizational](), [posts](), [process](), [product](), [writing]() ### AI overview The article revisits Amazon's two-pizza team model, emphasizing small autonomous teams, end-to-end ownership, rapid reversible decisions, experimentation, and customer-focused product development through the Working Backwards process. It also explains how organizational growth, service dependencies, and approval cycles can reduce delivery speed. ### Source excerpt What made the Quick desktop team successful is the same thing that has always produced the best work I've seen at Amazon: a small group of people who trusted each other, owned the problem end-to-end, and acted on their conviction. ## The invisible engineering behind Lambda's network DevFeed: [The invisible engineering behind Lambda's network]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-04-22T14:00:00Z Content type: article Language: en Sources: [All Things Distributed]() Topics: [flat network topology](), [networking](), [Serverless](), [Latency](), [Cloud](), [Amazon Web Services](), [systems](), [Routing (disambiguation)](), [data]() Tags: [aws](), [blog](), [blog-post](), [cloud](), [devices](), [engineering](), [infrastructure](), [kernel](), [latency](), [network](), [networking](), [posts](), [routing](), [scale](), [serverless](), [systems]() ### AI overview This article explains how AWS Lambda's networking team redesigned underlying network infrastructure over years to support latency-sensitive workloads in serverless functions. It introduces network topology and highlights optimizations involving iptables, kernel contention, packet headers, virtual devices, tunnels, routing rules, and packet filters. ### Source excerpt The subtle inventiveness that reduced cold start setup from seconds to 200μs. ## S3 Files and the changing face of S3 DevFeed: [S3 Files and the changing face of S3]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-04-07T14:00:00Z Content type: article Language: en Sources: [All Things Distributed]() Topics: [Amazon S3](), [data](), [datasets](), [systems](), [Cloud](), [amazon]() Tags: [amazon](), [blog](), [cloud](), [data](), [post](), [posts](), [s3](), [systems]() ### AI overview An article about S3 Files and the changing role of S3, framed around the difficulties of moving and managing large datasets. It describes data friction experienced in genomics research and introduces S3 Files as a solution developed by Andy Warfield's team. ### Source excerpt Andy Warfield writes about the hard-won lessons dealing with data friction that lead to S3 Files ## A chat with Byron Cook on automated reasoning and trust in AI systems DevFeed: [A chat with Byron Cook on automated reasoning and trust in AI systems]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-02-17T14:00:00Z Content type: article Language: en Sources: [All Things Distributed]() Topics: [Automated reasoning](), [Artificial Intelligence](), [AI Agent](), [Lean](), [large-language-models](), [Authorization](), [Architecture & Design]() Tags: [agentic-ai](), [ai](), [ai-agent](), [authorization](), [automated-reasoning](), [cryptographic](), [deployment](), [financial](), [generate](), [language-models](), [large-language-models](), [neural-networks](), [posts](), [reasoning](), [regulatory](), [safety](), [security](), [security-vulnerabilities]() ### AI overview An interview with Byron Cook examines how automated reasoning can establish trust in AI systems. It discusses applying formal verification techniques to agentic systems, generated code, authorization, cryptographic implementations, and virtualization, with neurosymbolic AI presented as a way to combine neural networks with mathematically rigorous symbolic procedures. ### Source excerpt Over the past decade, Byron's team has proven the correctness of our authorization engine, our cryptographic implementations, and our virtualization layer. Now they're taking those same techniques and applying them to agentic systems. ## Education is not one-size-fits-all for the latest CTO Fellows DevFeed: [Education is not one-size-fits-all for the latest CTO Fellows]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2026-01-27T14:00:00Z Content type: article Language: en Sources: [All Things Distributed]() Topics: [Learning](), [Generative AI]() Tags: [careers](), [education](), [generative-ai](), [india](), [posts](), [technology]() ### AI overview The article introduces the latest cohort of Now Go Build CTO Fellows, whose work focuses on using technology to address varied educational needs. The fellows support refugee reskilling, use generative AI to create multilingual learning materials for health workers, and develop education initiatives for rural communities with limited connectivity and access to teachers. ### Source excerpt Today, I'm honored to introduce you to the latest cohort of Now Go Build CTO Fellows. Individuals from around the world with diverse backgrounds (engineering, the humanities, business, healthcare, entrepreneurship, law) that have found their calling at the intersection of technology and education. ## A little bit uncomfortable DevFeed: [A little bit uncomfortable]() Original publisher: [Read original article]() Author: werner@allthingsdistributed.com (Dr. Werner Vogels) Published: 2025-12-16T12:30:00Z Content type: opinion Language: en Sources: [All Things Distributed]() Topics: [vulnerability](), [Computer science](), [Operating system](), [distributed-systems]() Tags: [article](), [posts](), [research](), [systems](), [vulnerability]() ### AI overview The article reflects on public-speaking anxiety and argues that discomfort and fear can signal meaningful growth. It recounts a difficult early research presentation, where criticism from the authors of Plan 9 left the writer shaken, while encouraging readers to pay attention to what currently makes them uncomfortable. ### Source excerpt Fear is actually a pretty good signal that you are pushing into the unknown, that real growth doesn't happen without a bit of that associated discomfort, and that it's worth becoming aware when it happens. Aware enough to consider actually leaning into it. ## The Secret Rules of the Terminal explains shells, emulators, escape codes, and TTY drivers DevFeed: [The Secret Rules of the Terminal explains shells, emulators, escape codes, and TTY drivers]() Original publisher: [Read original article]() Published: 2025-07-09T06:36:00Z Content type: opinion Language: en Sources: [ipSpace.net blog]() Topics: [Terminal](), [Command-line interface](), [Linux]() Tags: [blog](), [posts](), [terminal](), [worth-reading](), [writing]() ### AI overview This commentary recommends Julia Evans's The Secret Rules of the Terminal, which explores why terminal behavior varies across shells, terminal emulators, escape codes, and the TTY driver. ### Source excerpt Did you ever wonder why pressing an up-arrow in a (Linux) terminal window sometimes recalls the previous command but other times creates ^[[A? Julia Evans did, and spent months exploring the quirks of the Linux terminal (and writing blog posts describing what she found), finally resulting in The Secret Rules of the Terminal (including the various shells, terminal emulators, escape codes, and TTY driver). A must-read if you're a newbie who wants to understand why things happen the way they do. ## Generate a CMS based on your Neon Postgres schema DevFeed: [Generate a CMS based on your Neon Postgres schema]() Original publisher: [Read original article]() Author: Felipe Freitag Published: 2025-05-30T17:39:32Z Content type: article Language: en Sources: [Blog -- Neon Docs]() Topics: [Content Management System](), [Structured-data](), [Authentication](), [Authorization](), [ui](), [Image](), [Documentation]() Tags: [authentication](), [authorization](), [community](), [content](), [database](), [documentation](), [images](), [posts](), [s3](), [uploads]() ### AI overview The article discusses building CMS capabilities around an app's Neon Postgres schema to manage HTML content, images, and file uploads without introducing a separate headless CMS data source. It covers authentication, authorization, rich text editing, storage integration, and the challenges of handling uploads and file references in HTML. ### Source excerpt You created your app with Neon Postgres and love having all your data in it. Now you need to manage HTML content, images, and file uploads for your app's data, such as products, events, profiles, etc. Your marketing team has to manage blog posts, landing pages, documentation, FAQ... ## Top 10 web hacking techniques of 2024: nominations open DevFeed: [Top 10 web hacking techniques of 2024: nominations open]() Original publisher: [Read original article]() Author: James Kettle Published: 2025-01-08T14:07:27Z Content type: article Language: en Sources: [PortSwigger Research]() Topics: [Web](), [Security](), [Vulnerabilities]() Tags: [2025](), [blog](), [hacking](), [posts](), [security](), [techniques](), [web]() ### AI overview PortSwigger has opened nominations for the top 10 new web hacking techniques of 2024. The annual community process seeks innovative, practical, and reusable web security research, with community nominations and voting followed by a panel vote and publication of the finalists. ### Source excerpt Nominations are now open for the top 10 new web hacking techniques of 2024! Every year, security researchers from all over the world share their latest findings via blog posts, presentations, PoCs, an ## Testing Images in Blog Posts DevFeed: [Testing Images in Blog Posts]() Original publisher: [Read original article]() Published: 2024-03-16T00:00:00Z Content type: tutorial Language: en Sources: [Gaurav Sarma's Blog]() Topics: [Testing](), [Image](), [test]() Tags: [blog](), [blog-posts](), [images](), [posts](), [testing]() ### AI overview A blog post demonstrates testing different types and sizes of images, including remote images loaded from URLs and local images stored in a directory. ### Source excerpt Testing Different Types of Images Remote Image Here's a remote image from a URL: . [Remote test image](https://picsum... ## Threads as a Social Platform for Developers DevFeed: [Threads as a Social Platform for Developers]() Original publisher: [Read original article]() Author: Eric Elliott Published: 2023-07-13T21:32:22Z Content type: opinion Language: en Sources: [Eric Elliot]() Topics: [ui](), [image slider](), [Dark Mode]() Tags: [ai](), [app](), [dark-mode](), [design](), [developers](), [features](), [instagram](), [javascript](), [posts](), [social-media](), [sudolang](), [technology](), [thread](), [ui]() ### AI overview This opinion article reviews Threads as a text-focused social network for developers. It highlights the app's minimal interface, dark mode, 500-character posts, media and link embedding, image carousels, and threaded posts, while arguing that the platform encourages frequent participation and engineering discussions. ### Source excerpt Threads is a new social network from the Instagram team that rocketed to 100 million members in just 5 days. Obviously, Threads membership got a boost from the 2.35 billion members of Instagram, but don't write off those membership growth numbers. To be counted as a threads member, you need to download the app and create your Threads profile, which means you have to be convinced to give the app a try. And it's definitely worth a try. In fact, since I signed up, Threads has become my new social media home base. I'm active on other platforms, of course, but sharing your thoughts in brief text snippets is such a low barrier to participation, for every post on another social network, I might post dozens of times on Threads. That's a big deal. So far, Threads seems to reward and encourage engagement in ways that Twitter never did. Minimal Design Built for Engagement The Threads app features a clean, minimal UI. It respects your OS preferences, which I appreciate. Maybe I just grew fond of dark terminals and IDEs, but Dark mode just feels right to me. Feeds feed: Simple. Zero clutter. Threads delivers an experience optimized for engaging with real people in an authentic way. The text-based format with room for up to 500 characters per post lowers the barrier for meaningful back-and-forth discussion. Embedding links, photos, and short videos is seamless. The image carousel is a particularly nice touch, letting you swipe through multi-image posts easily. The carousel feature is great. One of my favorite features is the ability to create "thread storms" by stringing multiple posts together. Just hit Enter a few times to start a new entry within the thread. You can go back and edit earlier entries before posting the full thread all at once. Great for the deep dives. Editing Threads on Threads I love the attention to detail in the UI. When you see a thread, it shows a little loop in the line on the left side. A Loop in the ThreadThreads is Great for Developers I'm really excit ## Trino at Apple DevFeed: [Trino at Apple]() Original publisher: [Read original article]() Author: Vinitha Gankidi, Yathi Peddyshetty, Brian Olsen Published: 2022-11-28T00:00:00Z Content type: article Language: en Sources: [Trino Blog]() Topics: [Apache Iceberg](), [Kubernetes](), [autoscaling](), [interoperability](), [A/B Testing](), [Apache Cassandra](), [Routing (disambiguation)]() Tags: [apple](), [autoscaling](), [cassandra](), [development](), [features](), [interoperability](), [kubernetes](), [operator](), [post](), [posts](), [scale](), [spark](), [summit](), [technical](), [testing](), [use-cases]() ### AI overview This post recaps how Apple deploys and operates Trino as a self-service service for analytics, federated querying, and A/B testing. Apple uses a Kubernetes operator and custom orchestrator to manage cluster lifecycles, provisioning, autoscaling, and related operational complexity. The article highlights Apple's use of Apache Iceberg, Hive, and Cassandra connectors, contributions improving Trino and Spark interoperability and Iceberg API coverage, current challenges, and planned community contributions. ### Source excerpt This post continues a larger series of posts on the Trino Summit 2022 sessions. Following the Keynote: State of Trino session, engineers from Apple shared the current usage of Trino at Apple. They discuss how they support Trino as a service for multiple end-users, and the critical features that drew Apple to Trino. They wrap up with some challenges they have faced and some development they have planned to contribute to Trino. ## Service Architecture at SoundCloud -- Part 3: Domain Gateways DevFeed: [Service Architecture at SoundCloud -- Part 3: Domain Gateways]() Original publisher: [Read original article]() Published: 2021-09-17T00:00:00Z Content type: article Language: en Sources: [SoundCloud Backstage Blog]() Topics: [BFF](), [Authorization](), [API](), [Network](), [Protocol (disambiguation)](), [Web]() Tags: [api](), [architecture](), [article](), [authorization](), [bff](), [blog-post](), [engineering](), [microservices](), [network](), [posts](), [productivity](), [responses](), [series](), [web]() ### AI overview SoundCloud's service architecture evolved from Value-Added Services into Domain Gateways: centralized services for each business domain that support both read and write operations. The article explains challenges caused by growing aggregate fanout and varying needs across BFFs, then presents partial responses and field masks as a way to customize aggregate data while reducing unnecessary dependency fetching. ### Source excerpt This article is the last part in a series of posts aiming to cast some light onto how service architecture has evolved at SoundCloud over... ## Questions of the Week 32 DevFeed: [Questions of the Week 32]() Original publisher: [Read original article]() Author: Shai Almog Published: 2016-11-17T00:00:00Z Content type: article Language: en Sources: [CodeName One]() Topics: [Stack Overflow](), [JavaScript](), [JavaFX](), [Google Maps](), [iOS](), [pdf](), [passwords]() Tags: [bridge](), [browser](), [google-maps](), [ios](), [java](), [javascript](), [pdf](), [posts](), [weekly]() ### AI overview A weekly Codename One update covering the standard release, plans for the 3.6 release, and answers to developer questions about permissions, running projects, UI components, galleries, dialogs, JavaScript integration, Android behavior, browser features, audio capture, iOS location authorization, and large PDF downloads. ### Source excerpt I practically had a vacation from blogging these past two weeks with all the great posts from Steve. I've put some work into something interesting but it's not yet ready for prime time. If you follow the git commits you might have noticed we'll discuss it when we think it's ready. This week we (thankfully) aren't releasing a plugin update and just releasing the standard weekly release as usual. Hopefully, things will cool down a bit and we'll be able to sprint to the 3.6 release easily. ## Web archeology - itblog DevFeed: [Web archeology - itblog]() Original publisher: [Read original article]() Author: Marek Published: 2016-03-10T23:00:00Z Content type: opinion Language: en Sources: [Marek Majkowski]() Topics: [Web](), [Website]() Tags: [article](), [blog](), [posts](), [web]() ### AI overview The author revisits blog posts they published on the now-defunct Polish social network Grono.net, including posts about shortest-path search, 32-bit versus 64-bit systems, and memoization. ### Source excerpt Web archeology - itblog 11 March 2016 Long time ago I spent some time working at now defunct Polish social network - Grono.net. I published a couple of blog posts there. Allow me to performa a bit of web archeology: - Jak działa wyszukiwanie najkrótszych scieżek - 32 czy 64 bity - Memoize cichy przyjaciel programisty which coincidentally was the last article published on that blog. ## Analyzing submissions in real time for social media updates DevFeed: [Analyzing submissions in real time for social media updates]() Original publisher: [Read original article]() Published: 2016-02-02T00:00:00Z Content type: article Language: en Sources: [HackerEarth]() Topics: [real-time](), [Redis](), [Kafka](), [Database](), [App](), [Persistence](), [payload](), [data](), [coding]() Tags: [app](), [coding](), [contests](), [data](), [database](), [kafka](), [leaderboard](), [marketing](), [payload](), [persistence](), [posts](), [real-time](), [redis](), [server](), [social-media]() ### AI overview This article explains how HackerEarth built a real-time social media update system for coding and hiring challenges. It uses Redis to maintain submission counters and state, Kafka to deliver submission payloads to an asynchronous worker, database queries at selected thresholds, and scheduled or delayed posting to avoid duplicate tweets and excessive update frequency. ### Source excerpt Objective In Jan 2015, HackerEarth conducted nearly 10-12 hiring challenges, 5-6 coding challenges and numerous college challenges. HackerEarth has a decent social media presence and we wanted to inform our followers about the events at HackerEarth. One of the main objectives of this project was to provide flexibility to the marketing team to automate simple jobs and to focus on sophisticated campaigns. Design Goals As a first step, we decided to post about our events and their highlights on twitter. We covered event reminders, start/end of contests, who scored first AC and leaderboard updates at the end of a contest. We chose to do it by reading from the biggest and the meanest table of our database of Submissions. Challenges The submissions table is a very large table. An additional query on the submissions table during peak hours was not favourable. Hence, we did not count the submissions in-place and instead queued them to be processed later. Preventing duplicate tweets while maintaining state is also a challenge. Solution The application made a high volume of reads, few writes/updates. So any key/value stored would do the job. We chose Redis in lieu of Memcached. Redis offers data persistence in the event of node failure. This is very useful to avoid duplicate tweets. For instance, Two different users being credited for first AC submission in an event. By setting key expiry time and less number of keys for a single event, we prevented our Redis server from being overloaded. We maintained a key in Redis to keep count of submissions for an event. Reading from the database was not recommended because it would make a read call per submission during peak time and Redis performed faster reads. The application is an asynchronous worker and payload containing submission_id and event_id are passed to it using Kafka. So when the Redis key counter hit the magic numbers (1, 100, 500, multiples of 1000), The worker makes a DB query and posts a tweet. Worker subscribes to a ## Clojure DevFeed: [Clojure]() Original publisher: [Read original article]() Author: Keith Smiley Published: 2015-03-15T01:27:00Z Content type: article Language: en Sources: [Keith Smiley]() Topics: [Clojure](), [Vim]() Tags: [articles](), [blog](), [clojure](), [posts](), [vim]() ### AI overview A brief personal note linking several posts about Clojure, including getting started with Liberator, using Yesql, and writing Clojure in Vim. The author says they were not a big fan of Clojure and prefer typed functional languages. ### Source excerpt I haven't had anything to write in a while. Mainly because, as I've been working at thoughtbot, I've been writing everything I can on the thoughtbot blog. I spent a little while writing Clojure, and out of that came a few posts. Overall I wasn't a big fan of Clojure. I definitely lean more towards functional languages with types but I wanted to link these articles here regardless. Getting Started with Liberator Using Yesql in Clojure Writing Clojure in Vim [Next page]()