# How Oxide Automates Infrastructure Operations with Rust, APIs, and a Database

DevFeed: [How Oxide Automates Infrastructure Operations with Rust, APIs, and a Database](<https://devfeed.tech/articles/the-automated-cio-35201.md>)

Original publisher: [Read original article](<https://blog.jessfraz.com/post/the-automated-cio/>)

Published: 2020-09-08T13:09:26Z

Content type: article

Language: en

Sources: [Jessie Frazelle](<https://devfeed.tech/sources/jessie-frazelle.md>)

Topics: [Automation](<https://devfeed.tech/topics/automation.md>), [Database](<https://devfeed.tech/topics/database.md>), [Rust](<https://devfeed.tech/topics/rust.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [automated](<https://devfeed.tech/tags/automated.md>), [automation](<https://devfeed.tech/tags/automation.md>), [caching](<https://devfeed.tech/tags/caching.md>), [code](<https://devfeed.tech/tags/code.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>)

## AI overview

The article describes Oxide's automated infrastructure operations system, which was cleaned up into a more coherent Rust-based architecture. Webhooks and cron jobs collect data from service APIs into an owned database, allowing scripts and bots to access cached information when services are unavailable. Examples include processing job applications and synchronizing RFD content.

## Source excerpt

I previously wrote a bit about our internal infrastructure in my post on The Art of Automation. This post is going to go into details about our automated Chief infrastructure Officer (CIO). I joke so much that I automated our CIO that I even named the repo holding the code... cio. I took the time this weekend to finally clean up some of this code. Previously, our infrastructure was held together with bash, popsicle sticks, glue, and some rust. Now, it is mostly rust and a much more sane architecture to grok. We also get the freedom of caching all our data in a database that we own so we can access it even when services are down. Previously, we called out to each service's API for every script, bot, or whatever, which can get expensive, slow, and potentially be riddled with rate limits, or worse, downtime. Let me give you a diagram of what this looks like now: Sending data to the database At the very bottom of the diagram, you can see where we are using webhooks and cron jobs to pull data out of various services APIs and send it to the database. Let's dive into a few of these because it is not as simple as a pipe from an API to a database in most cases. Applicants Every applicant to Oxide completes our candidate materials. This is a series of questions about things they've worked on. Those get submitted with their resume and other details into a Google Form. A cron job parses the spreadsheet from the Google Form. In doing so, it knows if an application is new and we need to send an email to the applicant that we received it. It will also send an email to the team that we got a new application. The cron job also parses the materials they submitted and their resume into plain text. Materials can be in the form of HTML, PDF, doc, docx, zip, and even PDF with zip headers ;). The resume and each question in the materials is saved in individual database columns, which makes search and indexing easier when we want to find an application based on something we remember from the