# Benchling

The official blog of the Benchling engineering team. - Medium

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Infra at Scale: Agents, Tools and Infra-as-Code

DevFeed: [Infra at Scale: Agents, Tools and Infra-as-Code](<https://devfeed.tech/articles/infra-at-scale-agents-tools-and-infra-as-code-122088.md>)

Original publisher: [Read original article](<https://benchling.engineering/infra-at-scale-agents-tools-and-infra-as-code-105645ea7b58?source=rss----3d4aa8fb07ea---4>)

Author: Brendan Neutra

Published: 2026-11-06T21:51:13Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [agentic workflows](<https://devfeed.tech/topics/agentic-workflows.md>), [Travis CI](<https://devfeed.tech/topics/travis-ci.md>), [GitOps](<https://devfeed.tech/topics/gitops.md>), [Pull Request](<https://devfeed.tech/topics/pull-request.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Slack](<https://devfeed.tech/topics/slack.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agent](<https://devfeed.tech/tags/ai-agent.md>), [ci-cd](<https://devfeed.tech/tags/ci-cd.md>), [clusters](<https://devfeed.tech/tags/clusters.md>), [github](<https://devfeed.tech/tags/github.md>), [human-review](<https://devfeed.tech/tags/human-review.md>), [iac](<https://devfeed.tech/tags/iac.md>), [infra](<https://devfeed.tech/tags/infra.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [kubernetes](<https://devfeed.tech/tags/kubernetes.md>), [terraform](<https://devfeed.tech/tags/terraform.md>), [tools](<https://devfeed.tech/tags/tools.md>)

### AI overview

Benchling uses scheduled Cursor Cloud Agents to raise staged Terraform PRs for routine Kubernetes component updates, increasing rollout cadence about fivefold across more than thirty components. Version constants keyed by development, staging, and production compile into per-cluster variables, so each stage's update is a one-line change; PRs advance through stages after testing and bake periods. Per-component configs guide release detection, compatibility research, health checks, and expected Terraform changes. Agents can run plans and test in a development sandbox but need no privileged production access. An authorized engineer reviews PRs, and production runs still require review: a Plan Matcher lets the Reviewer service apply only Terraform changes matching the approved expression; unmatched changes wait for human review. The article's example shows an agent identifying a release-note warning that applied to Benchling's configuration, preserving a controller default and verifying it in development. Staging is intended to limit production risk if an agent misses a breaking change.

### Source excerpt

In a previous post, How We Run Terraform At Scale, we shared how Benchling moved from a painful, laptop-driven, apply-then-merge workflow to an automated infrastructure release system built on Terraform Cloud (TFC). That migration dramatically reduced conflicts and drift, saved an estimated 8,000 developer hours annually, and gave us the foundation to scale from 350 to 625 workspaces managing many thousands of cloud resources across 38 AWS accounts. We also walked through the ecosystem of custom tooling we built around TFC to make it work at our scale: a CLI for bulk review and approval, Slack notifications for pending runs, and automated workspace dependency mapping. Benchling continues to expand both in platform capabilities and geographical reach (close to a thousand Terraform workspaces!). This post will discuss how we leverage AI agents to operate faster and more safely than ever before. We will focus on how we now roll out routine updates to our Kubernetes clusters. What's good for humans is good for agents Before we dive into our AI agent workflows it's important to state what has not changed: Infrastructure as Code (IaC). It turns out the values we embraced to ensure human change management was safe still hold true when AI agents are introduced into the mix: Modular, declarative Terraform: changes at scale are clearly represented in code Changes require authorized human review/approval in a GitHub Pull Request (PR) before merge Terraform Cloud (or any CI/CD) provides a secure environment where approved changes are rolled out automatically Since our customers and their work are our highest priority, it was critical that AI agents be incorporated into this process in the safest manner possible, with a human in the loop. So this is where we landed: AI agents can produce PRs but an authorized engineer must review and approve them. This is a great fit because the majority of effort lies in understanding the required change, making the right code change and testin

## Fragmentation to framework: Spec-first development at Benchling

DevFeed: [Fragmentation to framework: Spec-first development at Benchling](<https://devfeed.tech/articles/fragmentation-to-framework-spec-first-development-at-benchling-20126.md>)

Original publisher: [Read original article](<https://benchling.engineering/fragmentation-to-framework-spec-first-development-at-benchling-9b97302bddcf?source=rss----3d4aa8fb07ea---4>)

Author: Eli Levine

Published: 2026-02-19T17:29:16Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [software-development](<https://devfeed.tech/topics/software-development.md>), [Spec Driven Development](<https://devfeed.tech/topics/spec-driven-development.md>), [benchling](<https://devfeed.tech/topics/benchling.md>), [development-process](<https://devfeed.tech/topics/development-process.md>), [Python](<https://devfeed.tech/topics/python.md>)

Tags: [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [analytics](<https://devfeed.tech/tags/analytics.md>), [api](<https://devfeed.tech/tags/api.md>), [apis](<https://devfeed.tech/tags/apis.md>), [architectures](<https://devfeed.tech/tags/architectures.md>), [benchling](<https://devfeed.tech/tags/benchling.md>), [biotechnology](<https://devfeed.tech/tags/biotechnology.md>), [capabilities](<https://devfeed.tech/tags/capabilities.md>), [data-warehouse](<https://devfeed.tech/tags/data-warehouse.md>), [development](<https://devfeed.tech/tags/development.md>), [dna](<https://devfeed.tech/tags/dna.md>), [domain](<https://devfeed.tech/tags/domain.md>), [enterprise](<https://devfeed.tech/tags/enterprise.md>), [entries](<https://devfeed.tech/tags/entries.md>), [field](<https://devfeed.tech/tags/field.md>), [framework](<https://devfeed.tech/tags/framework.md>), [functionality](<https://devfeed.tech/tags/functionality.md>), [indexing](<https://devfeed.tech/tags/indexing.md>), [integrate](<https://devfeed.tech/tags/integrate.md>), [platform-engineering](<https://devfeed.tech/tags/platform-engineering.md>), [spec-first-development](<https://devfeed.tech/tags/spec-first-development.md>)

### AI overview

Benchling's Object Framework replaces object-by-capability integrations with a shared contract: define each object once so platform capabilities can work across objects generically. Its Domain Graph, written in GraphQL SDL, defines types, fields, relationships, and metadata; Python Data Connectors expose standardized CRUD and domain-specific operations; Data Mappers translate between domain objects and storage. In the Bulk Import API example, platform code handles chunking, job distribution, and per-record results without object-specific integrations, while newly onboarded objects work without additional integration. Benchling says this improves consistency across APIs and the warehouse; several V3 capabilities are listed as beta or internal. Adoption depends on cross-team migration work, and teams must balance flexibility with stability because promotion can lock in fields and behavior. The project is ongoing.

### Source excerpt

Reaching the limit of manual platform development Benchling's platform handles diverse scientific data, such as DNA sequences, antibodies, notebook entries, inventory containers, workflow runs, and dozens more. Each object type carries unique domain logic: how it is validated, what relationships it holds, and what actions users can perform on it. As Benchling matured, capabilities were added that customers expected to work across all these objects including REST APIs for integration, a data warehouse for analytics, search indexing, and configuration migration tools for moving setups between tenants, among many others. Each product team is expected to expose their data in all platform surface areas. However, because this process is manual it can also be brittle and costly. With M object types and N platform capabilities, and each object requires custom integration with each capability, you're maintaining MxN integration points. Add a new object? You'll need to integrate it with every platform capability. Add a new capability? You'll need to integrate it with every object. In practice, this meant product and tech debt: some objects were available via API but missing from the warehouse, or a feature was exposed in the UI but not in other platform surface areas. It also meant behavioral drift. The same object would have slightly different field names or validation logic depending on which surface you accessed it through. As Benchling grew, so did its customers. Enterprise customers expect platforms that are designed for multi-modal integration that covers the full spectrum of Benchling's data and functionality. AI is quickly reshaping how knowledge work is done across all industries. But some fundamentals have not shifted. The same integration capabilities that make enterprise architectures more powerful are what makes agents more powerful too: both require data access and interoperability. Thus a different approach was needed. Ideally one where the cost of adding types

## The Multi-Model Playbook

DevFeed: [The Multi-Model Playbook](<https://devfeed.tech/articles/the-multi-model-playbook-20131.md>)

Original publisher: [Read original article](<https://benchling.engineering/the-multi-model-playbook-20d5fba48562?source=rss----3d4aa8fb07ea---4>)

Author: Sumedh Bhattacharya

Published: 2026-01-16T16:02:06Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [Prompt Engineering](<https://devfeed.tech/topics/prompt-engineering.md>), [AI Context Management](<https://devfeed.tech/topics/ai-context-management.md>), [AI-assisted coding](<https://devfeed.tech/topics/ai-assisted-coding.md>), [AI Integration Strategies](<https://devfeed.tech/topics/ai-integration-strategies.md>), [agentic-coding](<https://devfeed.tech/topics/agentic-coding.md>), [Large Language Model](<https://devfeed.tech/topics/llm.md>), [Code Modularity](<https://devfeed.tech/topics/code-modularity.md>), [context window](<https://devfeed.tech/topics/context-window.md>), [Multi-step AI processes](<https://devfeed.tech/topics/multi-step-ai-processes.md>), [AI Chat](<https://devfeed.tech/topics/ai-chat.md>), [CI/CD](<https://devfeed.tech/topics/cicd.md>), [Test-driven development](<https://devfeed.tech/topics/tdd.md>), [Claude](<https://devfeed.tech/topics/claude.md>)

Tags: [agent](<https://devfeed.tech/tags/agent.md>), [agentic-engineering](<https://devfeed.tech/tags/agentic-engineering.md>), [agents](<https://devfeed.tech/tags/agents.md>), [ai](<https://devfeed.tech/tags/ai.md>), [ai-agents](<https://devfeed.tech/tags/ai-agents.md>), [ai-coding](<https://devfeed.tech/tags/ai-coding.md>), [ai-coding-assistants](<https://devfeed.tech/tags/ai-coding-assistants.md>), [ai-systems](<https://devfeed.tech/tags/ai-systems.md>), [amazon](<https://devfeed.tech/tags/amazon.md>), [amazon-nova](<https://devfeed.tech/tags/amazon-nova.md>), [analysis](<https://devfeed.tech/tags/analysis.md>), [anthropic](<https://devfeed.tech/tags/anthropic.md>), [anthropic-claude](<https://devfeed.tech/tags/anthropic-claude.md>), [api](<https://devfeed.tech/tags/api.md>), [api-keys](<https://devfeed.tech/tags/api-keys.md>), [apply](<https://devfeed.tech/tags/apply.md>), [benchling](<https://devfeed.tech/tags/benchling.md>), [best](<https://devfeed.tech/tags/best.md>), [best-practices](<https://devfeed.tech/tags/best-practices.md>), [biotechnology](<https://devfeed.tech/tags/biotechnology.md>), [building](<https://devfeed.tech/tags/building.md>), [llm](<https://devfeed.tech/tags/llm.md>), [model](<https://devfeed.tech/tags/model.md>)

### AI overview

Reliable multi-provider AI systems depend on modular task design and stable interfaces, not just interchangeable API keys. In a PDF import example, splitting transcription into parallel chunks with Haiku 4.5 and combining them with MapReduce was more accurate, faster, and cheaper than asking Sonnet 4.5 to process the whole file. For repeated completions, keep the system prompt unchanged and vary user prompts to preserve cache matches; use indexed tables for larger datasets, JSON for nested data, and XML tags to separate source files in context. The provider comparisons are specific to Benchling's use cases: Anthropic models scored strongly on accuracy, speed, and cost; Gemini was accurate with structured data but slow; OpenAI was fast and produced well-structured JSON but could miss nuanced instructions; Llama struggled with large JSON responses; and Nova had the weakest scores and JSON reliability. For coding assistants, the author suggests planning and modular execution when context is limited, then detailed instructions and close review for scoped production work. These observations reflect the author's systems and evaluations, not universal rankings.

### Source excerpt

The Multi-Model Playbook: Patterns in Agentic Engineering Building production AI systems that work reliably across multiple model providers requires more than just swapping API keys. Over the past year, working on AI agents like the Data Entry Agent and Compose Agent at Benchling, I've learned that successful multi-provider strategies come down to understanding what's universal versus what's provider-specific, and designing around those constraints. The clearest revelation here was that the architectural principles underlying reliable software -- modularity, separation of concerns, clear interfaces -- apply just as fundamentally to AI systems as they do to traditional code. The Data Entry Agent (DEA) extracts structured data from PDFs and images, while Compose is an agent that helps scientists write electronic lab notebooks (ELNs) by extracting content from attached files, connecting that with data in Benchling's Registry, and outputting structured scientific protocols, analysis, and more. These systems currently support five different model families (OpenAI GPT, Anthropic Claude, Google Gemini, Meta Llama, and Amazon Nova), typically using four in any given run. This experience has revealed patterns that hold true across providers -- patterns around task decomposition, prompt structure, caching strategies, and data presentation. While each provider has its quirks, these foundational strategies have proven consistently effective. In this post, I'll cover: How to break down problems for optimal LLM performance Why the system versus user prompt distinction matters for caching Best practices for presenting structured data as context Practical comparisons between model providers How to apply these principles when using AI coding assistants. Breaking Down Problems: Small & Complex versus Large & Simple LLMs lose accuracy when handling multiple separate tasks simultaneously or when operating on large input contexts. The sweet spot is to give them either a small, complex task

## How We Run Terraform At Scale

DevFeed: [How We Run Terraform At Scale](<https://devfeed.tech/articles/how-we-run-terraform-at-scale-20127.md>)

Original publisher: [Read original article](<https://benchling.engineering/how-we-run-terraform-at-scale-da7bb75dc394?source=rss----3d4aa8fb07ea---4>)

Author: Christian Monaghan

Published: 2025-03-04T16:32:39Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [Terraform](<https://devfeed.tech/topics/terraform.md>), [Amazon Web Services](<https://devfeed.tech/topics/aws.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>), [Amazon DynamoDB](<https://devfeed.tech/topics/amazon-dynamodb.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [cloud-infrastructure](<https://devfeed.tech/tags/cloud-infrastructure.md>), [dynamodb](<https://devfeed.tech/tags/dynamodb.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [infrastructure-as-code](<https://devfeed.tech/tags/infrastructure-as-code.md>), [management](<https://devfeed.tech/tags/management.md>), [operations](<https://devfeed.tech/tags/operations.md>), [s3](<https://devfeed.tech/tags/s3.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>), [terraform](<https://devfeed.tech/tags/terraform.md>)

### AI overview

Benchling describes how it manages more than 165,000 cloud resources across 625 Terraform workspaces and 38 AWS accounts. The article explains that its earlier laptop-based Terraform workflow, using S3 state files and DynamoDB locks, became inefficient as the number of workspaces grew.

### Source excerpt

Managing over 165k cloud resources across hundreds of workspaces could seem daunting. But for us, it's just another day at Benchling. Here's how we do it. We currently have: 165k cloud resources under management 625 Terraform workspaces 38 AWS accounts 170 engineers (40 of whom are infra specialists) We perform: 225 infrastructure releases daily (terraform apply operations) 723 plans daily (terraform plan operations) We've been successfully operating Benchling's infrastructure release system for the past two years (spoiler, it's Terraform Cloud), over which time we've doubled our infrastructure footprint with minimal additional release overhead. Before Terraform Cloud: The Chaos Our infra release process wasn't always this smooth. Let me rewind and take you back to how it was before. As is common guidance for small Terraform projects, our team would previously apply all infrastructure changes via laptop. Also in line with common guidance, our team used S3 to store state files, with DynamoDB state locks, which prevented any apply-time collisions. This is a great strategy for a small team working on up to a dozen workspaces. However, this slowly starts to break down as the team's workspace footprint grows. It's like the proverbial frog in the pot of water, slowly heated to a boil. By the time we made the switch, Benchling was managing 350 workspaces. We were approaching the boiling point. Pain Points: Developer Toil and Inefficiency Managing 350 workspaces with this approach had several downsides: Necessitated elevated AWS access permissions for the infrastructure team. It was time-consuming as the engineer had to navigate to each directory, run terraform apply, review and approve the run, then verify it succeeded. Very commonly a single change could affect over 120 workspaces, which would mean repeating this process 120 times. (We had developed a custom python script which helped parallelize this somewhat.) Accumulated infra drift. Often an engineer would go to apply

## Scaling Scientific Data: Migrating Benchling's Schema Model for Performance at Scale

DevFeed: [Scaling Scientific Data: Migrating Benchling's Schema Model for Performance at Scale](<https://devfeed.tech/articles/scaling-scientific-data-migrating-benchling-s-schema-model-for-performance-at-scale-20129.md>)

Original publisher: [Read original article](<https://benchling.engineering/scaling-scientific-data-migrating-benchlings-schema-model-for-performance-at-scale-2a91cf971040?source=rss----3d4aa8fb07ea---4>)

Author: Melody Ding

Published: 2024-12-04T15:01:06Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [benchling](<https://devfeed.tech/topics/benchling.md>), [schema design](<https://devfeed.tech/topics/schema-design.md>)

Tags: [accessible](<https://devfeed.tech/tags/accessible.md>), [ai](<https://devfeed.tech/tags/ai.md>), [approach](<https://devfeed.tech/tags/approach.md>), [at-the-core](<https://devfeed.tech/tags/at-the-core.md>), [automate](<https://devfeed.tech/tags/automate.md>), [automated](<https://devfeed.tech/tags/automated.md>), [benchling](<https://devfeed.tech/tags/benchling.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [challenges](<https://devfeed.tech/tags/challenges.md>), [configure](<https://devfeed.tech/tags/configure.md>), [customer](<https://devfeed.tech/tags/customer.md>), [customers](<https://devfeed.tech/tags/customers.md>), [data](<https://devfeed.tech/tags/data.md>), [data-collection](<https://devfeed.tech/tags/data-collection.md>), [data-ingestion](<https://devfeed.tech/tags/data-ingestion.md>), [data-modeling](<https://devfeed.tech/tags/data-modeling.md>), [database-optimization](<https://devfeed.tech/tags/database-optimization.md>), [model](<https://devfeed.tech/tags/model.md>), [performance](<https://devfeed.tech/tags/performance.md>), [postresql](<https://devfeed.tech/tags/postresql.md>), [scalable-architecture](<https://devfeed.tech/tags/scalable-architecture.md>), [scale](<https://devfeed.tech/tags/scale.md>), [scaling](<https://devfeed.tech/tags/scaling.md>), [schema](<https://devfeed.tech/tags/schema.md>), [software-engineering](<https://devfeed.tech/tags/software-engineering.md>)

### AI overview

Benchling replaced its Entity-Attribute-Value storage for schema field values with PostgreSQL JSONB, placing each entity's attributes in one row. This reduced row overhead and sped up bulk reads and writes; Benchling reports up to 7x faster assay-result ingestion, with improvements varying by workload and data volume. JSONB made some field queries more complex and increased contention when concurrent updates touched different fields on the same entity. Rather than add a GIN index that would slow inserts, Benchling kept entity linkages in a separate table. The three-year migration used dual writes and nightly integrity checks, switched reads to JSONB while checks continued, then deprecated the old tables. The team also addressed race conditions, deadlocks, and looser type handling during the transition.

### Source excerpt

Benchling is a unified platform for scientific data. It allows scientists to collaborate on complex science, automate work, and power AI. Customers store large volumes of data on our platform, leveraging it across many applications both within Benchling and in their own infrastructure. It's critical that customer data is accessible in a performant and scalable way. In this article, we'll explore a recent shift in how we store and retrieve customer data. By migrating to a more compact structure, we've tackled key performance challenges associated with increased data volumes. This transition has required a careful balance between speed and flexibility, as well as a phased approach that minimized disruption for users. Benchling Schemas At the core of Benchling's system is Schemas, a product that allows both Benchling internal teams and customers to configure the various shapes of data, defining fields, attributes, and constraints that entities must follow. These data structures represent entities like equipment, storage, biological molecules, workflows, tasks, lab notes, and recorded results from scientific tests. Schemas reside in what we refer to as the definition layer. An example schema for defining the data structure of a molecule Each instance of a schema, referred to as a schematizable item, represents the actual data input by scientists. We call this the instance layer. These items are populated with field values conforming to the schema's defined fields. As Benchling's user base grows and the amount of schematized data ingested into the platform increases every year, optimizing the storage of field values has become crucial. Relationship between actual instances of a molecule and its defined schemaThe Challenge: Scale and Performance Historically, Benchling saw a shift from manual data upload by scientists to integrations with lab equipment, leading to automated data collection. This significantly increased the speed and volume of data ingestion. Assay results

## A behind-the-scenes look at building interactive analysis capabilities in Benchling

DevFeed: [A behind-the-scenes look at building interactive analysis capabilities in Benchling](<https://devfeed.tech/articles/a-behind-the-scenes-look-at-building-interactive-analysis-capabilities-in-benchling-20123.md>)

Original publisher: [Read original article](<https://benchling.engineering/a-behind-the-scenes-look-at-building-interactive-analysis-capabilities-in-benchling-fa6ec1bab1e5?source=rss----3d4aa8fb07ea---4>)

Author: Wonja Fairbrother

Published: 2024-06-11T13:01:25Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [benchling](<https://devfeed.tech/topics/benchling.md>), [Data analysis](<https://devfeed.tech/topics/data-analysis.md>), [data transformation](<https://devfeed.tech/topics/data-transformation.md>), [Data visualization](<https://devfeed.tech/topics/data-visualization.md>), [Web app](<https://devfeed.tech/topics/webapp.md>), [Cloud Architecture](<https://devfeed.tech/topics/cloud-architecture.md>), [decision-making](<https://devfeed.tech/topics/decision-making.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [Data pipelines](<https://devfeed.tech/topics/data-pipelines.md>), [Amazon Elastic Kubernetes Service](<https://devfeed.tech/topics/amazon-elastic-kubernetes-service.md>), [real-time](<https://devfeed.tech/topics/real-time.md>), [Tool](<https://devfeed.tech/topics/tool.md>), [IO](<https://devfeed.tech/topics/io.md>)

Tags: [analysis](<https://devfeed.tech/tags/analysis.md>), [apache-arrow](<https://devfeed.tech/tags/apache-arrow.md>), [apache-parquet](<https://devfeed.tech/tags/apache-parquet.md>), [api](<https://devfeed.tech/tags/api.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [backend](<https://devfeed.tech/tags/backend.md>), [behind-the-scenes](<https://devfeed.tech/tags/behind-the-scenes.md>), [benchling](<https://devfeed.tech/tags/benchling.md>), [building](<https://devfeed.tech/tags/building.md>), [curve-fitting](<https://devfeed.tech/tags/curve-fitting.md>), [data](<https://devfeed.tech/tags/data.md>), [data-analysis](<https://devfeed.tech/tags/data-analysis.md>), [data-transformation](<https://devfeed.tech/tags/data-transformation.md>), [data-visualization](<https://devfeed.tech/tags/data-visualization.md>), [datasets](<https://devfeed.tech/tags/datasets.md>), [design](<https://devfeed.tech/tags/design.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [eks](<https://devfeed.tech/tags/eks.md>), [engine](<https://devfeed.tech/tags/engine.md>), [frontend](<https://devfeed.tech/tags/frontend.md>), [interactive](<https://devfeed.tech/tags/interactive.md>), [json](<https://devfeed.tech/tags/json.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [real-time-data](<https://devfeed.tech/tags/real-time-data.md>), [s3](<https://devfeed.tech/tags/s3.md>)

### AI overview

Benchling's Interactive Analysis lets scientists select data from Benchling records, instruments, notebook tables, or uploads, then transform and visualize it in place. The web app sends synchronous requests to a stateless, autoscaling EKS service, which processes each request in an isolated in-memory engine and exchanges inputs and outputs through temporary S3 storage. Parquet and Arrow support fast analytical reads and preserve type information through BenchlingDataFrame; an ExecutionEngine uses DuckDB for common tabular operations and can route advanced calculations, such as four-parameter logistic regression, to Pandas or NumPy via Arrow's zero-copy interoperability. The team kept the service stateless because testing found S3 loading overhead of about 350 ms even for some expected large datasets, within latency requirements. On-demand containers could handle larger or automated jobs but add seconds of startup latency. The initial UI reads full results from S3; partial loading is planned to reduce latency.

### Source excerpt

Authors: Wonja Fairbrother and Eli Levine Science is iterative. To design the next experiment, scientists need to analyze the results of previous ones. Interactive Analysis in Benchling allows scientists to perform real-time data transformation, visualization, and analysis without having to transfer it into other systems. In this post we will describe the architecture behind interactive analysis capabilities in Benchling and give a peek into the decision journey we took along the way¹. Interactive Analysis allows scientists to: 1. Select data from many sources: Benchling entity and results data Instrument data Notebook tables Data upload via both API and UI 2. Transform, visualize, and analyze data in real time, without leaving Benchling: Data transformations: filtering, aggregations, window functions, etc. Visualizations: line chart, bar chart, scatter plot, etc. Scientific analysis methods: IC50 and various curve fitting functions Overall architecture The architecture backing Interactive Analysis consists of: The Benchling web application An auto-scaling stateless internal service running on EKS that performs the transformations Temporary S3 storage locations for input and output data, shared between the web app and the service The frontend of the application is responsible for taking in input datasets and transformation configurations from users. The backend of the web application collects all the input data from the appropriate sources, serializes and uploads the data to S3, and sends a synchronous transformation request to the service. The service's API consists of one main endpoint that takes in a JSON payload of transformation parameters. The service can accept a single transformation, or a list of many transformations to perform. In this endpoint, the service downloads and deserializes the input data, performs the transformation with an analysis engine, and serializes and uploads the resulting data to S3. Each request spins up its own self-contained in-memor

## 10x faster python test iteration via fork(2)

DevFeed: [10x faster python test iteration via fork(2)](<https://devfeed.tech/articles/10x-faster-python-test-iteration-via-fork-2-20122.md>)

Original publisher: [Read original article](<https://benchling.engineering/10x-faster-python-test-iteration-via-fork-2-3aae52d2f6?source=rss----3d4aa8fb07ea---4>)

Author: raylu

Published: 2023-07-20T16:01:45Z

Content type: tutorial

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [Software Testing](<https://devfeed.tech/topics/software-testing.md>), [Test automation](<https://devfeed.tech/topics/test-automation.md>), [Python](<https://devfeed.tech/topics/python.md>), [import](<https://devfeed.tech/topics/import.md>), [Process](<https://devfeed.tech/topics/process.md>)

Tags: [allowed](<https://devfeed.tech/tags/allowed.md>), [also](<https://devfeed.tech/tags/also.md>), [batching](<https://devfeed.tech/tags/batching.md>), [benchling](<https://devfeed.tech/tags/benchling.md>), [block](<https://devfeed.tech/tags/block.md>), [bootstrap](<https://devfeed.tech/tags/bootstrap.md>), [browser](<https://devfeed.tech/tags/browser.md>), [build](<https://devfeed.tech/tags/build.md>), [built-in](<https://devfeed.tech/tags/built-in.md>), [but](<https://devfeed.tech/tags/but.md>), [can](<https://devfeed.tech/tags/can.md>), [challenge](<https://devfeed.tech/tags/challenge.md>), [change](<https://devfeed.tech/tags/change.md>), [code](<https://devfeed.tech/tags/code.md>), [dependencies](<https://devfeed.tech/tags/dependencies.md>), [developer-productivity](<https://devfeed.tech/tags/developer-productivity.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [faster](<https://devfeed.tech/tags/faster.md>), [fork](<https://devfeed.tech/tags/fork.md>), [iteration](<https://devfeed.tech/tags/iteration.md>), [python](<https://devfeed.tech/tags/python.md>), [test](<https://devfeed.tech/tags/test.md>)

### AI overview

Benchling reduced second-run Python test startup from 10 seconds to 1 by using `fork()` to preserve already-loaded dependencies and SQLAlchemy models in a parent process, then loading app and test code in a child that can be discarded and recreated after edits. The article explains why `importlib.reload()` becomes unreliable when changed modules have reverse dependencies or instantiated classes. Benchling further tiers imports by edit frequency and dependency position so changes require discarding less setup. It reports that calling `gc.freeze()` before forking reduced a typical test's unique set size by 160 MiB; it advises against `gc.collect()` first because reused allocator holes can trigger copy-on-write of whole pages. The approach requires `fork` support and runtime module loading, and web-server use needs server-specific integration. The text notes the benefit depends on how imports are tiered for a particular codebase.

### Source excerpt

It's ideal to get feedback on your code faster -- to make a code change and see the result instantly. But, as projects get larger, reload times get longer. Each incremental dependency or bootstrap code block that adds 200ms feels worth it, but 50 of them later and it takes 10 seconds to see the result of a code change. On the Build team at Benchling, that's where we found ourselves one day. We used 146 packages which pull in 128 transitive dependencies for a total of 274 packages. We also spent a lot of time waiting for SQLAlchemy models to initialize. The result is our test harness took 10 seconds to set up. After making a code change, you'd start the test runner, wait a few seconds, alt+tab to your browser, get distracted for a few minutes, and then find out you had a typo in your code. This is a common challenge for a growing codebase, but it's something we knew we needed to fix. Here's the process we arrived at which allowed the second run of tests to start 10x faster -- 90% less waiting. While it'll work a little differently for your codebase depending on the language, dependencies, etc. you're using, hopefully this can inspire you on your journey to faster feedback and testing. importlib.reload() Since the problem is that we spend so long setting up a bunch of modules just right and then want to see the change in a single file we're editing, the most obvious solution is to use importlib.reload from the standard library. import importlib import sys import test_harness_stuff # takes 10 seconds import tests def rerun_tests(changed_path): for mod in sys.modules.values(): if mod.__file__ == changed_path: importlib.reload(mod) tests.run_tests() break if __name__ == '__main__': setup_file_watcher(rerun_tests) tests.run_tests() This (with some special handling for built-in modules, relative path resolution, and batching to handle editors that perform multiple filesystem operations per save) works alright when the file being changed is a test file (or any other leaf node

## Exposing AWS KMS Asymmetric Keys as a JWKS

DevFeed: [Exposing AWS KMS Asymmetric Keys as a JWKS](<https://devfeed.tech/articles/exposing-aws-kms-asymmetric-keys-as-a-jwks-20125.md>)

Original publisher: [Read original article](<https://benchling.engineering/exposing-aws-kms-asymmetric-keys-as-a-jwks-7f183657f0d9?source=rss----3d4aa8fb07ea---4>)

Author: Brian Maloney

Published: 2023-02-02T20:53:31Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [public key](<https://devfeed.tech/topics/public-key.md>), [OpenID connect (OIDC)](<https://devfeed.tech/topics/oidc.md>), [Secrets Management](<https://devfeed.tech/topics/secrets-management.md>), [private key](<https://devfeed.tech/topics/private-key.md>), [JSON Web Tokens](<https://devfeed.tech/topics/jwt.md>), [digital signature](<https://devfeed.tech/topics/digital-signature.md>), [Cryptography](<https://devfeed.tech/topics/cryptography.md>), [Secure token management](<https://devfeed.tech/topics/secure-token-management.md>), [threat detection](<https://devfeed.tech/topics/threat-detection.md>)

Tags: [aws](<https://devfeed.tech/tags/aws.md>), [aws-kms](<https://devfeed.tech/tags/aws-kms.md>), [benchling](<https://devfeed.tech/tags/benchling.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [jwks](<https://devfeed.tech/tags/jwks.md>), [kms](<https://devfeed.tech/tags/kms.md>), [oauth2](<https://devfeed.tech/tags/oauth2.md>), [security](<https://devfeed.tech/tags/security.md>)

### AI overview

Benchling describes exposing AWS KMS asymmetric public keys as a JWKS so an identity provider can discover rotated keys for OIDC private_key_jwt authentication. This avoids static administrator API tokens, which require careful handling and rotation and tie privileges to their creator; KMS keeps private keys in AWS while clients use AWS APIs to sign. The proposed service is a Lambda Function URL that selects enabled keys by alias, since ListKeys omits tags and per-key tag lookups would not scale to a short response window. It converts GetPublicKey's DER output to PEM with Python's Cryptography library, then uses JWCrypto to create JWKs and returns them as JSON. JWT signing is treated as a separate, already documented client-side step. The code is described as proof of concept: frequent JWKS reloads may call for writing updates to object storage and serving from there or a CDN, and production use should add error handling.

### Source excerpt

Here at Benchling, interaction with services is a large part of our business, from employees interacting with the software-as-a-service products with which we conduct our daily business, all the way down to interactions between the services that make up the Benchling application platform itself. Secure authentication and authorization to services is a long-standing issue in the industry, but one that has been improving in recent years due to the widespread adoption of modern standards such as OAuth 2.0 and OpenID Connect (OIDC). One specific use case for service-to-service authentication that is important to Benchling Security is connecting our Threat Detection Pipeline to our enterprise identity services vendor. We use this connection to connect log and other data provided by the vendor to our centralized Threat Detection Platform, where we correlate this with other sources of intelligence to detect risky or suspicious user activity in near real-time. Modern Authentication with OIDC Our specific identity services vendor offers two options for authenticating to its API: either an API token that an administrator can generate, or interaction by acting as an Application. API tokens, while very easy to use, are a poor choice for two reasons: First, they are a static secret that must be handled carefully and rotated frequently to mitigate the risk of a leaked key, which causes significant management overhead. Second, the identity services vendor links the privileges and identity of an API token inextricably to the administrator who generated it. This causes actions using the key to be attributed to the administrator and also makes it impossible to implement the principle of least privilege. Client authentication when acting as an Application allows the use of OIDC, and this vendor specifically requires the use of the private_key_jwt Client Authentication method. Enforcing this requirement is a good choice on the part of the vendor -- by using public-key encryption, no sec