# A behind-the-scenes look at building interactive analysis capabilities in Benchling

DevFeed: [A behind-the-scenes look at building interactive analysis capabilities in Benchling](<https://devfeed.tech/articles/a-behind-the-scenes-look-at-building-interactive-analysis-capabilities-in-benchling-20123.md>)

Original publisher: [Read original article](<https://benchling.engineering/a-behind-the-scenes-look-at-building-interactive-analysis-capabilities-in-benchling-fa6ec1bab1e5?source=rss----3d4aa8fb07ea---4>)

Author: Wonja Fairbrother

Published: 2024-06-11T13:01:25Z

Content type: article

Language: en

Sources: [Benchling](<https://devfeed.tech/sources/benchling.md>)

Topics: [data-processing](<https://devfeed.tech/topics/data-processing.md>), [web applications](<https://devfeed.tech/topics/web-applications.md>), [Amazon EKS](<https://devfeed.tech/topics/amazon-eks.md>), [Amazon S3](<https://devfeed.tech/topics/amazon-s3.md>), [API](<https://devfeed.tech/topics/api.md>), [Back end](<https://devfeed.tech/topics/backend.md>), [Front end](<https://devfeed.tech/topics/frontend.md>), [JSON](<https://devfeed.tech/topics/json.md>)

Tags: [apache-arrow](<https://devfeed.tech/tags/apache-arrow.md>), [apache-parquet](<https://devfeed.tech/tags/apache-parquet.md>), [api](<https://devfeed.tech/tags/api.md>), [architecture](<https://devfeed.tech/tags/architecture.md>), [backend](<https://devfeed.tech/tags/backend.md>), [data](<https://devfeed.tech/tags/data.md>), [data-analysis](<https://devfeed.tech/tags/data-analysis.md>), [data-transformation](<https://devfeed.tech/tags/data-transformation.md>), [data-visualization](<https://devfeed.tech/tags/data-visualization.md>), [duckdb](<https://devfeed.tech/tags/duckdb.md>), [frontend](<https://devfeed.tech/tags/frontend.md>), [json](<https://devfeed.tech/tags/json.md>), [real-time](<https://devfeed.tech/tags/real-time.md>), [s3](<https://devfeed.tech/tags/s3.md>), [storage](<https://devfeed.tech/tags/storage.md>), [systems](<https://devfeed.tech/tags/systems.md>)

## AI overview

This article explains the architecture and design decisions behind Benchling's Interactive Analysis capabilities. The system lets scientists select data from multiple sources, transform and analyze it in real time, and visualize results within Benchling. Its architecture uses the Benchling web application, a stateless service running on EKS, temporary S3 storage, and a JSON-based transformation API.

## Source excerpt

Authors: Wonja Fairbrother and Eli Levine Science is iterative. To design the next experiment, scientists need to analyze the results of previous ones. Interactive Analysis in Benchling allows scientists to perform real-time data transformation, visualization, and analysis without having to transfer it into other systems. In this post we will describe the architecture behind interactive analysis capabilities in Benchling and give a peek into the decision journey we took along the way¹. Interactive Analysis allows scientists to: 1. Select data from many sources: Benchling entity and results data Instrument data Notebook tables Data upload via both API and UI 2. Transform, visualize, and analyze data in real time, without leaving Benchling: Data transformations: filtering, aggregations, window functions, etc. Visualizations: line chart, bar chart, scatter plot, etc. Scientific analysis methods: IC50 and various curve fitting functions Overall architecture The architecture backing Interactive Analysis consists of: The Benchling web application An auto-scaling stateless internal service running on EKS that performs the transformations Temporary S3 storage locations for input and output data, shared between the web app and the service The frontend of the application is responsible for taking in input datasets and transformation configurations from users. The backend of the web application collects all the input data from the appropriate sources, serializes and uploads the data to S3, and sends a synchronous transformation request to the service. The service's API consists of one main endpoint that takes in a JSON payload of transformation parameters. The service can accept a single transformation, or a list of many transformations to perform. In this endpoint, the service downloads and deserializes the input data, performs the transformation with an analysis engine, and serializes and uploads the resulting data to S3. Each request spins up its own self-contained in-memor