# Engineering at Premise - Medium

Premise engineers close the information gap in a complex, evolving world. We empower millions around the globe to collect and share information with decision makers, providing them with Data for Every Decision™ - Medium

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Configuring a Multi-Instance Looker Deployment

DevFeed: [Configuring a Multi-Instance Looker Deployment](<https://devfeed.tech/articles/configuring-a-multi-instance-looker-deployment-23874.md>)

Original publisher: [Read original article](<https://engineering.premise.com/configuring-a-multi-instance-looker-deployment-0f1eec1b8e7a?source=rss----c5fada0a103d---4>)

Author: Dennis Mutia

Published: 2023-11-13T15:00:28Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [dashboards](<https://devfeed.tech/topics/dashboards.md>), [GitHub](<https://devfeed.tech/topics/github.md>), [Backstage](<https://devfeed.tech/topics/backstage.md>), [releases](<https://devfeed.tech/topics/releases.md>), [Development](<https://devfeed.tech/topics/development.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>)

Tags: [backstage](<https://devfeed.tech/tags/backstage.md>), [business-intelligence](<https://devfeed.tech/tags/business-intelligence.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [dashboards](<https://devfeed.tech/tags/dashboards.md>), [developer](<https://devfeed.tech/tags/developer.md>), [developer-portal](<https://devfeed.tech/tags/developer-portal.md>), [development](<https://devfeed.tech/tags/development.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [github](<https://devfeed.tech/tags/github.md>), [looker](<https://devfeed.tech/tags/looker.md>), [operational](<https://devfeed.tech/tags/operational.md>), [production](<https://devfeed.tech/tags/production.md>), [releases](<https://devfeed.tech/tags/releases.md>)

### AI overview

This article explains how Premise configured three Looker instances to separate development from production for LookML and dashboards. It describes standardized data connections, view-only production access, GitHub-based synchronization and releases, Backstage-driven promotion, and Slack notifications.

### Source excerpt

By Dennis Mutia, Software Engineer Image by ismagilov on Unsplash This article provides a high level overview of how we have configured multiple Looker deployments to separate development and production environments for both LookML and dashboards. Having a Looker deployment on a single instance has many challenges which include: a dashboard can get edited when someone else is presenting it to a client or a stakeholder someone testing a large dashboard can slow down Looker which will affect other dashboards Looker users cannot test the effects of model changes to their dashboards without having to request for developer access which has cost implications Multiple Looker instances We have set up 3 Looker instances: a development instance where LookML updates are made and dashboards created and reviewed before being released to production, and two production instances. One of the production instances is for internal looks and dashboards and the other for creating dashboards that can be shared with external clients. All three Looker instances have a standardized data connection to read from the same data warehouse. This is to ensure that dashboards and looks work the same access all environments. To ensure all development actually occurs on the development instance, both production instances enforce view only access for all users. This set up allows us to test both data models and dashboards without affecting production content. It also leads to standardized data models across all production instances. Linking development to production To take advantage of the benefits of multiple Looker instances, while limiting the operational maintance, we have linked all three looker instances using GitHub, and use GitHub Releases to update the production instances whenever changes have been tested and approved in development. To release dashboards from development to production we use Backstage, which is also our developer portal. We have created a template which dashboard builders

## Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run

DevFeed: [Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run](<https://devfeed.tech/articles/publishing-kotlin-multiplatform-swift-packages-using-google-cloud-storage-and-cloud-run-23879.md>)

Original publisher: [Read original article](<https://engineering.premise.com/publishing-kotlin-multiplatform-swift-packages-to-google-cloud-storage-be5c6987e5d?source=rss----c5fada0a103d---4>)

Author: Nate Ebel

Published: 2023-10-18T05:28:28Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Kotlin Multiplatform](<https://devfeed.tech/topics/kotlin-multiplatform.md>), [Cloud Run](<https://devfeed.tech/topics/cloud-run.md>), [Gradle](<https://devfeed.tech/topics/gradle.md>), [iOS](<https://devfeed.tech/topics/ios.md>), [kotlin-multiplatform-libraries](<https://devfeed.tech/topics/kotlin-multiplatform-libraries.md>), [Swift](<https://devfeed.tech/topics/swift.md>), [Cloud](<https://devfeed.tech/topics/cloud.md>)

Tags: [cloud](<https://devfeed.tech/tags/cloud.md>), [cloud-run](<https://devfeed.tech/tags/cloud-run.md>), [github](<https://devfeed.tech/tags/github.md>), [google-cloud-platform](<https://devfeed.tech/tags/google-cloud-platform.md>), [google-cloud-run](<https://devfeed.tech/tags/google-cloud-run.md>), [gradle](<https://devfeed.tech/tags/gradle.md>), [ios](<https://devfeed.tech/tags/ios.md>), [kotlin](<https://devfeed.tech/tags/kotlin.md>), [kotlin-multiplatform](<https://devfeed.tech/tags/kotlin-multiplatform.md>), [kotlin-multiplatform-libraries](<https://devfeed.tech/tags/kotlin-multiplatform-libraries.md>), [mobile-app-development](<https://devfeed.tech/tags/mobile-app-development.md>), [multiplatform](<https://devfeed.tech/tags/multiplatform.md>), [swift](<https://devfeed.tech/tags/swift.md>)

### AI overview

This tutorial describes Premise's approach to publishing and consuming Kotlin Multiplatform Swift Packages. It uses a custom Gradle plugin to publish XCFrameworks to Google Cloud Storage and a Cloud Run service to download the Swift Package binaries requested by Xcode, reducing the need to store large binaries in GitHub.

### Source excerpt

By Nate Ebel, Android developer In this post, we'll detail our solution for publishing, and consuming, Kotlin Multiplatform Swift Packages. Our solution leverages a custom Gradle plugin publishing XCFrameworks to Google Cloud Storage and a Google Cloud Run service to download Swift Package binaries requested by XCode. With this solution in place, we've been able to more efficiently serve multiple Kotlin Multiplatform libraries to our iOS application. This is a part of an ongoing series on our usage of Kotlin Multiplatform at Premise: Part 1: Kotlin Multiplatform at Premise Part 2: Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications Part 3: Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions Part 4: Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run -- This Post Part 5: Generating BuildConfig Files for a Kotlin Multiplatform Library -- Coming Soon Part 6: Optimizing Local Build Times for Kotlin Multiplatform Mobile Projects -- Coming Soon Premise and Kotlin Multiplatform Swift Packages We've been using Kotlin Multiplatform in production since early 2021 in the form of our mobile-shared project. During that time, we've consumed our shared code as a Swift Package within our iOS application. The integration of that Swift Package has gone through several iterations. v1: Use the multiplatform-swiftpackage plugin to build the Swift Package and store the XCFramework binary in GitHub v2: Use our own custom Gradle plugin to build the Swift Package and store the XCFramework binary in GitHub These two solutions were very similar. Build the XCFramework. Generate the Package.swiftfile. Check both into git with the desired version tag. These approaches worked fine for a while, but eventually we started to pay the price for our simple initial solution. An XCFramework binary can be pretty large. Ours were in the ballpark of 100MB. So checking 2-3 of these into each commit (1 for each iOS archit

## Serverless Data Pipelines in GCP using Dataform and BigQuery Remote Functions

DevFeed: [Serverless Data Pipelines in GCP using Dataform and BigQuery Remote Functions](<https://devfeed.tech/articles/serverless-data-pipelines-in-gcp-using-dataform-and-bigquery-remote-functions-23880.md>)

Original publisher: [Read original article](<https://engineering.premise.com/serverless-data-pipelines-in-gcp-using-dataform-and-bigquery-remote-functions-9ee235d0cb18?source=rss----c5fada0a103d---4>)

Author: Austen Novis

Published: 2023-08-23T13:35:22Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [BigQuery](<https://devfeed.tech/topics/bigquery.md>), [SQL](<https://devfeed.tech/topics/sql.md>), [GitHub](<https://devfeed.tech/topics/github.md>), [GitLab](<https://devfeed.tech/topics/gitlab.md>), [dataset](<https://devfeed.tech/topics/dataset.md>)

Tags: [bigquery](<https://devfeed.tech/tags/bigquery.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [dataform](<https://devfeed.tech/tags/dataform.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [github](<https://devfeed.tech/tags/github.md>), [gitlab](<https://devfeed.tech/tags/gitlab.md>), [goblet](<https://devfeed.tech/tags/goblet.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [pipelines](<https://devfeed.tech/tags/pipelines.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [sql](<https://devfeed.tech/tags/sql.md>)

### AI overview

This tutorial explains how to build serverless data pipelines on Google Cloud Platform using Dataform and BigQuery Remote Functions. It describes Dataform's SQL-based pipeline orchestration, version-control and monitoring integrations, and shows how Remote Functions can add custom logic through external Cloud Run or Cloud Functions services.

### Source excerpt

Photo by Growtika on Unsplash Dataform is an amazing new offering from Google Cloud Platform (GCP) that allows for serverless orchestration for data pipelines. Previously, data engineering pipelines were either supported by separate data engineering teams, which many smaller organizations don't have the resources to support, or were stitched together using a series of interrelated BigQuery scheduled queries, which could be time-consuming to build and error-prone to run and maintain. Dataform solves these problems, especially for data scientists and data analysts, by including features such as Ability to build data pipelines using SQL Integrates with GitHub and GitLab for code versioning and deployment Monitor pipelines for failures and trigger alerts The main limitation of Dataform is that it only supports SQL, with limited ability to add javascript, but this limitation can be overcome with BigQuery Remote Functions. BigQuery Remote Functions allow you to call external services running in Cloudrun or Cloudfunctions from BigQuery, making it possible to have a serverless data pipeline that can run custom logic in any programming language that you may need. The rest of this blog post will go through a complete example on how to setup both Dataform and BigQuery Remote Functions in just a few steps. Setup Dataform We will begin by setting up a simple Dataform pipeline in the GCP console. Navigate to Dataform and create a repository. Create Repository Make sure to grant the default Dataform service account service-PROJECT_ID@gcp-sa-dataform.iam.gserviceaccount.com BigQuery read and write permissions. This can be accomplished using the roles/bigquery.user role. Next create a new development workspace and allow GCP to initialize it, which will add the default files needed to run the pipeline. Dataform files Next create a new BigQuery dataset called tutorial , and inside a table called test with schema id: int, val: int, type: str . BigQuery Schema We can insert some test da

## Concurrent Programming in Kotlin: Ensuring Thread Safety with Mutex

DevFeed: [Concurrent Programming in Kotlin: Ensuring Thread Safety with Mutex](<https://devfeed.tech/articles/concurrent-programming-in-kotlin-ensuring-thread-safety-with-mutex-23873.md>)

Original publisher: [Read original article](<https://engineering.premise.com/concurrent-programming-in-kotlin-ensuring-thread-safety-with-mutex-5d9c6b80644b?source=rss----c5fada0a103d---4>)

Author: Kwabena Bio Berko

Published: 2023-07-18T03:33:24Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Kotlin](<https://devfeed.tech/topics/kotlin.md>), [Concurrent Programming](<https://devfeed.tech/topics/concurrent-programming.md>), [Coroutines](<https://devfeed.tech/topics/coroutines.md>), [Database](<https://devfeed.tech/topics/database.md>), [Software Engineering](<https://devfeed.tech/topics/software-engineering.md>)

Tags: [atomic](<https://devfeed.tech/tags/atomic.md>), [code](<https://devfeed.tech/tags/code.md>), [component](<https://devfeed.tech/tags/component.md>), [concurrency](<https://devfeed.tech/tags/concurrency.md>), [concurrent](<https://devfeed.tech/tags/concurrent.md>), [concurrent-programming](<https://devfeed.tech/tags/concurrent-programming.md>), [coroutine](<https://devfeed.tech/tags/coroutine.md>), [coroutines](<https://devfeed.tech/tags/coroutines.md>), [data](<https://devfeed.tech/tags/data.md>), [database](<https://devfeed.tech/tags/database.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [function](<https://devfeed.tech/tags/function.md>), [implementation](<https://devfeed.tech/tags/implementation.md>), [io](<https://devfeed.tech/tags/io.md>), [issue](<https://devfeed.tech/tags/issue.md>), [kotlin](<https://devfeed.tech/tags/kotlin.md>), [let](<https://devfeed.tech/tags/let.md>), [mobile-app-development](<https://devfeed.tech/tags/mobile-app-development.md>), [mutex](<https://devfeed.tech/tags/mutex.md>), [safety](<https://devfeed.tech/tags/safety.md>), [suspend](<https://devfeed.tech/tags/suspend.md>), [thread](<https://devfeed.tech/tags/thread.md>)

### AI overview

This Kotlin tutorial explains how Mutex provides mutual exclusion when multiple threads or coroutines call a refresh function. It shows how locking can prevent duplicate network and database operations, avoid race conditions, and preserve atomicity, while withLock can suspend callers until the current lock is released.

### Source excerpt

By Kwabena Bio Berko, Android Engineer Photo by Chris Ried on Unsplash I have recently been experimenting with mutual exclusion(Mutex), a concept in software engineering to solve race conditions when accessing shared resources. Let's assume we have a component with a refresh function that can be accessed from multiple threads or coroutines: https://medium.com/media/e19aa2a9d57bc105b841d28061a199b8/href In the implementation of this component, the function retrieves data from a remote resource and inserts it into the local database. However, these operations need to be atomic. Now, imagine if 10 threads call this function simultaneously, or even a few milliseconds apart. What happens? If you guessed that we make 10 network calls and 10 database IO operations to save the items, then you would be right, as confirmed by the below test: https://medium.com/media/9b279658dbf87021068ee21a913dc80e/hrefhttps://medium.com/media/81cdc164ad807fd45df77389c00bd4cc/href But that's not what we want. It's definitely not resource-friendly. To address this issue, we can use a Mutex. As the name suggests, a Mutex provides mutual exclusion for a specific portion of your code, imposing restrictions on the access to that portion in situations where multiple threads or coroutines may attempt to access it concurrently. In essence, Mutex allows only one thread or coroutine to work within the confines of that portion of the code at any given time. By introducing a Mutex in our refresh function, we can ensure that only one thread is executing the critical sections of the code, preventing race conditions and ensuring atomicity of the operations. https://medium.com/media/9d88a802b4c73f14ceb4d5527158af5d/href In the example above, whenever the refresh function is called, we check to see if the Mutex is already locked or been used. If not, we lock it and then perform our network call and save the results in our local database. This means that whenever another thread or coroutine calls this refresh

## Easily Manage IAM Policies for Serverless REST Applications in GCP with Goblet

DevFeed: [Easily Manage IAM Policies for Serverless REST Applications in GCP with Goblet](<https://devfeed.tech/articles/easily-manage-iam-policies-for-serverless-rest-applications-in-gcp-with-goblet-23877.md>)

Original publisher: [Read original article](<https://engineering.premise.com/easily-manage-iam-policies-for-serverless-rest-applications-in-gcp-with-goblet-f1580a97b74?source=rss----c5fada0a103d---4>)

Author: Austen Novis

Published: 2023-07-10T20:09:55Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [IAM](<https://devfeed.tech/topics/iam.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [Authorization](<https://devfeed.tech/topics/authorization.md>), [Python](<https://devfeed.tech/topics/python.md>), [Framework](<https://devfeed.tech/topics/framework.md>), [API](<https://devfeed.tech/topics/api.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [framework](<https://devfeed.tech/tags/framework.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [goblet](<https://devfeed.tech/tags/goblet.md>), [google-cloud-platform](<https://devfeed.tech/tags/google-cloud-platform.md>), [iam](<https://devfeed.tech/tags/iam.md>), [identity-and-access](<https://devfeed.tech/tags/identity-and-access.md>), [permission](<https://devfeed.tech/tags/permission.md>), [python](<https://devfeed.tech/tags/python.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [tutorial](<https://devfeed.tech/tags/tutorial.md>)

### AI overview

This tutorial explains how the Goblet Python framework automates IAM policy management for serverless REST applications on Google Cloud Platform. It covers identifying deployment permissions, enabling required APIs, creating custom roles and service accounts, and adding invoker bindings for connected services.

### Source excerpt

By Austen Novis, Staff Software Engineer Photo by Akhilesh Sharma on Unsplash Writing and deploying serverless applications has never been easier, especially on Google Cloud Platform (GCP). With a few clicks developers are able to deploy their application to the cloud and then trigger calls in a variety of ways from Cloud Schedulers and Pubsub Subscriptions to API Gateways. Deploying your resources to GCP seems straightforward until you get a 403 error, permission denied. This error causes frustration as the root cause can be a number of issues, and often the solution requires knowing GCP roles and permissions in an in depth manner. The issue can be caused by the user not having the correct permissions to deploy their services or the services themselves not having the correct permissions to connect to each other. Once you have identified the problem you will need to understand what roles have the required permissions, how to apply those roles, and where to apply them. Instead of application developers wasting their time researching GCP documentation, searching stack overflow, or reaching out to devops teams, IAM access should be as automated as much as possible. Using the Goblet framework, we are now able to view exactly what permissions are needed to deploy our serverless applications, enable required GCP service API's, create a new custom role with these permissions, create a service account with this role, and add the correct invoker bindings so that all serverless services can connect to each other. Goblet is a python framework for writing serverless applications in GCP with the goal of making it as simple as possible to write and deploy REST applications. Goblet uses simple decorators, similar to flask, to create the necessary configurations and automatically deploy the required services and infrastructure. For example, with code below we can deploy a simple cloudfunction that is triggered by a pubsub subscription and a cloud scheduler. Goblet will also take ca

## Tutorial: Low Usage Alerting On Slack for Google Cloud Platform (GCP)

DevFeed: [Tutorial: Low Usage Alerting On Slack for Google Cloud Platform (GCP)](<https://devfeed.tech/articles/tutorial-low-usage-alerting-on-slack-for-google-cloud-platform-gcp-23882.md>)

Original publisher: [Read original article](<https://engineering.premise.com/tutorial-low-usage-alerting-on-slack-for-google-cloud-platform-gcp-cc68ac8ca4d?source=rss----c5fada0a103d---4>)

Author: Mauricio Martinez

Published: 2023-04-24T15:57:09Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Cloud Run](<https://devfeed.tech/topics/cloud-run.md>), [Python](<https://devfeed.tech/topics/python.md>), [Monitoring](<https://devfeed.tech/topics/monitoring.md>), [Slack](<https://devfeed.tech/topics/slack.md>)

Tags: [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cloud-run](<https://devfeed.tech/tags/cloud-run.md>), [cloudrun](<https://devfeed.tech/tags/cloudrun.md>), [cost-savings](<https://devfeed.tech/tags/cost-savings.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [github](<https://devfeed.tech/tags/github.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [metrics](<https://devfeed.tech/tags/metrics.md>), [monitoring](<https://devfeed.tech/tags/monitoring.md>), [python](<https://devfeed.tech/tags/python.md>), [slack](<https://devfeed.tech/tags/slack.md>)

### AI overview

This tutorial explains how to deploy a Python-based Cloud Function that monitors weekly CPU and memory usage for Google Cloud services, including Cloud Run, and sends Slack notifications to service owners when a service may be safely downscaled to reduce costs. It covers creating MQL queries, retrieving monitoring metrics, checking minimum thresholds, and deploying the scheduled function.

### Source excerpt

Google Cloud Platform (GCP) is a powerful cloud computing platform that allows businesses to run their applications and workloads with ease. However, as the number of services and applications increases, it becomes challenging to keep track of the usage of each service and ensure they are cost optimized. To address this issue, we can deploy a Python-based Cloud Function that will monitor the low usage of GCP services and notify service owners weekly via Slack that their service can be safely downscaled to minimize costs. This tutorial will walk you through setting up this entire flow in a few quick and easy steps, while allowing you to easily customize the service thresholds, Slack message ui, and the cron schedule. The code for this tutorial can be found on our gcp-tutorials GitHub repository. ArchitectureGet CPU/Memory Usage Metrics We can use the Metrics Explorer UI to build a MQL query to retrieve the desired data from the monitoring metrics API. In this tutorial we will monitor Cloud Run services, but you can monitor other resources by using a different MQL query. In our example we selected the CPU and Memory Cloud Run metrics: We group by service name and location and select the alignment for the 99th percentage to get the max usage over the 1 week duration. Then click on CODE EDITOR to generate a sample MQL query: fetch cloud_run_revision | metric 'run.googleapis.com/container/cpu/utilizations' | group_by 1w, [value_utilizations_percentile: percentile(value.utilizations, 99)] | every 1w | group_by [resource.service_name, resource.location], [value_utilizations_percentile_max: max(value_utilizations_percentile)]fetch cloud_run_revision | metric 'run.googleapis.com/container/memory/utilizations' | group_by 1w, [value_utilizations_percentile: percentile(value.utilizations, 99)] | every 1w | group_by [resource.service_name, resource.location], [value_utilizations_percentile_max: max(value_utilizations_percentile)]Code Once we have our MQL queries we will use the

## Deploy and Handle GCP CloudTasks with Goblet in minutes.

DevFeed: [Deploy and Handle GCP CloudTasks with Goblet in minutes.](<https://devfeed.tech/articles/deploy-and-handle-gcp-cloudtasks-with-goblet-in-minutes-23875.md>)

Original publisher: [Read original article](<https://engineering.premise.com/deploy-and-handle-gcp-cloudtasks-with-goblet-in-minutes-ee138e9dd2c5?source=rss----c5fada0a103d---4>)

Author: Mauricio Wittenberg

Published: 2023-04-18T13:55:58Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Python](<https://devfeed.tech/topics/python.md>), [Microservices](<https://devfeed.tech/topics/microservices.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [IAM](<https://devfeed.tech/topics/iam.md>), [Architecture & Design](<https://devfeed.tech/topics/architecture-design.md>), [Docker](<https://devfeed.tech/topics/docker.md>), [Docker Image](<https://devfeed.tech/topics/docker-image.md>)

Tags: [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cloud-tasks](<https://devfeed.tech/tags/cloud-tasks.md>), [cloudrun](<https://devfeed.tech/tags/cloudrun.md>), [container-registry](<https://devfeed.tech/tags/container-registry.md>), [docker](<https://devfeed.tech/tags/docker.md>), [docker-image](<https://devfeed.tech/tags/docker-image.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [goblet](<https://devfeed.tech/tags/goblet.md>), [google-cloud](<https://devfeed.tech/tags/google-cloud.md>), [google-cloud-platform](<https://devfeed.tech/tags/google-cloud-platform.md>), [iam](<https://devfeed.tech/tags/iam.md>), [infrastructure](<https://devfeed.tech/tags/infrastructure.md>), [microservices](<https://devfeed.tech/tags/microservices.md>), [permissions](<https://devfeed.tech/tags/permissions.md>), [python](<https://devfeed.tech/tags/python.md>), [rate-limiting](<https://devfeed.tech/tags/rate-limiting.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [tutorial](<https://devfeed.tech/tags/tutorial.md>)

### AI overview

This tutorial explains how Goblet can deploy and manage Google Cloud Tasks for serverless microservices on Google Cloud Platform. It covers defining a queue, enqueuing and handling tasks in Python, configuring IAM permissions, and deploying related infrastructure, Docker images, and Cloud Run revisions.

### Source excerpt

by Mauricio Wittenberg. Goblet DeployIntroduction CloudTasks is a Google Cloud Platform (GCP) service that allows users to enqueue tasks in a queue. It is somewhat similar to PubSub, but with CloudTasks, the queue has features such as rateLimits and retryConfig. This can be very useful when having to enforce a rate limit over an external API or retrying tasks that have a known probability of failure. If you want to use CloudTasks you will have to manually deploy cloud infrastructure, understand the CloudTask object, deal with client libraries and figure out multiple IAM role bindings. Or you can use Goblet for all the heavy lifting and in just a few minutes be able push a task and later process the task in a Python function of our choosing. Goblet is a Python based cloud framework for building serverless microservices on Google Cloud Platform. It enables developers to quickly and reliably deploy cloud resources together with the code that will make use of those resources. In this blog post we will review how Goblet deploys CloudTaskQueues and provides the user with an simple code interface to enqueue and handle CloudTasks⁶. Using the short snipped of code below we can define a queue, create a client, push a task to the queue, and handle tasks from the queue! https://medium.com/media/d81fb2d8bd47a3f69479708d61dd5230/href See the rest of the tutorial for setting up your Goblet environment, setting permissions, and deploying you infrastructure and application in a few quick steps. Infrastructure For this tutorial we will use an example GCP project called goblet-cloudtask and deploy region specific resources to us-central1. If you would like to follow along the complete code can be found at gcp-tutorials. The account used to run this example must have enough IAM role binding to run all the gcloud commands described in the Set-Up section below. Now, let's take a look at the infrastructure Goblet will create. We won't have to take care of any of these steps ourselves. All

## Dynamic Data On Backstage Templates

DevFeed: [Dynamic Data On Backstage Templates](<https://devfeed.tech/articles/dynamic-data-on-backstage-templates-23876.md>)

Original publisher: [Read original article](<https://engineering.premise.com/dynamic-data-on-backstage-templates-ad8a0ae316e1?source=rss----c5fada0a103d---4>)

Author: Mauricio Martinez

Published: 2023-03-13T17:47:07Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Backstage](<https://devfeed.tech/topics/backstage.md>), [Extension](<https://devfeed.tech/topics/extension.md>), [Forms](<https://devfeed.tech/topics/forms.md>), [API](<https://devfeed.tech/topics/api.md>), [HTTP](<https://devfeed.tech/topics/http.md>), [JSON](<https://devfeed.tech/topics/json.md>)

Tags: [api](<https://devfeed.tech/tags/api.md>), [backstage](<https://devfeed.tech/tags/backstage.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [dynamic-data](<https://devfeed.tech/tags/dynamic-data.md>), [extension](<https://devfeed.tech/tags/extension.md>), [forms](<https://devfeed.tech/tags/forms.md>), [http](<https://devfeed.tech/tags/http.md>), [javascript](<https://devfeed.tech/tags/javascript.md>), [json](<https://devfeed.tech/tags/json.md>), [plugins](<https://devfeed.tech/tags/plugins.md>)

### AI overview

This tutorial explains how to use the dynamic-data-form-extension plugin collection with Backstage templates. It covers a frontend custom field extension that populates Select components from HTTP endpoints and a backend plugin for custom providers and authenticated or transformed data.

### Source excerpt

Dynamic Data in Backstage TemplatesTable Of ContentsIntroductionFrontend Plugin (dynamic-pick-extension)Backend Plugin (form-data-backend)Introduction If you've ever built a Backstage template, you know that creating user-friendly forms is a must. One of the most common form elements is the enumcomponent. It allows users to choose from a list of options, which is especially useful when you have a large number of choices. But what happens when you have to dynamically populate that list with data from an external API? That's where the dynamic-data-form-extension plugin collection comes in. The dynamic-data-form-extension plugin collection allows you to populate a <Select> component with data fetched from an HTTP endpoint. It's a simple, yet powerful plugin that can save you a lot of time and effort. Frontend Plugin (dynamic-pick-extension)https://www.loom.com/share/9de1f5c431864f19982fcdfab4cd029a This plugin is a Custom Field Extension that allow you to create <Select> components that fetches data dynamically from an endpoint. This can be used together with the form-data-backend plugin to write custom logic to fill the field. Installationyarn add --cwd packages/app @premise/plugin-dynamic-pick-extensionConfiguration Add the import to your App.tsx on the frontend package of your backstage instance: import { DynamicPickFieldExtension } from '@premise/plugin-dynamic-pick-extension'; Then add the imported field extension as a child of ScaffolderFieldExtensions. <ScaffolderFieldExtensions> <DynamicPickFieldExtension /> </ScaffolderFieldExtensions>Usage To use the extension on a Backstage Template Action just add the ui-field and ui-options fields to the parameter. Basic Usage:parameters: - category: title: Category type: string ui:field: DynamicPickExtension ui:options: # IMPORTANT: The endpoint needs to return a JSON array of strings. external_data: https://dummyjson.com/products/categoriesUsing the form-data-backend plugin:parameters: - team: title: Github Team to add a

## Hosting a fully Serverless Web-Based Postgres Admin Client on GCP using Pgweb, Cloud Run, & IAP

DevFeed: [Hosting a fully Serverless Web-Based Postgres Admin Client on GCP using Pgweb, Cloud Run, & IAP](<https://devfeed.tech/articles/hosting-a-fully-serverless-web-based-postgres-admin-client-on-gcp-using-pgweb-cloud-run-iap-23878.md>)

Original publisher: [Read original article](<https://engineering.premise.com/hosting-a-fully-serverless-web-based-postgres-admin-client-on-gcp-using-pgweb-cloud-run-iap-cff0ce8f471b?source=rss----c5fada0a103d---4>)

Author: Austen Novis

Published: 2023-03-06T13:17:20Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Cloud Run](<https://devfeed.tech/topics/cloud-run.md>), [PostgreSQL](<https://devfeed.tech/topics/postgresql.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [Authentication](<https://devfeed.tech/topics/authentication.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>), [container](<https://devfeed.tech/topics/container.md>)

Tags: [authentication](<https://devfeed.tech/tags/authentication.md>), [cli](<https://devfeed.tech/tags/cli.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cloud-run](<https://devfeed.tech/tags/cloud-run.md>), [cloud-sql](<https://devfeed.tech/tags/cloud-sql.md>), [cloudrun](<https://devfeed.tech/tags/cloudrun.md>), [container](<https://devfeed.tech/tags/container.md>), [deploy](<https://devfeed.tech/tags/deploy.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud-platform](<https://devfeed.tech/tags/google-cloud-platform.md>), [identity-aware-proxy](<https://devfeed.tech/tags/identity-aware-proxy.md>), [pgweb](<https://devfeed.tech/tags/pgweb.md>), [postgresql](<https://devfeed.tech/tags/postgresql.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [vpc](<https://devfeed.tech/tags/vpc.md>)

### AI overview

A tutorial explains how to deploy Pgweb, a Go-based PostgreSQL administration client, as a serverless web application on Google Cloud. It uses Cloud Run, CloudSQL, a CloudSQL connector, Secret Manager, Artifact Registry, and optionally an HTTP load balancer with Identity-Aware Proxy for authentication.

### Source excerpt

By Austen Novis, Staff Software Engineer Pgweb example screenshot There are a number of quality Postgres open source administration tools such as pgAdmin or DBeaver, but these tools require a persistent server to run. This means that each user needs to install the tool locally and setup their connections, or you need to host the tool in a cloud server, which can get expensive. A much cheaper alternative is to use Pgweb, a lightweight web-based database explorer for PostgreSQL written in Go, and deploy to Cloud Run. This allows you to utilize a web-based tool, for minimal cost, that can scale for any number of users. We will leverage several Google Cloud Platform services for the complete setup starting with a http load balancer to allow for a custom DNS name as well as enable Identity Aware Proxy (IAP), which is what we will be using for authentication. We will host the service in Cloud Run and connect directly to CloudSQL using a CloudSQL connector. You can bypass the load balancer and IAP if you would like to use username and passwords for authentication. Cloud componentsCloud Setup The first step is setting up your CloudSQL instance. This can be done via terraform or in the GCP console. The main requirement is that you enable a public ip address, which will allow us to use the CloudSQL Auth proxy to connect our Cloud Run instance to our CloudSQL instance. If this is not possible you can still connect to your CloudSQL instance to Cloud Run through a VPC Connector instead. Next we will need to save our database connection credentials in GCP's Secret Manger by creating a new secret called PGWEB_DATABASE_URL in the format of postgres:///DB_NAME?host=/cloudsql/PROJECT:REGION:INSTANCE_NAME&user=DB_USER&password=DB_PASSWORD . Now that we have our CloudSQL instance and connection secret we will deploy our Cloud Run instance using the gcloud cli. First we will need to push the desired Pgweb container to GCP's Artifact Registry, which we can do using the following commands

## Tutorial: Connecting Cloudrun and Cloudfunctions to Redis and other Private Services using Goblet

DevFeed: [Tutorial: Connecting Cloudrun and Cloudfunctions to Redis and other Private Services using Goblet](<https://devfeed.tech/articles/tutorial-connecting-cloudrun-and-cloudfunctions-to-redis-and-other-private-services-using-goblet-23881.md>)

Original publisher: [Read original article](<https://engineering.premise.com/tutorial-connecting-cloudrun-and-cloudfunctions-to-redis-and-other-private-services-using-goblet-5782f80da6a0?source=rss----c5fada0a103d---4>)

Author: Qua Jones

Published: 2023-02-13T16:38:05Z

Content type: tutorial

Language: en

Sources: [Engineering at Premise - Medium](<https://devfeed.tech/sources/engineering-at-premise-medium.md>)

Topics: [Tutorial](<https://devfeed.tech/topics/tutorial.md>), [Cloud Run](<https://devfeed.tech/topics/cloud-run.md>), [Redis](<https://devfeed.tech/topics/redis.md>), [VPC](<https://devfeed.tech/topics/vpc.md>), [Google Cloud Platform (GCP)](<https://devfeed.tech/topics/google-cloud.md>), [Serverless](<https://devfeed.tech/topics/serverless.md>), [Command-line interface](<https://devfeed.tech/topics/cli.md>), [Deployment](<https://devfeed.tech/topics/deployment.md>), [Python](<https://devfeed.tech/topics/python.md>), [Caching](<https://devfeed.tech/topics/caching.md>)

Tags: [caching](<https://devfeed.tech/tags/caching.md>), [cli](<https://devfeed.tech/tags/cli.md>), [cloud-computing](<https://devfeed.tech/tags/cloud-computing.md>), [cloud-run](<https://devfeed.tech/tags/cloud-run.md>), [deployment](<https://devfeed.tech/tags/deployment.md>), [gcp](<https://devfeed.tech/tags/gcp.md>), [google-cloud-run](<https://devfeed.tech/tags/google-cloud-run.md>), [python](<https://devfeed.tech/tags/python.md>), [redis](<https://devfeed.tech/tags/redis.md>), [serverless](<https://devfeed.tech/tags/serverless.md>), [tutorial](<https://devfeed.tech/tags/tutorial.md>), [vpc](<https://devfeed.tech/tags/vpc.md>)

### AI overview

A tutorial showing how to deploy a Cloud Run service that privately connects to a Redis instance through a VPC Connector. It explains how Goblet can provision the connector and inject the required configuration during deployment, using Google Cloud services and Python prerequisites.

### Source excerpt

By Qua Jones, Senior Software Engineer Photo by israel palacio on Unsplash Google Cloud Platform's (GCP) Private Services Access allows you to connect to GCP services without an external IP being assigned. Redis is an in-memory DB that is used for caching and is a service that supports private access. In order to take advantage of this feature, a private connection is necessary. For serverless environments, this can be done using a VPC Connector. VPC Connectors allow serverless environments to connect to VPC Networks by handling traffic between the network and environment, restricting requests to use only internal IP addresses. This can be beneficial for applications that do not require exposure to the public internet or if latency for requests across services needs to be improved. Currently you can provision a connector using the console, cli, or terraform but you will still need to configure your serverless environment to utilize this connector. Instead of manually going through the segmented process of creating a connector and updating your environment, you can leverage Goblet to provision your connector and inject needed configuration values into your deployment. This tutorial will walk you through the steps of deploying a Cloud Run service that privately connects to a Redis Instance using a VPC Connector. Prerequisites: GCP Account Python Environment (>= 3.7) Gcloud CLI You will also need to have goblet and redis installed which can be done by running pip install goblet-gcp && pip install redis Getting Started: Ensure you have credentials configured by running gcloud auth login and sign in to the desired project. Make sure to have the correct services enabled in your GCP project which include: Cloud Run Cloud Build Artifact Registry Serverless VPC Access Memorystore for Redis To get started, clone the code from the gcp-tutorials repository and navigate to the directory. This can be done with the Github CLI by running: gh repo clone premisedata/gcp-tutorials/014-