# Automating the data engineer: Addressing the talent shortage

DevFeed: [Automating the data engineer: Addressing the talent shortage](<https://devfeed.tech/articles/automating-the-data-engineer-addressing-the-talent-shortage-80250.md>)

Original publisher: [Read original article](<https://dlthub.com/blog/automating-data-engineers>)

Author: Adrian Brudaru

Published: 2023-06-15T00:00:00Z

Content type: article

Language: en

Sources: [DLT Hub](<https://devfeed.tech/sources/dlt-hub.md>)

Topics: [data-engineering](<https://devfeed.tech/topics/data-engineering.md>), [schema-evolution](<https://devfeed.tech/topics/schema-evolution.md>), [data observability](<https://devfeed.tech/topics/data-observability.md>)

Tags: [automate](<https://devfeed.tech/tags/automate.md>), [data-engineering](<https://devfeed.tech/tags/data-engineering.md>), [engineer](<https://devfeed.tech/tags/engineer.md>), [engineering](<https://devfeed.tech/tags/engineering.md>), [growth](<https://devfeed.tech/tags/growth.md>), [product](<https://devfeed.tech/tags/product.md>), [shortage](<https://devfeed.tech/tags/shortage.md>), [talent](<https://devfeed.tech/tags/talent.md>)

## AI overview

The article argues that automating data structuring and shifting data curation to analysts and producers could reduce data engineers' workload and help address talent shortages. It presents estimated time savings of 40-80% and recommends schema inference and evolution, notifying stakeholders of data changes, and letting analysts explore structured data.

## Source excerpt

## Why is there a data engineer shortage? 1. High Demand and Rapid Growth: The increasing reliance on data-driven decision-making and the rise of big data technologies have created a surge in demand for skilled data engineers. 2. Skill Gap and Specialization: Data engineering requires a unique blend of technical skills, and finding individuals with the right combination of programming, database management, and cloud computing expertise can be challenging. 3.