# Neural network inference pipeline for videos in Tensorflow

DevFeed: [Neural network inference pipeline for videos in Tensorflow](<https://devfeed.tech/articles/neural-network-inference-pipeline-for-videos-in-tensorflow-21537.md>)

Original publisher: [Read original article](<http://lifepluslinux.blogspot.com/2019/08/neural-network-inference-pipeline-for.html>)

Author: Suresh Alse (noreply@blogger.com)

Published: 2019-08-08T18:03:00Z

Content type: tutorial

Language: en

Sources: [Life Plus Linux](<https://devfeed.tech/sources/life-plus-linux.md>)

Topics: [Tensorflow](<https://devfeed.tech/topics/tensorflow.md>), [Inference](<https://devfeed.tech/topics/inference.md>), [Machine learning](<https://devfeed.tech/topics/machine-learning.md>), [GPU](<https://devfeed.tech/topics/gpu.md>), [cpu](<https://devfeed.tech/topics/cpu.md>), [dataset](<https://devfeed.tech/topics/dataset.md>), [Keras](<https://devfeed.tech/topics/keras.md>)

Tags: [cpu](<https://devfeed.tech/tags/cpu.md>), [dataset](<https://devfeed.tech/tags/dataset.md>), [deeplearning](<https://devfeed.tech/tags/deeplearning.md>), [gpu](<https://devfeed.tech/tags/gpu.md>), [inference](<https://devfeed.tech/tags/inference.md>), [keras](<https://devfeed.tech/tags/keras.md>), [performance](<https://devfeed.tech/tags/performance.md>), [pipeline](<https://devfeed.tech/tags/pipeline.md>), [tensorflow](<https://devfeed.tech/tags/tensorflow.md>)

## AI overview

A tutorial on building an efficient TensorFlow pipeline for video inference. It describes processing video frames in batches with tf.data.Dataset, parallelizing CPU preprocessing and I/O, and prefetching to keep CPU and GPU work supplied.

## Source excerpt

Just as we saw a huge influx of images in the past decade or so, we are now seeing a lot of videos being produced on social media. The need to understand and moderate videos using machine learning has never been greater. In this post, I will show you how to build an efficient pipeline to processes videos in Tensorflow. For simplicity, let us consider a Resnet50 model pre-trained on Imagenet. Pretty straightforward, using tf.keras.applications Now, let us break it up to see what exactly is happening: We are loading the model with weights. We are reading an image and resizing it to 224x224. Do some preprocessing of the image. Run inference. Do some post processing. If we want to do something similar for large videos, we need to have a pipeline that takes a stream of frames from the video, applies preprocess transformations, run inference of frames, unravel the inferences and apply post processing. We can see that doing all these in a sequence - frame by frame is clearly not the right thing as it is slow and inefficient. In order to tackle this, we will use tf.data.Dataset and run inference in batch. First, lets create a generator that can produce frames from a video: We will use the tf.data.Dataset.from_generator method to create a dataset object out of this. Now let us define a function which does resizing, normalization and other preprocessing steps that are required on a batch of frames. Then, using the batch operation on the dataset created above, create a batch of size 64. Map the preprocess method that we defined onto the batch in parallel on CPU as it is a CPU intensive task. It is important to make sure that I/O is parallelized as much as possible. For best performance, instructions that are well suited for CPU should run on CPU and the ones suited for GPU should run on GPU. Also, If you observe the code above, we are prefetching. What this means is that, before consuming the dataset, a batch of 64 frames are preprocessed and is ready for consumption. By the t