# Deeplearning Published articles for Deeplearning. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Neural network inference pipeline for videos in Tensorflow DevFeed: [Neural network inference pipeline for videos in Tensorflow]() Original publisher: [Read original article]() Author: Suresh Alse (noreply@blogger.com) Published: 2019-08-08T18:03:00Z Content type: tutorial Language: en Sources: [Life Plus Linux]() Topics: [Tensorflow](), [Inference](), [Machine learning](), [GPU](), [cpu](), [dataset](), [Keras]() Tags: [cpu](), [dataset](), [deeplearning](), [gpu](), [inference](), [keras](), [performance](), [pipeline](), [tensorflow]() ### AI overview A tutorial on building an efficient TensorFlow pipeline for video inference. It describes processing video frames in batches with tf.data.Dataset, parallelizing CPU preprocessing and I/O, and prefetching to keep CPU and GPU work supplied. ### Source excerpt Just as we saw a huge influx of images in the past decade or so, we are now seeing a lot of videos being produced on social media. The need to understand and moderate videos using machine learning has never been greater. In this post, I will show you how to build an efficient pipeline to processes videos in Tensorflow. For simplicity, let us consider a Resnet50 model pre-trained on Imagenet. Pretty straightforward, using tf.keras.applications Now, let us break it up to see what exactly is happening: We are loading the model with weights. We are reading an image and resizing it to 224x224. Do some preprocessing of the image. Run inference. Do some post processing. If we want to do something similar for large videos, we need to have a pipeline that takes a stream of frames from the video, applies preprocess transformations, run inference of frames, unravel the inferences and apply post processing. We can see that doing all these in a sequence - frame by frame is clearly not the right thing as it is slow and inefficient. In order to tackle this, we will use tf.data.Dataset and run inference in batch. First, lets create a generator that can produce frames from a video: We will use the tf.data.Dataset.from_generator method to create a dataset object out of this. Now let us define a function which does resizing, normalization and other preprocessing steps that are required on a batch of frames. Then, using the batch operation on the dataset created above, create a batch of size 64. Map the preprocess method that we defined onto the batch in parallel on CPU as it is a CPU intensive task. It is important to make sure that I/O is parallelized as much as possible. For best performance, instructions that are well suited for CPU should run on CPU and the ones suited for GPU should run on GPU. Also, If you observe the code above, we are prefetching. What this means is that, before consuming the dataset, a batch of 64 frames are preprocessed and is ready for consumption. By the t ## Why Curalate migrated from Caffe to MXNet for deep learning development and deployment DevFeed: [Why Curalate migrated from Caffe to MXNet for deep learning development and deployment]() Original publisher: [Read original article]() Published: 2018-08-01T00:00:00Z Content type: article Language: en Sources: [Curalate]() Topics: [Deep learning](), [Computer vision](), [Framework](), [Deployment](), [Development](), [Microservice](), [ImageNet](), [onnx]() Tags: [computer-vision](), [deep-learning](), [deeplearning](), [deployment](), [development](), [fork](), [framework](), [model](), [mxnet](), [onnx](), [python](), [scala]() ### AI overview Curalate describes migrating from Caffe to MXNet for computer vision systems. The post explains how MXNet supports experimentation, pre-trained models, model conversion, and integration into microservices and production environments. ### Source excerpt This post was simultaneously published to Medium. At Curalate, we use state of the art deep learning and computer vision to add a layer of magic to our products. Intelligent Product Tagging, for example, identifies our clients' products in user-generated photos. Being a startup, we need to build these deep learning and computer vision systems the same way we do the rest of our products: quickly. Our computer vision systems are built in two phases, research and productization, and we require a deep learning framework that accelerates both. During the research phase, we need a framework that's quick to get started with and is flexible enough to experiment with new ideas. Once we have a solution, we need a framework that can easily be integrated into a microservice and deployed to multiple production environments. In the past, we used Caffe for experimentation and our own custom inference interface to deploy the trained models to production. Experimentation was slow due to Caffe's dated Python API, lack of automatic differentiation, unreliable build/install process, and clunky support for advanced layers which required us to maintain our own custom fork. Productization of Caffe was challenging since we had to maintain our own JNI interface. We needed new and modern framework that fulfilled all of our needs while saving us from the shortcomings of Caffe. After a review of all the available options, we decided to move to MXNet. In this post, we'll discuss why we migrated to MXNet as our deep learning framework of choice to facilitate our speed of experimentation, development, and deployment. Training and Experimentation Whenever we are faced with a new computer vision problem, we start by looking at existing state-of-the-art implementations. If we are lucky the functionality of the service we are implementing is similar to an existing pre-trained model for MXNet. MXNet has a fairly fleshed out and maintained Model Zoo that contains all of the standard pre-trained models ## Finding Where's Waldo using Mask R-CNN DevFeed: [Finding Where's Waldo using Mask R-CNN]() Original publisher: [Read original article]() Author: Suresh Alse (noreply@blogger.com) Published: 2018-06-14T05:51:00Z Content type: tutorial Language: en Sources: [Life Plus Linux]() Topics: [Deep learning](), [Training AI Models](), [Algorithm](), [datasets](), [data](), [Code]() Tags: [algorithm](), [code](), [data](), [dataset](), [deep-learning](), [deeplearning](), [github](), [ml](), [paper](), [quality](), [training](), [validation]() ### AI overview This tutorial explains how to use Mask R-CNN to detect and mask Where's Waldo in images. It covers collecting and annotating a small dataset, training the model, and running predictions. The author reports that performance was better on similar, high-quality images where Waldo was clearly visible. ### Source excerpt When I was a kid, I really loved solving Where's Waldo. There were few books (it used to be called Where's Wally) in our school library on which I spent hours finding Waldo. For people who do not know what it is, basically Waldo - a unique character is hidden among hundreds of other characters and you have to find him in all the chaos in the image. Now that I am too old to be solving it and too busy to spend hours on such things, I decided to build a system that uses deep learning to automatically solve it and spent weeks to build it. I started off by treating this like a classification problem with two classes - Waldo and not Waldo, similar to Hot dog - not Hot dog . Once we can get the classification problem successfully solved, we can just apply a classification action mapping (CAM) layer to find Waldo's activations in the image and thus finding Waldo. However I couldn't find enough images of Waldo. I found this repo which has about 20 images. And as there are only 20 Waldo vs thousands of not-Waldo characters, there is very high imbalance in the classes. I still tried though. But the results weren't that great. When I looked if someone has already worked on it, I found a medium post which used Tensorflow's Faster R-CNN model to do this. But I didn't want to just find bounding boxes, I wanted to actually mask out Waldo in the image. But I got more images of Where's Waldo from it. Then I came across this paper on Mask R-CNN which sounded promising for this usecase. And it was indeed much better than my earlier approach: Waldo masked out in the image Original Image In this post I would like to share how I was able to get the data, tag it and train a model to be able to solve Where's Waldo. You can checkout my code on github here. Fork deepwaldo on Github Mask R-CNN The main idea here is to: Take the input image and pass it into a set of convolutional layers that sort of generates a feature map for the given image. Now, you take this feature map and pass it into a r ## Higher level ops for building neural network layers with deeplearn.js DevFeed: [Higher level ops for building neural network layers with deeplearn.js]() Original publisher: [Read original article]() Author: Suresh Alse (noreply@blogger.com) Published: 2018-01-23T22:36:00Z Content type: tutorial Language: en Sources: [Life Plus Linux]() Topics: [Neural Network](), [Tensorflow](), [Code](), [Graphs]() Tags: [batching](), [code](), [datasets](), [deeplearning](), [graph](), [javascript](), [ml](), [neural](), [tensorflow](), [visualization]() ### AI overview This tutorial describes higher-level neural-network operations built for deeplearn.js, specifically implementations of tf.layers.conv2d and tf.layers.flatten. The operations are designed to closely follow corresponding TensorFlow function definitions, with documented arguments and return behavior. ### Source excerpt I have been meddling with google's deeplearn.js lately for fun. It is surprisingly good given how new the project is and it seems to have a sold roadmap. However it still lacks something like tf.layers and tf.contrib.layers which have many higher level functions that has made using tensorflow so easy. It looks like they will be added to Graphlayers in future but their priorities as of now is to fix the lower level APIs first - which totally makes sense. So, I quickly built one for tf.layers.conv2d and tf.layers.flatten which I will share in this post. I have made them as close to function definitions in tensorflow as possible. 1. conv2d - Functional interface for the 2D convolution layer. Arguments: inputs Tensor input. filters Integer, the dimensionality of the output space (i.e. the number of filters in the convolution). kernel_size Number to specify the height and width of the 2D convolution window. graph Graph opbject. strides Number to specify the strides of convolution. padding One of "valid" or "same" (case-insensitive). data_format "channels_last" or "channel_first" activation Optional. Activation function which is applied on the final layer of the function. Function should accept Tensor and graph as parameters kernel_initializer An initializer object for the convolution kernel. bias_initializer An initializer object for bias. name string which represents name of the layer. Returns: Tensor output. Usage: Add this to your code: 2. flatten - Flattens an input tensor. I wrote these snippets while building a tool using deeplearnjs where I do things like loading datasets, batching, saving checkpoints along with visualization. I will share more on that in my future posts. ## Hacking FaceNet using Adversarial examples DevFeed: [Hacking FaceNet using Adversarial examples]() Original publisher: [Read original article]() Author: Suresh Alse (noreply@blogger.com) Published: 2018-01-11T23:29:00Z Content type: tutorial Language: en Sources: [Life Plus Linux]() Topics: [face recognition](), [Machine Learning, Security Attacks](), [Deep learning](), [Neural Network](), [Authentication]() Tags: [authentication](), [deep-learning](), [deeplearning](), [embedding](), [face-recognition](), [hacking](), [ml](), [neural](), [security]() ### AI overview This tutorial explains how FaceNet performs face recognition using embeddings and triplet loss, then demonstrates generating small adversarial noise intended to make an attacker's photo be identified as a target. ### Source excerpt With the rise in popularity of face recognition systems with deep learning and it's application in security/ authentication, it is important to make sure that it is not that easy to fool them. I recently finished the 4th course on deeplearning.ai where there is an assignment which asks us to build a face recognition system - FaceNet. While I was working on the assignment, I couldn't stop thinking about how easy it is to fool it with adversarial examples. In this post I will tell you how I managed to do it. First off, some basics about FaceNet. Unlike image recognition systems which map every image with a class, it is not possible to assign a class label to every face in face recognition. This is because one, there are way too many faces that a system should handle in the real world to assign class to each of them and two, if there are new people the system should handle, it can't do it. So, what we do is, we build a system that learns similarities and dissimilarities. Basically, there is a neural network similar to what we have in image recognition and instead of applying softmax in the end, we just take the logits as embedding for the given image input and then minimize something called the triplet loss. Consider face A, we have a positive match P and negative match N. If f is the embedding function and L is the triplet loss, we have this: Triplet loss Basically, it is incentivizing small distance between A - P and large distance between A - N. Also, I really recommend watching Ian Goodfellow's lecture from Stanford's CS231n course if you want to know about adversarial examples. Like I said earlier, this thought came to me while doing an assignment from 4th course from deeplearning.ai which can be found here and I have built on top of it. The main idea here is to find small noise that when added to someone's photo although causing virtually no visual changes, can make faceNet identify them as the target. Benoit (attacker) Add noise Kian Kian Actual (Target) First l ## Tensorflow and AEM DevFeed: [Tensorflow and AEM]() Original publisher: [Read original article]() Author: Suresh Alse (noreply@blogger.com) Published: 2017-12-17T20:40:00Z Content type: tutorial Language: en Sources: [Life Plus Linux]() Topics: [Tensorflow](), [Java](), [configuration](), [interoperability](), [Deep learning](), [Machine learning]() Tags: [aem](), [c-plus-plus](), [configuration](), [deep-learning](), [deeplearning](), [gpu](), [guide](), [interoperability](), [java](), [ml](), [tensorflow](), [testing]() ### AI overview A tutorial on setting up TensorFlow for Java in Adobe Experience Manager and other Felix-based systems. It covers Maven and bundle configuration, JNI installation, library placement, and testing with a Sling servlet. ### Source excerpt It has been a while since google released Tensorflow support for java. Even though it is still in its infancy, I feel like it has everything we need. Build computation graphs - check, run session and compute stuff - check, GPU support - check. Now if you have all the time in the world to reinvent the wheel, you can pretty much build anything in java that we can build using python or c++. So, I have been working on Adobe Experience Manager since I joined Adobe and recently, I started experimenting with several use cases where machine learning can help in content creation and discovery. As I have zero knowledge in building any deep learning models in java, I decided to build everything in java. How hard can it be? Right? Right? Sarcasm aside, as I mentioned earlier, Tensorflow for java has everything we need and as it internally uses JNI we can have interoperability with python and c++ (that's why I preferred this over deeplearning4j). First off, I followed their official guide for the setup and had to face a lot of hurdles along the way. In this post I will show you how I managed to successfully setup Tensorflow on AEM (or any felix based systems). Step 1 Add the dependency to your pom.xml file. Note that the scope set to compile. Step 2 Add this configuration to your maven-bundle-plugin. Step 3 Build and install to your AEM instance. Then, navigate to /system/console/bundles/ and look for the bundle which contains the dependency. See if the "Exported Packages" section has the following packages: Step 4 Install JNI if necessary (this is mentioned in the link that I shared earlier). Then place the library file in the appropriate place. Testing Lets write a simple sling servlet to check if everything is working as expected. Like I told earlier, Tensorflow for java is still in its infancy. So, I wrote a helper class a while back to manipulate the computation graph. Get GraphBuilder.java and place it where it is accessible to the sling servlet. GraphBuilder.java The foll ## Most original prize at The 2017 Deep Learning Hackathon DevFeed: [Most original prize at The 2017 Deep Learning Hackathon]() Original publisher: [Read original article]() Author: Suresh Alse (noreply@blogger.com) Published: 2017-03-29T20:38:00Z Content type: article Language: en Sources: [Life Plus Linux]() Topics: [Deep learning](), [Hackathon](), [Development](), [Nvidia](), [Google](), [cudnn]() Tags: [ai](), [cudnn](), [deep-learning](), [deeplearning](), [dev](), [developers](), [engineering](), [hackathon](), [ml](), [nvidia]() ### AI overview The author describes participating in Deepgram's 2017 Deep Learning Hackathon and building Medivh, a tool intended to predict where users might look on a website before deployment by generating heat maps. The project won the Most original prize, which was an Nvidia Titan X Pascal. ### Source excerpt Although I have worked on several deep learning projects in the past, I still consider myself to be a newbie in deep learning because of all the new things that keep coming up and it is so hard to keep up with all that. So, I decided to take part in "The 2017 Deep Learning Hackathon" by Deepgram to work on something I have been wanting to do for a while now. I built something called Medivh - prophet from Warcraft who has seen the future. The idea was to build a tool for web developers to predict how users are going to see / use the site even before deploying. Basically, it generates heat maps on websites which show where the user might look at. Example: I will write another post with all the technical details. Here is the sneak peak of how it was done. Apart from building that, We got an opportunity to interact with people like Bryan Catanzaro - maker of CUDNN and VP at Nvidia, Jiaji Huangform from Baidu, Jonathan Hseu from Google Brain etc. We also got to interact with people from Deepgram and their caffe like framework called Kur which seems pretty good. I think I'll write a review about Kur after playing around with it for some more time. Also this: Shenanigans at the @DeepgramAI #DLhackathon !! #ai #hackathon pic.twitter.com/ympPZpPWFG -- BEAST Pets (@beastpets) March 26, 2017 This is me presenting before the results. @DeepgramAI #deeplearning #hackathon @GPUComputing @awscloud @googlecloud class activation mapping web dev guidance @Adobe engineering #AI pic.twitter.com/sDD4vrIUAf -- Leo K Tam (@LeoKTam) March 26, 2017 For Medivh, I won the "Most original prize" - Nvidia Titan X pascal. What a beauty!