# gradient descent

Published articles for gradient descent.

This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles.

## Why Gradient Descent Works

DevFeed: [Why Gradient Descent Works](<https://devfeed.tech/articles/the-math-you-missed-behind-gradient-descent-38813.md>)

Original publisher: [Read original article](<https://thepalindrome.org/p/the-math-you-missed-behind-gradient>)

Author: Tivadar Danka

Published: 2026-06-17T09:17:28Z

Content type: article

Language: en

Sources: [The Palindrome](<https://devfeed.tech/sources/the-palindrome.md>)

Topics: [math](<https://devfeed.tech/topics/math.md>)

Tags: [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [math](<https://devfeed.tech/tags/math.md>)

### AI overview

An explanation of why gradient descent works.

### Source excerpt

Why gradient descent works

## Regression and Linear Combinations

DevFeed: [Regression and Linear Combinations](<https://devfeed.tech/articles/regression-and-linear-combinations-40446.md>)

Original publisher: [Read original article](<https://www.jeremykun.com/2021/03/29/regression-and-linear-combinations/>)

Published: 2021-03-29T09:00:00Z

Content type: tutorial

Language: en

Sources: [Jeremy Kun](<https://devfeed.tech/sources/jeremy-kun.md>)

Topics: [linear-regression](<https://devfeed.tech/topics/linear-regression.md>), [math](<https://devfeed.tech/topics/math.md>), [Optimization](<https://devfeed.tech/topics/optimization.md>), [Algorithms, Complexity](<https://devfeed.tech/topics/algorithms-complexity.md>)

Tags: [data](<https://devfeed.tech/tags/data.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [kernelization](<https://devfeed.tech/tags/kernelization.md>), [linear-algebra](<https://devfeed.tech/tags/linear-algebra.md>), [linear-combination](<https://devfeed.tech/tags/linear-combination.md>), [linear-regression](<https://devfeed.tech/tags/linear-regression.md>), [mathematics](<https://devfeed.tech/tags/mathematics.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [programming](<https://devfeed.tech/tags/programming.md>), [regression](<https://devfeed.tech/tags/regression.md>)

### AI overview

The article explains why linear combinations matter to programmers, using linear regression as a practical example. It describes representing inputs and weights as vectors, incorporating an intercept into the input vector, and formulating regression as a least-squares optimization problem. The supplied excerpt then begins introducing basis functions for modeling nonlinearity.

### Source excerpt

Recently I've been helping out with a linear algebra course organized by Tai-Danae Bradley and Jack Hidary, and one of the questions that came up a few times was, "why should programmers care about the concept of a linear combination?" For those who don't know, given vectors $ v_1, \dots, v_n$, a linear combination of the vectors is a choice of some coefficients $ a_i$ with which to weight the vectors in a sum $ v = \sum_{i=1}^n a_i v_i$.

## A Reminder of Lagrange Multipliers for Optimization Problems

DevFeed: [A Reminder of Lagrange Multipliers for Optimization Problems](<https://devfeed.tech/articles/lagrangians-for-the-amnesiac-40334.md>)

Original publisher: [Read original article](<https://www.jeremykun.com/2013/11/30/lagrangians-for-the-amnesiac/>)

Published: 2013-11-30T09:00:50Z

Content type: tutorial

Language: en

Sources: [Jeremy Kun](<https://devfeed.tech/sources/jeremy-kun.md>)

Topics: [Optimization](<https://devfeed.tech/topics/optimization.md>), [function](<https://devfeed.tech/topics/function.md>), [Variable](<https://devfeed.tech/topics/variable.md>)

Tags: [convex-functions](<https://devfeed.tech/tags/convex-functions.md>), [function](<https://devfeed.tech/tags/function.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [lagrange](<https://devfeed.tech/tags/lagrange.md>), [lagrange-multipliers](<https://devfeed.tech/tags/lagrange-multipliers.md>), [linear-algebra](<https://devfeed.tech/tags/linear-algebra.md>), [optimization](<https://devfeed.tech/tags/optimization.md>), [problems](<https://devfeed.tech/tags/problems.md>), [variable](<https://devfeed.tech/tags/variable.md>), [vectors](<https://devfeed.tech/tags/vectors.md>)

### AI overview

A tutorial-style reminder of Lagrange multipliers for optimization problems. It reviews gradients, partial derivatives, dot products, and directional change for multivariable functions.

### Source excerpt

For a while I've been meaning to do some more advanced posts on optimization problems of all flavors. One technique that comes up over and over again is Lagrange multipliers, so this post is going to be a leisurely reminder of that technique. I often forget how to do these basic calculus-type things, so it's good practice. We will assume something about the reader's knowledge, but it's a short list: know how to operate with vectors and the dot product, know how to take a partial derivative, and know that in single-variable calculus the local maxima and minima of a differentiable function $ f(x)$ occur when the derivative $ f'(x)$ vanishes.

## Neural Networks and the Backpropagation Algorithm

DevFeed: [Neural Networks and the Backpropagation Algorithm](<https://devfeed.tech/articles/neural-networks-and-the-backpropagation-algorithm-40296.md>)

Original publisher: [Read original article](<https://www.jeremykun.com/2012/12/09/neural-networks-and-backpropagation/>)

Published: 2012-12-09T21:24:48Z

Content type: tutorial

Language: en

Sources: [Jeremy Kun](<https://devfeed.tech/sources/jeremy-kun.md>)

Topics: [Neural Network](<https://devfeed.tech/topics/neural-network.md>), [Algorithm](<https://devfeed.tech/topics/algorithm.md>), [datasets](<https://devfeed.tech/topics/datasets.md>), [Graphs](<https://devfeed.tech/topics/graphs.md>)

Tags: [algorithm](<https://devfeed.tech/tags/algorithm.md>), [classes](<https://devfeed.tech/tags/classes.md>), [data](<https://devfeed.tech/tags/data.md>), [gradient-descent](<https://devfeed.tech/tags/gradient-descent.md>), [graph-theory](<https://devfeed.tech/tags/graph-theory.md>), [machine-learning](<https://devfeed.tech/tags/machine-learning.md>), [neural-networks](<https://devfeed.tech/tags/neural-networks.md>), [perceptron](<https://devfeed.tech/tags/perceptron.md>)

### AI overview

This tutorial introduces neurons as an extension of the perceptron model. It explains how multiple perceptron models can introduce nonlinearity into classification and begins defining a neuron using weighted inputs, an inner product, and a bias term.

### Source excerpt

Neurons, as an Extension of the Perceptron Model In a previous post in this series we investigated the Perceptron model for determining whether some data was linearly separable. That is, given a data set where the points are labelled in one of two classes, we were interested in finding a hyperplane that separates the classes. In the case of points in the plane, this just reduced to finding lines which separated the points like this: