# Theory Published articles for Theory. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Tambara Equipment DevFeed: [Tambara Equipment]() Original publisher: [Read original article]() Author: Bartosz Milewski Published: 2026-07-11T08:13:34Z Content type: article Language: en Sources: [Bartosz Milewski's Programming Cafe]() Topics: [modules](), [Haskell](), [Code]() Tags: [category-theory](), [code](), [double-categories](), [double-category](), [haskell](), [modules](), [optics](), [proarrow-equipment](), [profunctors](), [structure](), [tambara-modules](), [tannakian-reconstruction](), [theory](), [transformation]() ### AI overview This article explains Tambara modules through category theory and illustrates the concepts with Haskell code. It discusses their relationship to profunctors, monoidal actions, double categories, proarrow equipment, and Tannakian reconstruction. ### Source excerpt I was originally attracted to category theory when trying to understand Haskell optics. I was puzzled by the van Laarhoven's functor representations and Kmett's use of Tambara modules. By playing Tetris with the Yoneda lemma I was able to make some progress, attacking more and more esoteric topics. With a group of researcher and students [...] ## The Problem of Pedagogy in Advanced Mathematics DevFeed: [The Problem of Pedagogy in Advanced Mathematics]() Original publisher: [Read original article]() Published: 2026-05-11T00:00:00Z Content type: opinion Language: en Sources: [Susam Pal]() Topics: [Mathematics](), [Math and Logic]() Tags: [advanced](), [mathematics](), [opinion](), [students](), [theory]() ### AI overview The article argues that pedagogy remains a serious problem in advanced mathematics. It focuses on graduate-level textbooks whose proofs are often presented as high-level outlines, leaving students and even professional mathematicians to reconstruct omitted intermediate steps. It advocates explanations that are correct, complete, and accessible to reasonably motivated students. ### Source excerpt It is a commonly held opinion that educational institutions could do more to improve the pedagogy of mathematics. This is especially applicable to primary and secondary schools, where students are first exposed to mathematics as a formal subject, along with other new subjects. Poor exposition can turn students away from mathematics for a lifetime. Only the highly motivated ones continue to engage with the subject. This is very unfortunate because mathematics is a beautiful subject and it is filled with wonder. It also teaches rigour in reasoning, clarity of thought and the discipline of constructing arguments from first principles to obtain intricate and often beautiful results. What is perhaps less known is that pedagogy is a problem even for graduate-level mathematics students and professional mathematicians. The proofs in many graduate-level mathematics textbooks are, in my humble opinion, not really proofs at all. They are closer to high-level outlines of proofs. The authors simply do not show their work. The student then has to put in an extraordinary amount of effort to understand and justify each line. Sometimes a 10-line argument in a textbook might expand into a 10-page proof if the student really wants to convince themselves that the argument works. I am not a mathematician, but out of personal interest, I have worked with professional mathematicians in the past to help refine notes that explain certain intermediate steps in textbooks (for example, Galois Theory by Stewart, in a specific case). I was surprised to find that it was not just me who found the intermediate steps of certain proofs obscure. Even professional mathematicians who had studied the subject for much of their lives found them obscure. It took us two days of working together to untangle a complicated argument and present it in a way that satisfied three properties: correctness, completeness and accessibility to a reasonably motivated student. There is a reason why jokes like 'proof by obv ## Analysis of PPPP "encryption" DevFeed: [Analysis of PPPP "encryption"]() Original publisher: [Read original article]() Author: Wladimir Palant Published: 2026-01-05T15:50:53Z Content type: article Language: en Sources: [Almost Secure]() Topics: [Encryption](), [Cryptography](), [Network](), [Protocol (disambiguation)](), [Algorithms]() Tags: [analysis](), [ascii](), [cryptanalysis](), [encryption](), [lan](), [network](), [pull-request](), [theory]() ### AI overview This analysis examines weaknesses in PPPP "encryption." It explains that app keys are reduced to a four-byte effective key and that the scheme is vulnerable to known-plaintext attacks. For LAN camera discovery when the app and traffic are unknown, the article describes sending all possible encrypted search packets, with at most 157,092 ciphertexts, and notes a pull request to update the device-detection script. ### Source excerpt My first article on the PPPP protocol already said everything there was to say about PPPP "encryption": Keys are static and usually trivial to extract from the app. No matter how long the original key, it is mapped to an effective key that's merely four bytes long. The "encryption" is extremely susceptible to known-plaintext attacks, usually allowing reconstruction of the effective key from a single encrypted packet. So this thing is completely broken, why look any further? There is at least one situation where you don't know the app being used so you cannot extract the key and you don't have any traffic to analyze either. It's when you are trying to scan your local network for potential hidden cameras. This script will currently only work for cameras using plaintext communication. Other cameras expect a properly encrypted "LAN search" packet and will ignore everything else. How can this be solved without listing all possible keys in the script? By sending all possible ciphertexts of course! TL;DR: What would be completely ridiculous with any reasonable protocol turned out to be quite possible with PPPP. There are at most 157,092 ways in which a "LAN search" packet can be encrypted. I've opened a pull request to have the PPPP device detection script adjusted. Note: Cryptanalysis isn't my topic, I am by no means an expert here. These issues are simply too obvious. Contents Mapping keys to effective keys Redundancies within the effective key ASCII to the rescue How large is n? How many ciphertexts is that? Understanding the response Mapping keys to effective keys The key which is specified as part of the app's "init string" is not being used for encryption directly. Nor is it being fed into any of the established key stretching algorithms. Instead, a key represented by the byte sequence b1,b2,...,bnb_1, b_2, \ldots, b_n is mapped to four bytes k1,k2,k3,k4k_1, k_2, k_3, k_4 that become the effective key. These bytes are calculated as follows (⌊x⌋\lfloor x \rfloor means r ## B-trees Require Fewer Comparisons Than Balanced Binary Search Trees DevFeed: [B-trees Require Fewer Comparisons Than Balanced Binary Search Trees]() Original publisher: [Read original article]() Author: Viktor Leis (noreply@blogger.com) Published: 2024-06-06T13:59:00Z Content type: article Language: en Sources: [Database Architects]() Topics: [Data structures](), [data]() Tags: [comparisons](), [data-structure](), [structure](), [theory]() ### AI overview The article compares B-trees with balanced binary search trees by analyzing the number of comparisons required for lookup operations. It explains that as the B-tree degree increases, the comparison bound approaches the lower bound, and for degree k>=8, B-trees are guaranteed to use fewer comparisons than AVL trees. ### Source excerpt Due to better access locality, B-trees are faster than binary search trees in practice -- but are they also better in theory? To answer this question, let's look at the number of comparisons required for a search operation. Assuming we store n elements in a binary search tree, the lower bound for the number of comparisons is log2 n in the worst case. However, this is only achievable for a perfectly balanced tree. Maintaining such a tree's perfect balance during insert/delete operations requires O(n) time in the worst case. Balanced binary search trees, therefore, leave some slack in terms of how balanced they are and have slightly worse bounds. For example, it is well known that an AVL tree guarantees at most 1.44 log2 n comparisons, and a Red-Black tree guarantees 2 log2 n comparisons. In other words, AVL trees require at most 1.44 times the minimum number of comparisons, and Red-Black trees require up to twice the minimum. How many comparisons does a B-tree need? In B-trees with degree k, each node (except the root) has between k and 2k children. For k=2, a B-tree is essentially the same data structure as a Red-Black tree and therefore provides the same guarantee of 2 log2 n comparisons. So how about larger, more realistic values of k? To analyze the general case, we start with a B-tree that has the highest possible height for n elements. The height is maximal when each node has only k children (for simplicity, this analysis ignores the special case of underfull root nodes). This implies that the worst-case height of a B-tree is logk n. During a lookup, one has to perform a binary search that takes log2 k comparisons in each of the logk n nodes. So in total, we have log2 k * logk n = log2 n comparisons. This actually matches the best case, and to construct the worst case, we have to modify the tree somewhat. On one (and only one) arbitrary path from the root to a single leaf node, we increase the number of children from k to 2k. In this situation, the tree height ## Exphormer: Scaling transformers for graph-structured data DevFeed: [Exphormer: Scaling transformers for graph-structured data]() Original publisher: [Read original article]() Author: Google AI (noreply@blogger.com) Published: 2024-01-23T22:27:00Z Content type: article Language: en Sources: [Google Research]() Topics: [Graphs](), [Transformers](), [Machine learning](), [Natural language processing]() Tags: [deep-learning](), [graph](), [graphs](), [icml](), [machine-learning](), [theory](), [transformers]() ### AI overview This article explains the scalability challenges of graph transformers, whose complete attention graphs create quadratic computational and memory costs. It introduces Exphormer, a sparse attention framework designed for graph data that uses expander graphs from spectral graph theory. ### Source excerpt Posted by Ameya Velingker, Research Scientist, Google Research, and Balaji Venkatachalam, Software Engineer, Google Graphs, in which objects and their relations are represented as nodes (or vertices) and edges (or links) between pairs of nodes, are ubiquitous in computing and machine learning (ML). For example, social networks, road networks, and molecular structure and interactions are all domains in which underlying datasets have a natural graph structure. ML can be used to learn the properties of nodes, edges, or entire graphs. A common approach to learning on graphs are graph neural networks (GNNs), which operate on graph data by applying an optimizable transformation on node, edge, and global attributes. The most typical class of GNNs operates via a message-passing framework, whereby each layer aggregates the representation of a node with those of its immediate neighbors. Recently, graph transformer models have emerged as a popular alternative to message-passing GNNs. These models build on the success of Transformer architectures in natural language processing (NLP), adapting them to graph-structured data. The attention mechanism in graph transformers can be modeled by an interaction graph, in which edges represent pairs of nodes that attend to each other. Unlike message passing architectures, graph transformers have an interaction graph that is separate from the input graph. The typical interaction graph is a complete graph, which signifies a full attention mechanism that models direct interactions between all pairs of nodes. However, this creates quadratic computational and memory bottlenecks that limit the applicability of graph transformers to datasets on small graphs with at most a few thousand nodes. Making graph transformers scalable has been considered one of the most important research directions in the field (see the first open problem here). A natural remedy is to use a sparse interaction graph with fewer edges. Many sparse and efficient transformers ## Why does the chromaticity diagram look like that? DevFeed: [Why does the chromaticity diagram look like that?]() Original publisher: [Read original article]() Author: James Long Published: 2024-01-08T12:00:00Z Content type: article Language: en Sources: [James Long]() Topics: [math](), [pixel]() Tags: [article](), [color](), [image](), [pixel](), [rgb](), [theory]() ### AI overview This article explains why the CIE 1931 chromaticity diagram has its characteristic shape and how its colors are computed. It introduces color-matching functions for red, green, and blue, then describes calculating color mixtures and rendering sampled points into a two-dimensional image. ### Source excerpt (note: skipping interactive content block) ^30b920 I've always wanted to understand color theory, so I started reading about the XYZ color space which looked like it was the mother of all color spaces. I had no idea what that meant, but it was created in 1931 so studying 93-year old research seemed like a good place to start. When reading about the XYZ color space, this cursed image keeps popping up: By BenRG - Own work based on: CIExy1931.svg, Public Domain, https://commons.wikimedia.org/w/index.php?curid=7889658 I say "cursed" because I have no idea what that means. What the heck is that shape?? I couldn't find any reasonably clear answer to my question. It's obviously not a formula like x = func(y). Why is it that shape, and where did the colors come from? Obviously the edges are wavelengths which have a specific color, but how did the image above compute every pixel? I became obsessed with this question. Below is the path I took to try to answer it. I'll spoil the answer but it might not make sense until you read this article: the shape comes from how our eyes perceive red, green, and blue relative to each other. Skip to the last section if you want to see some direct examples. The fill colors inside the shape are another story, but a simple explanation is there is some math to calculate the mixture of colors and we can draw the above by sampling millions of points in the space and rendering them onto the 2d image. Let's dig in more. (note: skipping interactive content block) ^f0af28 (note: skipping interactive content block) ^82e29b Color matching functions The first place to start is color matching functions. These functions determine the strength of specific wavelengths (color) to contribute so that our eyes perceive a target wavelength (color). We have 3 color matching functions for red, green, and blue (at wavelengths 700, 546, and 435 respectively), and these functions specify how to mix RGB to so that we visually see a spectral color. More simply put: ima ## Graydon Hoare: 21 compilers and 3 orders of magnitude in 60 minutes DevFeed: [Graydon Hoare: 21 compilers and 3 orders of magnitude in 60 minutes]() Original publisher: [Read original article]() Published: 2022-02-27T14:47:26Z Content type: opinion Language: en Sources: [Lambda the Ultimate]() Topics: [Compiler](), [Code](), [Lisp](), [Machine learning]() Tags: [bytecode](), [compiler](), [compilers](), [efficiency](), [implementation](), [lisp](), [ml](), [pdf](), [presentation](), [talk](), [theory]() ### AI overview The article discusses Graydon Hoare's 2019 undergraduate talk about compiler design, summarizing different approaches ranging from large traditional compilers to variants using selective optimization, compiler-friendly languages, theory-driven tools, intermediate representations, interpretation, partial evaluation, or hand-written implementations. ### Source excerpt In 2019, Graydon Hoare gave a talk to undergraduates (PDF of slides) trying to communicate a sense of what compilers looked like from the perspective of people who did it for a living. I've been aware of this talk for over a year and meant to submit a story here, but was overcome by the sheer number of excellent observations. I'll just summarise the groups he uses: The giants: by which he means the big compilers that are built the old-fashioned way that throw massive resources at attaining efficiency The variants, which use tricks to avoid being so massive: Fewer optimisations: be traditional, but be selective and only the optimisations that really pay off Use compiler-friendly languages, by which he is really taking about languages that are good for implementing compilers, like Lisp and ML Theory-driven meta-languages, esp. how something like yacc allows a traditional Dragon-book style compiler to be written more easily Base compiler on a carefully designed IR that is either easy to compile or reasonable to bytecode-interpret Exercise discretion to have the object code be a mix of compiled and interpreted Use sophisticated partial evaluation Forget tradition and implement everything directly by hand I really recommend spending time working through these slides. While much of the material I was familiar with, enough was new, and I really appreciated the well-made points, shout-outs to projects that deserve more visibility, such as Nanopass compilers and CakeML, and the presentation of the Futamura projections, a famously tricky concept, at the undergraduate level. ## Latent Effects for Reusable Language Components DevFeed: [Latent Effects for Reusable Language Components]() Original publisher: [Read original article]() Published: 2021-10-14T14:02:48Z Content type: article Language: en Sources: [Lambda the Ultimate]() Topics: [Haskell](), [Programming](), [Development](), [implementation]() Tags: [abstraction](), [deferred](), [effects](), [examples](), [functional](), [haskell](), [modular](), [monad](), [programming-languages](), [semantics](), [theory]() ### AI overview The article discusses latent effects, a generic class of control-flow mechanisms for modular language definitions. It describes how function abstractions, lazy computations, and MetaML-like staging can be expressed and combined, with a full Haskell implementation and examples. ### Source excerpt Latent Effects for Reusable Language Components, by Birthe van den Berg, Tom Schrijvers, Casper Bach Poulsen, Nicolas Wu: The development of programming languages can be quite complicated and costly. Hence, much effort has been devoted to the modular definition of language features that can be reused in various combinations to define new languages and experiment with their semantics. A notable outcome of these efforts is the algebra-based "datatypes "a la carte" (DTC) approach. When combined with algebraic effects, DTC can model a wide range of common language features. Unfortunately, the current state of the art does not cover modular definitions of advanced control-flow mechanisms that defer execution to an appropriate point, such as call-by-name and call-by-need evaluation, as well as (multi-)staging. This paper defines latent effects, a generic class of such control-flow mechanisms. We demonstrate how function abstractions, lazy computations and a MetaML-like staging can all be expressed in a modular fashion using latent effects, and how they can be combined in various ways to obtain complex semantics. We provide a full Haskell implementation of our effects and handlers with a range of examples. Looks like a nice generalization of the basic approach taken by algebraic effects to more subtle contexts. Algebraic effects have been discussed here on LtU many times. I think this description from section 2.3 is a pretty good overview of their approach: LE&H is based on a different, more sophisticated structure than AE&H's free monad. This structure supports non-atomic operations (e.g., function abstraction, thunking, quoting) that contain or delimit computations whose execution may be deferred. Also, the layered handling is different. The idea is still the same, to replace bit by bit the structure of the tree by its meaning. Yet, while AE&H grows the meaning around the shrinking tree, LE&H grows little "pockets of meaning" around the individual nodes remaining in the ## Queueing theory for fun and practice #3: системы с потерями DevFeed: [Queueing theory for fun and practice #3: системы с потерями]() Original publisher: [Read original article]() Author: Ruslan Cheremin (noreply@blogger.com) Published: 2020-08-09T10:59:00Z Content type: article Language: ru Sources: [\>рабочие заметки]() Topics: [queueing theory]() Tags: [queueing-theory](), [tag-e5017782b67f](), [theory]() ### AI overview This article explains queueing systems with losses, where tasks or customers are dropped because of limited buffers, queue capacity, or expiration timeouts. It discusses Erlang-C1 and Erlang-A2 models and their qualitative stability properties. ### Source excerpt Начальник отдела челобитных Апполинарий Матвеевич любит порядок, поэтому просители могут ожидать его внимания только смиренно сидя в приемной, а не толкаясь возле дверей присутственного места - оттуда их гоняет казак Семен. Какова должна быть посадочная вместимость приемной, чтобы не более 1 просителя в день ушло не солоно хлебавши, если пропускная способность Апполинария Матвеевича не более ## Queueing theory for fun and practice #2: нагрузка и время отклика DevFeed: [Queueing theory for fun and practice #2: нагрузка и время отклика]() Original publisher: [Read original article]() Author: Ruslan Cheremin (noreply@blogger.com) Published: 2020-07-27T15:13:00Z Content type: tutorial Language: ru Sources: [\>рабочие заметки]() Topics: [queueing theory]() Tags: [fifo](), [queueing-theory](), [random](), [tag-e5017782b67f](), [theory]() ### AI overview This article explains how system load affects response time through internal queues and buffers. It discusses the characteristic J-curve, the difficulty of deriving a general analytical formula, and how queueing discipline, workload distributions, server allocation, and utilization influence waiting time. ### Source excerpt Во храме Божьей Матери Поклонской батюшка Иннокентий принимает исповедь у раба божьего обыкновенно минут за 10, а утешения жаждут около 5-и рабов божьих в час. Много ли стульев надобно поставить во храме, дабы исповеди ожидающие не толпились в праздности пред святым алтарем? "Массовое окормление паствы: пособие для начинающих" (редакция 3-я, неизданная) (Часть 2, начало: ТМО, square ## Queueing theory for fun and practice (#1): square root staffing, Little's law DevFeed: [Queueing theory for fun and practice (#1): square root staffing, Little's law]() Original publisher: [Read original article]() Author: Ruslan Cheremin (noreply@blogger.com) Published: 2020-07-24T12:35:00Z Content type: tutorial Language: ru Sources: [\>рабочие заметки]() Topics: [queueing theory]() Tags: [operations](), [queueing-theory](), [research](), [square](), [tag-88bad1e8f274](), [tag-e5017782b67f](), [theory]() ### AI overview This introductory article begins a series on queueing theory, presenting the author's plan to explain simple, broadly applicable principles, their practical uses, and the assumptions behind them. The planned topics include terminology, capacity and scaling, Little's law, response time versus utilization, and Erlang systems. The author notes that the material is an informal personal summary rather than an expert treatment. ### Source excerpt На 28-ом этаже центра разработки крупного инвестиционного банка есть 8 туалетных кабинок для людей, идентифицирующих себя с мужским гендером... ## Category theory DevFeed: [Category theory]() Original publisher: [Read original article]() Published: 2020-04-26T00:00:00Z Content type: tutorial Language: en Sources: [Vasya Drobushkov]() Topics: [Category Theory](), [Programming]() Tags: [category-theory](), [morphisms](), [object](), [programming](), [theory]() ### AI overview A personal synopsis of category theory covering categories, objects, morphisms, composition, universal constructions, order relations, monoids, terminal and initial objects, products, sums, and semirings. It also connects products and sums to programming concepts such as pairs, Either, and algebraic data types. ### Source excerpt Source Disclaimer This is short synopsis of great set of lectures. What is written here is by no means true, one should refer to original lectures or some books etc. This is written mostly for myself in case I wanted to revisit the topic in the future. Everything below is not "what it is" but mostly "how I understood that". So, there might be mistakes and so on. Category Category consists of: ## Waiting time paradox #1: автобусы, очереди, и хэш-таблицы DevFeed: [Waiting time paradox #1: автобусы, очереди, и хэш-таблицы]() Original publisher: [Read original article]() Author: Ruslan Cheremin (noreply@blogger.com) Published: 2019-04-30T07:37:00Z Content type: article Language: ru Sources: [\>рабочие заметки]() Topics: [queueing theory]() Tags: [queueing-theory](), [tag-35a4c7fafefe](), [tag-e5017782b67f](), [theory](), [time]() ### AI overview The article explains the waiting time paradox using bus arrivals. Although the average interval between buses may be 10 minutes, people are more likely to arrive during longer-than-average intervals, so their average wait can exceed the intuitive estimate of five minutes. It also introduces related examples involving hash-table searches and request processing. ### Source excerpt Ничего не доводи до крайности: человек, желающий трапезовать слишком поздно, рискует трапезовать на другой день поутру. (Козьма Прутков) ...парадокс времен ожидания, или почему автобуса приходится ждать дольше, чем казалось бы, почему успешный поиск в хэш-таблице скорее всего медленнее, чем неуспешный, и почему иногда среднее время обработки запроса можно уменьшить, если добавить в цикл ## Deep Probabilistic Modelling with Gaussian Processes #NIPS2017 DevFeed: [Deep Probabilistic Modelling with Gaussian Processes #NIPS2017]() Original publisher: [Read original article]() Published: 2017-12-04T12:00:00Z Content type: tutorial Language: en Sources: [Alex Korbonits]() Topics: [Tutorial](), [VAE](), [Machine learning](), [Deep learning](), [Neural Network](), [NeurIPS]() Tags: [analysis](), [bias](), [conference](), [data](), [deep-learning](), [gaussian](), [inference](), [modelling](), [neural-network](), [neurips](), [probabilistic](), [research](), [supervised-learning](), [theory](), [tutorial](), [unsupervised-learning](), [videos]() ### AI overview Lecture notes from a NeurIPS 2017 tutorial introduce deep probabilistic modelling with Gaussian processes, covering probabilistic neural networks, uncertainty, graphical models, and the computational challenge of inference. ### Source excerpt Lecture notes from Neil Lawrence's NIPS 2017 tutorial on deep probabilistic modelling with Gaussian processes -- from GPs to deep GPs and variational inference. ## Zero-Knowledge: Definitions and Theory DevFeed: [Zero-Knowledge: Definitions and Theory]() Original publisher: [Read original article]() Published: 2016-09-19T09:00:00Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [Zero-knowledge proof](), [Graphs](), [Protocol (disambiguation)](), [class]() Tags: [graph-isomorphism](), [np](), [permutation](), [protocol](), [theory](), [zero-knowledge]() ### AI overview This article explains definitions and theory behind zero-knowledge proofs. It contrasts graph isomorphism and 3-coloring protocols, focusing on their interaction between prover and verifier, cryptographic assumptions, transcript distributions, and simulation. ### Source excerpt The next Monday, when the fathers were all back at work, we kids were playing in a field. One kid says to me, "See that bird? What kind of bird is that?" I said, "I haven't the slightest idea what kind of a bird it is." He says, "It's a brown-throated thrush. Your father doesn't teach you anything!" But it was the opposite. He had already taught me: "See that bird? ## Deriving the Reddit Formula DevFeed: [Deriving the Reddit Formula]() Original publisher: [Read original article]() Author: Evan Miller Published: 2015-07-14T15:15:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [Reddit](), [math](), [Users]() Tags: [analysis](), [expression](), [lambda](), [math](), [reddit](), [theory]() ### AI overview Evan Miller derives a Reddit "hot" formula using expected-utility theory. The analysis explains the formula's constants, logarithm, absolute value, and lack of current time, estimates optimization for users who visit every 5.43 hours, and proposes a caveated amendment. ### Source excerpt Deriving the Reddit Formula -- What can expected-utility theory tell us about Reddit's "hot" formula? ## Deep Learning with Python DevFeed: [Deep Learning with Python]() Original publisher: [Read original article]() Published: 2015-05-04T11:49:39Z Content type: tutorial Language: en Sources: [Alex Korbonits]() Topics: [Deep learning](), [Python](), [Deep neural networks](), [Artificial Intelligence]() Tags: [deep-learning](), [introduction](), [practical](), [python](), [theory]() ### AI overview A practical introduction to deep learning with Python. The article explains what deep learning is, confirms that Python can be used as glue between components written in other languages, and outlines ways to get started while noting their advantages and pitfalls. ### Source excerpt A practical introduction to deep learning with Python -- from zero to insights in minutes, no theory required. ## Don't use the CAP theorem for packet losses DevFeed: [Don't use the CAP theorem for packet losses]() Original publisher: [Read original article]() Author: Nicolas Liochon (noreply@blogger.com) Published: 2015-03-18T15:16:00Z Content type: article Language: en Sources: [Nicolas Liochon]() Topics: [CAP theorem](), [consistency](), [Networks](), [Protocol (disambiguation)]() Tags: [acid](), [availability](), [cap-theorem](), [consistency](), [database](), [distributed-system](), [durability](), [ip](), [latency](), [messages](), [network](), [networks](), [partition](), [protocol](), [tcp](), [theory](), [udp]() ### AI overview The article explains why ordinary packet loss on IP networks should not automatically be treated as a CAP theorem partition. It distinguishes congestion-related packet loss from the arbitrary message loss assumed by the CAP model and discusses how TCP responds to congestion. ### Source excerpt In the previous post, we looked at this common saying: "nodes fail, network packets get lost, partitions happen so you need to use CAP to understand your trade-offs." We saw that node failures were not partitions. What about packet losses? Most distributed applications use TCP or UDP on top of IP, and it is well known that IP is an asynchronous protocol and that it can lose packets. So should we use all the results from the theory of asynchronous networks? Must we use CAP to do some trade-offs if we are using a network that can drop packets? The answer is no. The root issue lies in the incompleteness of our description of IP. "IP is an asynchronous protocol and it can lose packets" is true, but incomplete, and this incompleteness is misleading. Let's discuss why. CAP - The usual reminder CAP says that a distributed system cannot be Consistent, Available and Partition tolerant. We use here the definitions from the proof [C2]. Consistent is: [C2] "Atomic, linearizable, consistency [...]. There must exist a total order on all operations such that each operation looks as if it were completed at a single instant. This is equivalent to requiring requests of the distributed shared memory to act as if they were executing on a single node, responding to operations one at a time." Available is: [C2] "For a distributed system to be continuously available, every request received by a non-failing node in the system must result in a response." Partition is: [C2] "The network will be allowed to lose arbitrarily many messages sent from one node to another. When a network is partitioned, all messages sent from nodes in one component of the partition to nodes in another component are lost." How can you lose packets on a network Packet loss happens when a network equipment receives more messages that it can send. Losing packets is common on an IP network. A TCP connection tries to use most of the bandwidth available. It sends nearly as many packets as it can, until it loses some. Losi ## A Proofless Introduction to Information Theory DevFeed: [A Proofless Introduction to Information Theory]() Original publisher: [Read original article]() Published: 2015-02-16T09:00:00Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [Encoding](), [digital]() Tags: [channel](), [coding-theory](), [communication](), [compression](), [digital](), [encoding](), [entropy](), [error](), [error-correction](), [example](), [explain](), [hamming](), [independent](), [information](), [information-theory](), [kolmogorov-complexity](), [language](), [markov](), [mathematics](), [messages](), [models](), [probabilistic-method](), [problems](), [shannon](), [theory]() ### AI overview A proofless introduction to information theory explains noiseless and noisy communication problems, encoding schemes, entropy, and how entropy differs from Kolmogorov complexity. ### Source excerpt There are two basic problems in information theory that are very easy to explain. Two people, Alice and Bob, want to communicate over a digital channel over some long period of time, and they know the probability that certain messages will be sent ahead of time. For example, English language sentences are more likely than gibberish, and "Hi" is much more likely than "asphyxiation." The problems are: Say communication is very expensive. ## The Complexity of Communication DevFeed: [The Complexity of Communication]() Original publisher: [Read original article]() Published: 2014-11-10T09:00:25Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [Computer science](), [Algorithms, Complexity](), [Streaming](), [circuit]() Tags: [communication](), [communication-complexity](), [computational-complexity](), [fourier-analysis](), [information-theory](), [log-rank-conjecture](), [lower-bounds](), [matrices](), [streaming-algorithms](), [theory]() ### AI overview This tutorial introduces communication complexity: how much information two parties must exchange to jointly compute a function of separate inputs. It presents the basic two-player model and explains the subject's use in proving lower bounds, including applications to circuit design and streaming algorithms. ### Source excerpt satellite One of the most interesting questions posed in the last thirty years of computer science is to ask how much "information" must be communicated between two parties in order for them to jointly compute something. One can imagine these two parties living on distant planets, so that the cost of communicating any amount of information is very expensive, but each person has an integral component of the answer that the other does not. ## When Greedy Algorithms are Good Enough: Submodularity and the (1--1/e)-Approximation DevFeed: [When Greedy Algorithms are Good Enough: Submodularity and the (1--1/e)-Approximation]() Original publisher: [Read original article]() Published: 2014-07-07T10:00:01Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [Algorithms, Complexity](), [Algorithms](), [Computer science](), [Mathematics]() Tags: [algorithm](), [algorithms](), [approximation](), [approximation-algorithms](), [computer-science](), [coverage](), [greedy](), [greedy-algorithm](), [matroids](), [optimization](), [submodularity](), [theory]() ### AI overview This article explains when greedy algorithms can produce optimal or near-optimal solutions. It introduces submodularity as a diminishing-returns property and discusses how it can provide a reasonably good approximation to the optimal solution. ### Source excerpt Greedy algorithms are among the simplest and most intuitive algorithms known to humans. Their name essentially gives their description: do the thing that looks best right now, and repeat until nothing looks good anymore or you're forced to stop. Some of the best situations in computer science are also when greedy algorithms are optimal or near-optimal. There is a beautiful theory of this situation, known as the theory of matroids. We haven't covered matroids on this blog (edit: we did), but in this post we will focus on the next best thing: when the greedy algorithm guarantees a reasonably good approximation to the optimal solution. ## Martingales and the Optional Stopping Theorem DevFeed: [Martingales and the Optional Stopping Theorem]() Original publisher: [Read original article]() Published: 2014-03-03T10:00:38Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [math]() Tags: [2-sat](), [conditional-probability](), [expectation](), [gambling](), [martingales](), [mathematics](), [optional-stopping-theorem](), [primer](), [probability-theory](), [random](), [random-variables](), [randomized-algorithm](), [stochastic-processes](), [theory]() ### AI overview This primer introduces martingales as models of fair betting games and explains their connection to probability theory. It begins with a geometric-distribution exercise involving repeated die throws, then introduces the ABRACADABRA problem using a monkey typing random letters. ### Source excerpt This is a guest post by my colleague Adam Lelkes. The goal of this primer is to introduce an important and beautiful tool from probability theory, a model of fair betting games called martingales. In this post I will assume that the reader is familiar with the basics of probability theory. For those that need to refresh their knowledge, Jeremy's excellent primers (1, 2) are a good place to start. ## Probably Approximately Correct -- a Formal Theory of Learning DevFeed: [Probably Approximately Correct -- a Formal Theory of Learning]() Original publisher: [Read original article]() Published: 2014-01-02T18:45:51Z Content type: tutorial Language: en Sources: [Jeremy Kun]() Topics: [Learning](), [Machine Learning & Artificial Intelligence](), [Computer science](), [Mathematics]() Tags: [learning](), [learning-theory](), [machine-learning](), [mathematics](), [occam-s-razor](), [pac-learning](), [theory]() ### AI overview A mathematical introduction to PAC learning, a foundational framework in computational learning theory. The article develops basic definitions, explains PAC-learnability through interval examples, and places the theory in its historical context. ### Source excerpt In tackling machine learning (and computer science in general) we face some deep philosophical questions. Questions like, "What does it mean to learn?" and, "Can a computer learn?" and, "How do you define simplicity?" and, "Why does Occam's Razor work? (Why do simple hypotheses do well at modelling reality?)" In a very deep sense, learning theorists take these philosophical questions -- or at least aspects of them -- give them fleshy mathematical bodies, and then answer them with theorems and proofs. ## Introducing Categories DevFeed: [Introducing Categories]() Original publisher: [Read original article]() Published: 2013-04-24T06:48:01Z Content type: article Language: en Sources: [Jeremy Kun]() Topics: [Category Theory]() Tags: [categories](), [category-theory](), [theory]() ### AI overview An introductory tutorial defines category theory and explains its purpose as a unified language for organizing mathematical structures across disciplines. It introduces examples and prepares readers for the formal definition of categories. ### Source excerpt For a list of all the posts on Category Theory, see the Main Content page. It is time for us to formally define what a category is, to see a wealth of examples. In our next post we'll see how the definitions laid out here translate to programming constructs. As we've said in our soft motivational post on categories, the point of category theory is to organize mathematical structures across various disciplines into a unified language. [Next page]()