# hash A cryptographic hash function produces a digest from data; the resulting value is called a hash or hash value. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Python sets and dictionaries can have quadratic-time performance DevFeed: [Python sets and dictionaries can have quadratic-time performance]() Original publisher: [Read original article]() Author: Daniel Lemire Published: 2026-09-03T14:01:45Z Content type: article Language: en Sources: [Daniel Lemire]() Topics: [Python](), [Data structures](), [hash](), [CPU Cache]() Tags: [array](), [cpu-cache](), [data-structure](), [hash](), [performance](), [python]() ### AI overview The article explains why Python sets and dictionaries are not formally constant-time in all cases. Hash collisions can make insertions and membership checks take quadratic time, while growing data structures can also incur reallocation and slower memory access as they exceed CPU cache capacity. ### Source excerpt In Python, the dict data structure is the conventional key-value structure. E.g., you might store a list of names as keys and have their phone numbers as values. Valentin Ignatev wrote this amusing post on X: It is indeed widely believed that, in the strict sense, the dict data structure and its companion, the set ... Continue reading Python sets and dictionaries can have quadratic-time performance ## 【eBPF 内核实现深度拆解】从验证器到 JIT，从 BTF 到调度器 DevFeed: [【eBPF 内核实现深度拆解】从验证器到 JIT，从 BTF 到调度器]() Original publisher: [Read original article]() Author: Liao Tonglang Published: 2026-06-12T00:00:00Z Content type: article Language: zh Sources: [土法炼钢 - 系统与基础设施]() Topics: [eBPF](), [Linux](), [JIT](), [Benchmark](), [clang](), [hash](), [benchmarking](), [RISC-V](), [ast-matchers]() Tags: [arm](), [array](), [benchmark](), [bpf-jit](), [bpf-maps](), [bpf-verifier](), [btf](), [clang](), [co-re](), [cpu](), [ebpf](), [fentry](), [hash](), [jit](), [kernel](), [libbpf](), [linux](), [linux-kernel](), [precision](), [risc-v](), [sched-ext](), [trampoline](), [x86](), [xdp]() ### AI overview This Chinese-language series systematically explains eBPF's Linux kernel implementation, covering the BPF instruction set and registers, verifier algorithms, JIT compilation, map data structures and concurrency, helper type checking, BTF and CO-RE relocation, libbpf loading, trampolines, and sched_ext interfaces. It is aimed at engineers who want to understand eBPF kernel source code and build production BPF programs. ### Source excerpt eBPF 内核虚拟机内部实现系统讲解：BPF 指令集与寄存器机器、验证器的抽象解释与状态裁剪、JIT 编译器后端、Map 各类型的并发与内存模型、helper 函数注册与类型检查、BTF 格式规范与 CO-RE 重定位引擎、libbpf 加载器工程、fentry/fexit 蹦床机制、sched_ext 调度器内核接口。面向想读懂 eBPF 内核源码、写生产级 BPF 程序的系统工程师。 ## Recent LLVM hash table improvements DevFeed: [Recent LLVM hash table improvements]() Original publisher: [Read original article]() Published: 2026-06-07T07:00:00Z Content type: article Language: en Sources: [MaskRay]() Topics: [LLVM](), [hash](), [Algorithms](), [benchmarking]() Tags: [algorithm](), [compiler](), [data-structures](), [hash](), [improvements](), [llvm](), [performance]() ### AI overview The article reviews recent improvements to LLVM hash tables, including replacing quadratic probing and tombstone or empty-key sentinels with linear probing, Algorithm R deletion, and bit-array occupancy. It also discusses pointer and iterator invalidation behavior and reports performance improvements in DenseMap. ### Source excerpt LLVM has several hash tables. They used quadratic probing with in-band sentinel keys (empty, tombstone); recent work has been replacing that with linear probing with tombstone key removed. DenseMap (replacement for std::unordered_map): DenseMapInfo::getEmptyKey() / getTombstoneKey(). DenseSet: implemented using DenseMap compiler-rt/lib/sanitizer_common/sanitizer_dense_map.h ports the implementation for sanitizers. SmallPtrSet (replacement for std::unordered_set): hard-coded -1 (empty) and -2 (tombstone). StringMap (replacement for std::unordered_map) StringSet: implemented using StringMap FoldingSet (uniquing/hash-consing container, not a general map) For the open-addressed DenseMap and SmallPtrSet, pointers, references, and iterators are invalidated by insert. StringMap is different: each entry lives in a heap-allocated StringMapEntry node, so entry pointers survive grow. std::unordered_map, being node-based, keeps surviving-element pointers valid across both insert and erase and only invalidates the erased element's own iterator. LLVM code rarely needs that stronger contract -- callers do not hold long-lived references into the container across mutation -- and that gap is what gives pass to relocating erase and bit-array occupancy. Recently, Tombstones have been removed from DenseMap and SmallPtrSet. erase() also invalidates pointers. DenseMap has also retired its empty-key sentinel, leading to significant performance improvements. DenseMap with integer keys (int/unsigned/size_t) had -1/-2 reserved -- a footgun, now fixed. StringMap got Algorithm R deletion too. Its entries are separately heap-allocated, so erase keeps entry pointers valid but invalidates iterators; erase-while-iterating moved to remove_if. FoldingSet dropped chaining for linear probing plus Algorithm R; the intrusive next-in-bucket pointer became a cached 32-bit hash. ## Only 17% of all 64-bit Integers are products of two 32-bit integers DevFeed: [Only 17% of all 64-bit Integers are products of two 32-bit integers]() Original publisher: [Read original article]() Author: Daniel Lemire Published: 2026-05-22T01:16:35Z Content type: article Language: en Sources: [Daniel Lemire]() Topics: [Programming](), [hash]() Tags: [cryptographic](), [hash](), [numbers](), [precision](), [programming]() ### AI overview The article examines what fraction of 64-bit integers can be represented as the full product of two 32-bit integers. It presents the result that only 17% can be produced this way and relates the question to hash-function design and multiplication behavior. ### Source excerpt In software programming, the product between two integers is often computed to a fixed number of bits with overflow. Consider 8-bit integers. If you multiply 127 by 127, you get back the number 1 as an 8-bit unsigned integer, with an overflow. The actual full product is 16129. To represent 16129, you typically use 16 ... Continue reading Only 17% of all 64-bit Integers are products of two 32-bit integers ## Fighting Hyrum's Law in LLVM DevFeed: [Fighting Hyrum's Law in LLVM]() Original publisher: [Read original article]() Published: 2026-05-10T07:00:00Z Content type: article Language: en Sources: [MaskRay]() Topics: [LLVM](), [Compiler](), [reproducible builds](), [hash]() Tags: [clang](), [compiler](), [hash](), [lld](), [llvm](), [reproducible-builds](), [test]() ### AI overview This article examines how LLVM can develop dependencies on unspecified or incidental behavior under Hyrum's Law, causing output variation that harms reproducible builds, bisection, and bug reports. It describes hash-seed perturbation, reverse container iteration, and iterator invalidation checks as mechanisms for exposing such dependencies. ### Source excerpt With a sufficient number of users of an API, it does not matter what you promise in the contract: all observable behaviors of your system will be depended on by somebody. -- Hyrum's Law In a compiler, the most common form of Hyrum's Law is dependence on unspecified behavior -- hash bucket order, the order of equal elements after std::sort, padding offsets. The same framing covers a few cases that are technically undefined behavior (use of an invalidated iterator) or plain incidental properties (ABI struct layout, ELF section offsets). When the compiler itself harbors such a dependency, the symptom is usually output that varies build-to-build: an unstable sort that lands differently after the standard library changes, a hash map whose iteration order shifts when the hash function does. Occasionally the variation is run-to-run within a single build -- DenseMap keys with an ASLR-derived seed reorder buckets each invocation. Either way, reproducible builds, bisection, and bug reports all assume same input -> same output, and a stealth Hyrum dependency breaks that. This post surveys some mechanisms that perturb the contract's blind spots so dependencies cannot quietly form. ## Cuckoo Filters: Cache-Friendly Membership Checks With Deletions DevFeed: [Cuckoo Filters: Cache-Friendly Membership Checks With Deletions]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-03-17T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [CPU Cache](), [Caching](), [hash]() Tags: [cache](), [capacity](), [cpu-cache](), [cuckoo-filter](), [data-structures](), [false-positive](), [hash](), [insert](), [performance](), [probabilistic](), [spatial-locality](), [systems]() ### AI overview This article explains how Cuckoo filters support deletions while improving CPU cache behavior compared with counting Bloom filters. They check two specific buckets using compact fingerprints, but insertions can fail when kick-out chains exceed their limit, requiring capacity planning or overflow handling. ### Source excerpt A Cuckoo filter stores a one-to-two byte fingerprint in a hash table and finds it by checking exactly two buckets, the primary index and its XOR-derived alternate, instead of k random bit positions scattered across a large array. That spatial locality is the whole win on real CPUs. The trade-off is a hard edge: when the kick-out chain exceeds its limit, the insert fails outright. ## Shazam finds songs by voting on time offsets, not by comparing audio DevFeed: [Shazam finds songs by voting on time offsets, not by comparing audio]() Original publisher: [Read original article]() Author: hello@ankit-rana.com Published: 2026-03-16T00:00:00Z Content type: tutorial Language: en Sources: [Ankit Rana | Mechanical Sympathy]() Topics: [Algorithm](), [hash](), [Databases]() Tags: [algorithm](), [algorithms](), [audio-fingerprinting](), [databases](), [hash](), [indexing](), [lookup](), [music-recognition](), [query](), [system-design]() ### AI overview The article explains how Shazam recognizes songs from short, noisy recordings. Instead of comparing audio similarity, it extracts spectrogram peaks, combines nearby peaks into hashes, and uses an inverted index to find tracks whose hash matches share a common time offset. The production system beyond the public 2003 paper is noted as unavailable. ### Source excerpt Shazam does not compare audio. It reduces each track to spectrogram peaks, pairs nearby peaks into ~32-bit hashes, and looks those up in an inverted index. A match is declared when many hashes from the sample agree on a single time offset into one track. The offset histogram is the whole trick: noise scatters offsets randomly, a real match stacks them into a spike. ## Analysis of Dahua CCTV Camera Authentication DevFeed: [Analysis of Dahua CCTV Camera Authentication]() Original publisher: [Read original article]() Published: 2025-12-01T23:00:00Z Content type: tutorial Language: en Sources: [Dennis Yurichev]() Topics: [webcam](), [hash](), [Python]() Tags: [auth](), [camera](), [hash](), [python](), [tcp]() ### AI overview A pentesting-oriented analysis of Dahua CCTV camera authentication over its proprietary TCP protocol. It describes a fake Python server used to observe a camera login and presents the known portion of the hash calculation, while noting that the remaining 16 bytes are not understood and no Hashcat workflow was found. ### Source excerpt [Pentesting] Dahua (CCTV cameras) auth ## A simplified p-fast trie for prefix and predecessor searches DevFeed: [A simplified p-fast trie for prefix and predecessor searches]() Original publisher: [Read original article]() Published: 2025-08-06T17:19:09Z Content type: article Language: en Sources: [Tony Finch's blog]() Topics: [Algorithm](), [hash](), [Cache](), [Query (disambiguation)]() Tags: [algorithm](), [cache](), [hash](), [query]() ### AI overview The article presents a simplified revision of a p-fast trie, a wide fan-out variant of an x-fast trie. It describes a hash-table layout for storing unique prefixes and outlines longest-prefix, predecessor, and successor searches. ### Source excerpt Previously, I wrote some sketchy ideas for what I call a p-fast trie, which is basically a wide fan-out variant of an x-fast trie. It allows you to find the longest matching prefix or nearest predecessor or successor of a query string in a set of names in O(log k) cache misses, where k is the key length. My initial sketch was more complicated and greedy for space than necessary, so here's a simplified revision. ("p" now stands for prefix.) layout A p-fast trie stores a lexicographically ordered set of names. A name is a sequence of characters from some small-ish character set. For example, DNS names can be represented as a set of about 50 letters, digits, punctuation and escape characters, usually one per byte of name. Names that are arbitrary bit strings can be split into chunks of 6 bits to make a set of 64 characters. Every unique prefix of every name is added to a hash table. An entry in the hash table contains: A shared reference to the closest name lexicographically greater than or equal to the prefix. Multiple hash table entries will refer to the same name. A reference to a name might instead be a reference to a leaf object containing the name. The length of the prefix. To save space, each prefix is not stored separately, but implied by the combination of the closest name and prefix length. A bitmap with one bit per possible character, corresponding to the next character after this prefix. For every other prefix that matches this prefix and is one character longer than this prefix, a bit is set in the bitmap corresponding to the last character of the longer prefix. search The basic algorithm is a longest-prefix match. Look up the query string in the hash table. If there's a match, great, done. Otherwise proceed by binary chop on the length of the query string. If the prefix isn't in the hash table, reduce the prefix length and search again. (If the empty prefix isn't in the hash table then there are no names to find.) If the prefix is in the hash table, check ## p-fast trie: lexically ordered hash map DevFeed: [p-fast trie: lexically ordered hash map]() Original publisher: [Read original article]() Published: 2025-08-04T20:52:21Z Content type: article Language: en Sources: [Tony Finch's blog]() Topics: [hash](), [Query (disambiguation)](), [Rust]() Tags: [array](), [bits](), [hash](), [map](), [maps](), [query]() ### AI overview This article sketches the p-fast trie, a proposed lexically ordered hash map that replaces a qp-trie's tree and interior pointers with stratified hash-map levels keyed by prefixes. It describes O(1) exact-match lookups and O(log k) predecessor and successor searches, while noting that the practical benefit is uncertain. ### Source excerpt Here's a sketch of an idea that might or might not be a good idea. Dunno if it's similar to something already described in the literature - if you know of something, please let me know via the links in the footer! The gist is to throw away the tree and interior pointers from a qp-trie. Instead, the p-fast trie is stored using a hash map organized into stratified levels, where each level corresponds to a prefix of the key. Exact-match lookups are normal O(1) hash map lookups. Predecessor / successor searches use binary chop on the length of the key. Where a qp-trie search is O(k), where k is the length of the key, a p-fast trie search is O(log k). This smaller O(log k) bound is why I call it a "p-fast trie" by analogy with the x-fast trie, which has O(log log N) query time. (The "p" is for popcount.) I'm not sure if this asymptotic improvement is likely to be effective in practice; see my thoughts towards the end of this note. layout A p-fast trie consists of: Leaf objects, each of which has a name. Each leaf object refers to its successor forming a circular linked list. (The last leaf refers to the first.) Multiple interior nodes refer to each leaf object. A hash map containing every (strict) prefix of every name in the trie. Each prefix maps to a unique interior node. Names are treated as bit strings split into chunks of (say) 6 bits, and prefixes are whole numbers of chunks. An interior node contains a (1<<6) == 64 wide bitmap with a bit set for each chunk where prefix+chunk matches a key. Following the bitmap is a popcount-compressed array of references to the leaf objects that are the closest predecessor of the corresponding prefix+chunk key. Prefixes are strictly shorter than names so that we can avoid having to represent non-values after the end of a name, and so that it's OK if one name is a prefix of another. The size of chunks and bitmaps might change; 6 is a guess that I expect will work OK. For restricted alphabets you can use something like my DNS trie n ## How Cursor Indexes Codebases Fast DevFeed: [How Cursor Indexes Codebases Fast]() Original publisher: [Read original article]() Author: Engineer's Codex Published: 2025-05-10T16:28:09Z Content type: article Language: en Sources: [Engineer's Codex]() Topics: [cursor](), [Software Engineering](), [Code](), [hash](), [Embeddings](), [structure](), [file](), [implementation]() Tags: [code](), [cursor](), [embeddings](), [hash](), [index](), [indexing](), [software-engineering]() ### AI overview This article explains how Cursor indexes codebases using Merkle trees. It describes local code chunking, hash-tree construction and synchronization with Cursor's server, followed by embedding generation through OpenAI's embedding API or a custom embedding model. ### Source excerpt Merkle Trees in the real world ## Inserting One New Element into Hashes of Varying Sizes DevFeed: [Inserting One New Element into Hashes of Varying Sizes]() Original publisher: [Read original article]() Author: Pat Shaughnessy Published: 2025-02-04T00:00:00Z Content type: tutorial Language: en Sources: [Pat Shaughnessy]() Topics: [Ruby](), [hash](), [Benchmark](), [Code]() Tags: [benchmark](), [hash](), [internals](), [ruby](), [table](), [updating-ruby-under-a-microscope]() ### AI overview This article presents a Ruby experiment measuring the time required to insert one new element into hashes of different sizes. Using Ruby 3.4.1, it observes timing spikes when the hash grows to certain sizes, consistent with rehashing or redistribution. ### Source excerpt I've started working on a new edition of Ruby Under a Microscope that covers Ruby 3.x. I'm working on this in my spare time, so it will take a while. Leave a comment or drop me a line and I'll email you when it's finished. RUM includes a serie ## Updating Ruby Under a Microscope DevFeed: [Updating Ruby Under a Microscope]() Original publisher: [Read original article]() Author: Pat Shaughnessy Published: 2025-01-28T00:00:00Z Content type: article Language: en Sources: [Pat Shaughnessy]() Topics: [Ruby](), [hash](), [Computer science](), [C]() Tags: [hash](), [internals](), [ruby](), [updating](), [updating-ruby-under-a-microscope]() ### AI overview An excerpt from a planned new edition of Ruby Under a Microscope explains how Ruby uses hash tables internally. It introduces hash-table organization and describes how Ruby stores keys, values, entries, bins, and related C structures. ### Source excerpt Ruby stores much of its own internal data in hash tables. I've started working on a new edition of ## A Curious Case of Mistaken Identity: How Lambdas Break Data Class Hashing DevFeed: [A Curious Case of Mistaken Identity: How Lambdas Break Data Class Hashing]() Original publisher: [Read original article]() Author: Matt McKenna Published: 2024-11-14T21:44:31Z Content type: tutorial Language: en Sources: [Matt McKenna]() Topics: [Kotlin](), [hashing](), [hash](), [consistency]() Tags: [android](), [behavior](), [class](), [data-class](), [hashcode](), [hashing](), [kotlin](), [maintenance]() ### AI overview This Kotlin article explains why data class instances containing lambdas may compare as unequal and produce different hash-based behavior. Each lambda instance has a distinct identity, so the article recommends excluding the callback from equality and hashCode calculations, while noting the resulting maintenance cost. ### Source excerpt Introduction: The Scene of the Crime It was a dark and stormy night. My hands were flying across the keys when suddenly the codebase began to exhibit strange behavior. Hashes, which once returned the same values for identical objects, suddenly became... ## Immutable URLs can simplify deployments DevFeed: [Immutable URLs can simplify deployments]() Original publisher: [Read original article]() Published: 2024-03-14T00:00:00Z Content type: article Language: en Sources: [Garnix Blog]() Topics: [hash]() Tags: [deployments](), [hash](), [idea](), [simplify]() ### AI overview The article explores making URLs immutable and suggests that this approach could improve and simplify deployments. ### Source excerpt What happens if we make URLs immutable? A somewhat unusual idea that can substantially improve and simplify deployments. ## Mainnet Merge Announcement DevFeed: [Mainnet Merge Announcement]() Original publisher: [Read original article]() Author: EF Protocol Support Published: 2022-08-24T00:00:00Z Content type: release Language: en Sources: [Ethereum Foundation Blog]() Topics: [Ethereum](), [upgrade](), [proof of work](), [Network](), [hash](), [JSON](), [Remote Procedure Call (RPC)]() Tags: [consensus](), [ethereum](), [execution](), [hash](), [json](), [network](), [node](), [proof-of-work](), [protocol-announcements](), [rpc](), [upgrade]() ### AI overview Ethereum's mainnet transition from proof-of-work to proof-of-stake, known as The Merge, was scheduled in two phases: the Bellatrix upgrade on the Beacon Chain followed by the Paris transition when the execution layer reached the specified Terminal Total Difficulty. ### Source excerpt Ethereum is moving to proof-of-stake! The transition, known as The Merge, must first be activated on the Beacon Chain with the Bellatrix upgrade. After this, the proof-of-work chain will migrate to proof-of-stake upon hitting a specific Total Difficulty value. The Bellatrix upgrade is scheduled for epoch 144896 on the Beacon... ## Supply chain integrity with GoReleaser using Go mod proxy DevFeed: [Supply chain integrity with GoReleaser using Go mod proxy]() Original publisher: [Read original article]() Author: Carlos Alexandro Becker Published: 2021-08-23T00:00:00Z Content type: tutorial Language: en Sources: [Carlos Becker]() Topics: [integrity](), [Go Language](), [proxy](), [builds](), [hash](), [opensource]() Tags: [builds](), [go](), [hash](), [making](), [opensource](), [proxy](), [supply-chain-integrity]() ### AI overview This tutorial explains how to verify Go binaries built with GoReleaser by using the Go module proxy and comparing module hashes. It also discusses signing, reproducible builds, and limitations involving dependencies and non-open-source GoReleaser Pro. ### Source excerpt Since the infamous SolarWinds attack, supply chain integrity is something a lot of people are discussing and working on. ## Searching for RH Counterexamples -- Scaling Up DevFeed: [Searching for RH Counterexamples -- Scaling Up]() Original publisher: [Read original article]() Published: 2021-02-16T09:00:00Z Content type: article Language: en Sources: [Jeremy Kun]() Topics: [Mathematics](), [hashing](), [scaling](), [sha256](), [hash](), [Database]() Tags: [architecture](), [disk-space](), [hashing](), [mathematics](), [performance](), [postgres](), [programming](), [refactor](), [riemann-hypothesis](), [scaling](), [sha256](), [software]() ### AI overview This article describes scaling a search for counterexamples to the Riemann Hypothesis. It focuses on reducing storage by keeping deterministic SHA-256 summaries instead of every witness value and refactoring the application into a worker architecture. ### Source excerpt We're ironically searching for counterexamples to the Riemann Hypothesis. Setting up Pytest Adding a Database Search Strategies Unbounded integers Deploying with Docker Performance Profiling Last time we made the audacious choice to remove primary keys from the RiemannDivisorSums table for performance reasons. To help with that, we will do two things in this post Reduce the storage footprint of the whole application (it was 60 GiB when it crashed, and we got up to 84 prime factors). ## Re-Introducing Hash Indexes in PostgreSQL DevFeed: [Re-Introducing Hash Indexes in PostgreSQL]() Original publisher: [Read original article]() Author: Haki Benita Published: 2021-01-10T22:00:00Z Content type: tutorial Language: en Sources: [Haki Benita]() Topics: [PostgreSQL](), [hash](), [Data structures](), [Databases]() Tags: [articles](), [data-structure](), [database](), [hash](), [index](), [indexes](), [performance](), [postgresql](), [sql]() ### AI overview This tutorial explains how PostgreSQL hash indexes work, including hash functions, buckets, tuple pointers, and collisions. It presents hash indexes as an option that can outperform B-Tree indexes under some circumstances. ### Source excerpt There is a type of index you are probably not using, and may have never even heard of. It is wildly unpopular, and until a few PostgreSQL versions ago it was highly discouraged and borderline unusable, but under some circumstances it can out-perform even a B-Tree index. ## A day in the life of a professional software engineer DevFeed: [A day in the life of a professional software engineer]() Original publisher: [Read original article]() Author: Graham King Published: 2020-12-20T23:05:31Z Content type: article Language: en Sources: [Graham King]() Topics: [Data structures](), [Software](), [coding](), [hash](), [implementation]() Tags: [functions](), [hash](), [implementation](), [software](), [software-engineer]() ### AI overview The article presents a humorous account of a typical day for a professional software engineer, describing work with linked lists, string search using Rabin-Karp and rolling hashes, sorting algorithms, Big O analysis, and tree data structures. ### Source excerpt Just a normal day reversing linked lists on the whiteboard. ## SlowerLogLog DevFeed: [SlowerLogLog]() Original publisher: [Read original article]() Author: Evan Miller Published: 2020-02-06T18:00:00Z Content type: article Language: en Sources: [Evan Miller]() Topics: [Data structures](), [Algorithm](), [math](), [hash](), [implementation]() Tags: [algorithm](), [data-structure](), [hash](), [implementation](), [math](), [probabilistic]() ### AI overview This article introduces SlowerLogLog, a variation of HyperLogLog for estimating the number of unique elements in a set. It explains how Maximum Likelihood Estimation can estimate cardinality and standard error, support small cardinalities, and avoid requiring a power-of-two register count. The method is easier to derive and implement but much more computationally expensive because each element is hashed once per register. ### Source excerpt What rolls down stairs and counts unique items imprecisely? HyperLogLog. Also SlowerLogLog ## Calling Python Methods - Subtleties with and without Parens DevFeed: [Calling Python Methods - Subtleties with and without Parens]() Original publisher: [Read original article]() Author: Fuzzygroup Published: 2019-10-28T00:00:00Z Content type: tutorial Language: en Sources: [Scott Johnson]() Topics: [Python](), [Ruby](), [debug](), [hash]() Tags: [debugger](), [function](), [hash](), [python](), [ruby](), [syntax](), [trace]() ### AI overview This article explains a Python syntax mistake caused by referencing a function without calling it. It contrasts Python with Ruby, where methods can be called with or without parentheses, and describes using pdb.set_trace() to identify the resulting type error. ### Source excerpt Photo courtesy of my boss and friend, Dave Sifry who is in London right now attending MozFest; lucky guy! In the continuing saga of "Once upon a time I did ruby exclusively, now I do python and ruby side by side and I keep being stupid", I have discovered something subtle and interesting about how python works internally. Let's start with a function definition in python: expert_name = "Anti Semitic Speech Expert" expert_version = 0.01 def make_opinion_template(): template = {} template["name"] = "slurs" template["raw"] = None template["score"] = None template["opinion"] = None template["expert_name"] = expert_name template["expert_version"] = expert_version return template The first two lines are globals and all the next line does is construct a hash (ok dict if I have to talk python). I then embedded this method in my service and ran it using Firefly so I could call into it across the network and I got this curious error: File "/Users/sjohnson/Sync/fuzzygroup/adl/antihate/antihate_experts/venv/lib/python3.7/site-packages/firefly/app.py", line 170, in __call__ result = self.function(**kwargs) File "/Users/sjohnson/Sync/fuzzygroup/adl/antihate/antihate_experts/expert_antisemitic_speech_service.py", line 107, in opinion new_opinion["raw"] = json_results TypeError: 'function' object does not support item assignment but new_opinion isn't a function - it is a dict - so what the heck is going on??? Note: If you're an experienced Python wrangler, you already got it - I know, I know - I'm slow as hell at times particularly when it comes to syntax changes. In ruby you can call methods with or without parens and the issue here is I had this syntax: opinion_template = make_opinion_template instead of: opinion_template = make_opinion_template() I simply forgot to add the parens when I called make_opinion_template. Sigh. The way I ultimately found this was to use a pdb.set_trace() call and evaluate opinion_template in the debugger. > /Users/sjohnson/Sync/fuzzygroup/adl/antihate/ ## How We Solved a Storage Problem in PostgreSQL Without Adding a Single Byte of Storage DevFeed: [How We Solved a Storage Problem in PostgreSQL Without Adding a Single Byte of Storage]() Original publisher: [Read original article]() Author: Haki Benita Published: 2018-12-21T22:00:00Z Content type: tutorial Language: en Sources: [Haki Benita]() Topics: [PostgreSQL](), [etl](), [Database](), [hash](), [Sorting]() Tags: [articles](), [dataset](), [disk-space](), [dump](), [etl](), [hash](), [performance](), [postgresql](), [query](), [sorting](), [sql](), [storage]() ### AI overview This article explains how an ETL task for removing duplicate binary records in PostgreSQL caused increasing temporary disk-space usage. The author reduced the sorting key by hashing the binary value with md5, lowering additional disk usage from about 30 MB to about 4 MB without adding storage. ### Source excerpt A while back we started getting alerts in the middle of the night on low disk space. A quick investigation led us to one of our ETL tasks. Every night the task was fired to eliminate duplicate dumps, and free up some space. This is a short story about how we found our silver bullet and solved the issue without adding a single byte of storage. ## Building a Matching Assistant for Couple Avatars with pHash and Elasticsearch DevFeed: [Building a Matching Assistant for Couple Avatars with pHash and Elasticsearch]() Original publisher: [Read original article]() Author: Joway Published: 2018-09-27T00:00:00Z Content type: tutorial Language: zh Sources: [Random Thoughts]() Topics: [hash](), [elasticsearch]() Tags: [elasticsearch](), [hash](), [tech]() ### AI overview The article describes building a couple-avatar matching assistant. It uses pHash to create image fingerprints that remain relatively similar after resizing or small edits, compares fingerprints by Hamming distance, and stores and searches them with Elasticsearch and an Elasticsearch Hamming plugin. The author also discusses collecting avatar pairs from online communities and presenting the tool as a community bot. ### Source excerpt 背景 情头(情侣头像)一般指成双成对的头像，可以是真人照片，也可以是卡通人物等图片。衍生出去还有闺密头像和基友头像等等。 社交媒体上有一个非常令人费解的现象是，如果你去即刻、豆瓣、百度贴吧、微博，会发现有大量的人在上面贴了一个头像，然后寻找和它匹配的另一半头像，例如这样: 市面上的情侣头像大多是一些社区里的大佬自己制作出来的，然后发到社区里，再慢慢流传开来。这种传播方式导致了很多情头在传播时候早就被拆散了。很多人可能只找到了其中一个，但是想要找到和它配对的另一半。 还有一个问题是，当你看到一个头像时，没有人能够确定这个头像是否在制作时候就有另外一半。 一些程序员朋友最早看到这个问题时，总会天真地想去搜索引擎里识图一下就行了。但这里有两个非常有趣的问题: 第一个是目前搜索引擎的识别图片能力其实并不强，比如以 All in AI 著称的百度: 这还是在图片是原图的情况下，经常一些小朋友会在情头上自己二次创作，比如裁剪，比如压缩，比如贴上什么爱心。那样基本上识图就废了。 第二个问题更加有趣，情头的特点就是大家都在用，所以在最理想的情况下，即便搜索引擎能够识别出所有有这个头像的网站，出来的结果也并没有什么用处，无非是找到了也在用这个头像的别的网站的用户。 只有一种情况是真正能够帮助到寻找情侣头像这件事情的，那就是搜索引擎出来的结果里是专门搜集匹配好的情头的站点。那样姑且用户还能点进网页里去找到另外一半。 从上述阐述里不难发现，指望一个未成年小女生使用一系列高级互联网骚操作找一个头像是有多么不现实，何况技术上可行性还很低。 [Next page]()