# Ruby's external malloc problem

DevFeed: [Ruby's external malloc problem](<https://devfeed.tech/articles/ruby-s-external-malloc-problem-22429.md>)

Original publisher: [Read original article](<https://samsaffron.com/archive/2018/06/13/ruby-x27-s-external-malloc-problem>)

Author: Sam Saffron

Published: 2018-06-13T04:20:53Z

Content type: article

Language: en

Sources: [Sam Saffron](<https://devfeed.tech/sources/sam-saffron.md>)

Topics: [Ruby](<https://devfeed.tech/topics/ruby.md>), [Process](<https://devfeed.tech/topics/process.md>), [C](<https://devfeed.tech/topics/c.md>)

Tags: [blog](<https://devfeed.tech/tags/blog.md>), [c](<https://devfeed.tech/tags/c.md>), [c-sharp](<https://devfeed.tech/tags/c-sharp.md>), [memory](<https://devfeed.tech/tags/memory.md>), [net](<https://devfeed.tech/tags/net.md>), [performance](<https://devfeed.tech/tags/performance.md>), [process](<https://devfeed.tech/tags/process.md>), [rails](<https://devfeed.tech/tags/rails.md>), [ruby](<https://devfeed.tech/tags/ruby.md>), [saffron](<https://devfeed.tech/tags/saffron.md>), [sam](<https://devfeed.tech/tags/sam.md>), [sql](<https://devfeed.tech/tags/sql.md>), [weblog](<https://devfeed.tech/tags/weblog.md>)

## AI overview

This article explains how Ruby's garbage collector accounts for memory allocated through malloc, including thresholds that trigger minor and major collections. It demonstrates the behavior with large Ruby objects and discusses how C extensions that bypass Ruby's allocation helpers can cause memory accounting problems, including in patterns involving Active Record and SQL results.

## Source excerpt

I have blogged a bit about the Ruby GC previously and covered some basics about malloc triggering GC runs. Over the years much in that blog post has been addressed in Ruby including dynamically growing malloc limits that mean we very rarely would need to amend malloc related GC vars. As an aside, the only GC var Discourse still overrides is RUBY_GLOBAL_METHOD_CACHE_SIZE for reasons that are specified in the Shopify blog post by Scott Francis. The GC in Ruby can be triggered by 2 different types of conditions. We are out of space in our managed heaps. We detected that data associated with Ruby objects via malloc calls has grown beyond a certain threshold. In this blog post I am covering (2) and demonstrating what happens when Ruby is not aware of malloc calls. Why malloc calls can trigger a GC? When reading through GC.stat we may be a bit surprised to see the amount of malloc related accounting: malloc_increase_bytes malloc_increase_bytes_limit oldmalloc_increase_bytes oldmalloc_increase_bytes_limit We keep track of the amount of memory allocated using malloc, if it hits the malloc_increase_bytes_limit we will trigger a minor GC. When we promote an object to the old generation we also try to estimate how much malloc increased since the last major GC. This way when we promote large objects from a young heap to an old heap we have a chance to GC as soon oldmalloc_increase_bytes_limit is hit. The oldmalloc_increase_bytes_limit and malloc_increase_bytes_limit dynamically size themselves growing as we hit GCs due to malloc limits. Seeing this in action Having this in place allows us to run code like this without bloating memory: def count_malloc(desc) start = GC.stat[:malloc_increase_bytes] yield delta = GC.stat[:malloc_increase_bytes] - start puts "#{desc} allocated #{delta} bytes" end def process_rss puts 'RSS is: ' + `ps -o rss -p #{$$}`.chomp.split("\n").last end def malloc_limits s = GC.stat puts "malloc limit #{s[:malloc_increase_bytes_limit]}, old object malloc lim