# How to get from theyâEUR ™re to they're

DevFeed: [How to get from theyâEUR ™re to they're](<https://devfeed.tech/articles/how-to-get-from-theya-tmre-to-they-re-26270.md>)

Original publisher: [Read original article](<https://www.justinweiss.com/articles/how-to-get-from-theyre-to-theyre/>)

Author: Justin Weiss

Published: 2015-09-22T06:35:48Z

Content type: tutorial

Language: en

Sources: [Justin Weiss](<https://devfeed.tech/sources/justin-weiss.md>)

Topics: [Ruby](<https://devfeed.tech/topics/ruby.md>)

Tags: [encoding](<https://devfeed.tech/tags/encoding.md>), [how-to](<https://devfeed.tech/tags/how-to.md>), [ruby](<https://devfeed.tech/tags/ruby.md>)

## AI overview

A tutorial explains how text becomes garbled when UTF-8 bytes are misread as Windows-1252 and then encoded again. It shows how to reverse those steps in Ruby to restore the original characters.

## Source excerpt

In last week's article, you learned a short process that solves most encoding problems. But there's one encoding problem that's much harder to solve. I know you've seen it. (Or maybe youâEUR ™ve seen it?) It's when a curly quote turns into âEUR ™, or an em-dash turns into âEUR ". It'll make you think you've gone crazy. It should just work! You could create a giant table, so you could find bad characters and replace them with good ones: [{broken: 'âEUR "', fixed: "--"} {broken: "âEUR "", fixed: "-"} {broken: "âEUR ˜", fixed: "'"} {broken: "âEUR ™", fixed: "'"} {broken: "âEUR œ", fixed: """} {broken: "âEUR ", fixed: """}, ...] But there's an easier, more reliable way to fix those broken characters. Why does good typography always break? Last week, you learned that an encoding is just a way to turn groups of meaningless bytes into displayable characters. Not every character can be represented in a single byte, because there are more than 256 possible characters. So some characters, like the curly quote ', are represented with more than one byte: irb(main):001:0> "they're".bytes => [116, 104, 101, 121, 226, 128, 153, 114, 101] Even though the string only has 7 characters, they're represented by 9 bytes! When you focus on just the curly quote: irb(main):002:0> "'".bytes => [226, 128, 153] You'll see it uses 3 bytes. And our messed up string, theyâEUR ™re, has three characters where it should just have one. That seems like more than a coincidence, right? It seems like those three bytes should be read as UTF-8, where they'd represent a curly quote. Instead, each byte is showing up as a different character. So, which encoding would represent [226, 128, 153] as âEUR ™? If you look at a few tables of popular encodings, you'll see it's Windows-1252. You can check this in irb: irb(main):003:0> "they're".force_encoding("Windows-1252").encode("UTF-8") => "theyâEUR ™re" (We need that last .encode("UTF-8") to display the string in the console.) Yep! That's the problem. But it gets worse. The data is supposed to be UT