# awk Published articles for awk. This is one page of public article previews, not the complete archive. Follow Next page to continue. Summaries are not the original full articles. ## Find palindrome dates in the Linux command line - explained DevFeed: [Find palindrome dates in the Linux command line - explained]() Original publisher: [Read original article]() Author: Lazarus Lazaridis Published: 2020-02-06T10:00:00Z Content type: tutorial Language: en Sources: [Lazarus Lazaridis]() Topics: [Command-line interface](), [Linux](), [Terminal]() Tags: [awk](), [bash](), [climagic](), [command-line](), [commands](), [how-to](), [linux](), [opensource](), [programming](), [scripts](), [terminal]() ### AI overview A tutorial explains a Linux command-line pipeline for finding palindrome dates. It breaks down brace expansion, date formatting, whitespace removal, reversing and comparing date strings with standard command-line tools. ### Source excerpt Yesterday I saw a great command that finds the palindrome dates between now and x days ago. The command was posted on Twitter by @climagic (a great account to follow if you want to find awesome command line stuff). In this post we are going to break it down and explain how it works. What is a palindrome Palindrome is a word/number/phrase or other sequence of characters that reads the same backward as forward. - Palindrome - Wikipedia Few days ago, the date was 02/02/2020 which was a palindrome: The command printf "now - %d days\n" {1..332044} |date -f- +%Y%_m%_d$'\n'%Y%m%d |tr -d ' ' > alldates.txt; rev alldates.txt >revdates.txt; paste alldates.txt revdates.txt |awk '$1==$2{print $1}' And here's a more readable version using new lines after each pipe. printf "now - %d days\n" {1..332044} | date -f- +%Y%_m%_d$'\n'%Y%m%d | tr -d ' ' > alldates.txt; rev alldates.txt >revdates.txt; paste alldates.txt revdates.txt | awk '$1==$2{print $1}' I am going to explain each part of the command separately. Note that the output of each part is the input of the next part (following the pipe |). printf "now - %d days\n" {1..332044} The first part seems pretty straight forward but it does have an interesting part, the brace expansion mechanism, which I didn't know of. Open a terminal and type the following commands: # From 1 to 10 $ echo {1..10} 1 2 3 4 5 6 7 8 9 10 # From a to z $ echo {a..z} a b c d e f g h i j k l m n o p q r s t u v w x y z # From 10 to 20 $ echo Before-{10..20}-After Before-10-After Before-11-After Before-12-After Before-13-After Before-14-After Before-15-After Before-16-After Before-17-After Before-18-After Before-19-After Before-20-After As you can see, the brace expansion allows the generation of values based on a "sequence" we set. Learn more about the Brace expansion mechanism here. Part explained we print the phrase "now - x days" so many times as the number of values expanded from the sequence {1..332044} will generate 332044 values starting from 1 and in ## How to remove duplicate lines from files preserving their order DevFeed: [How to remove duplicate lines from files preserving their order]() Original publisher: [Read original article]() Author: Lazarus Lazaridis Published: 2019-05-16T08:30:00Z Content type: tutorial Language: en Sources: [Lazarus Lazaridis]() Topics: [Script](), [file]() Tags: [approach](), [awk](), [bash](), [featured](), [file](), [files](), [how-to](), [linux](), [programming](), [scripts]() ### AI overview A tutorial explains how to remove duplicate lines from a text file with awk while preserving their original order. It breaks down the associative array, line-occurrence tracking, negation, post-increment behavior, and why other approaches may not preserve order. ### Source excerpt Suppose you have a text file and you need to remove all of its duplicate lines. TL;DR To remove the duplicate lines preserving their order in the file use: awk '!visited[$0]++' your_file > deduplicated_file How it works The script keeps an associative array with indices equal to the unique lines of the file and values equal to their occurrences. For each line of the file, if the line occurrences are zero then it increases them by one and prints the line, otherwise it just increases the occurrences without printing the line. I was not familiar with awk and I wanted to understand how is this accomplished with such a short script (awkward). I did my research and here is what is going on: the awk "script" !visited[$0]++ is executed for each line of the input file visited[] is a variable of type associative array (a.k.a. Map). We don't have to initialize it, awk will do this for us the first time we access it. the $0 variable holds the contents of the line currently being processed visited[$0] accesses the value stored in the map with key equal to $0 (the line being processed), a.k.a. the occurrences (which we set below) the ! negates the occurrences value: In awk, any nonzero numeric value or any nonempty string value is true By default, variables are initialized to the empty string, which is zero if converted to a number That being said: if visited[$0] returns a number greater than zero, this negation is resolved to false. if visited[$0] returns a number equal to zero or an empty string, this negation is resolved to true. the ++ operation increases the variable's value (visited[$0]) by one. If the value is empty, awk converts it to 0 (number) automatically and then it gets increased. Note: the operation is executed after we access the variable's value. Summing up, the whole expression evaluates to: true if the occurrences are zero/empty string false if the occurrences are greater than zero awk statements consist of a pattern-expression and an associated action.