Answer by aviro for Remove lines from a csv file based on column value
TL;DRRedirecting your gawk's stdout to /dev/null or piping it to cat will greatly accelerate it and reduce the runtime significantly.gawk -i inplace [...] myfile.csv >/dev/nullOr:gawk -i inplace...
View ArticleAnswer by Romeo Ninov for Remove lines from a csv file based on column value
One possible way (via rewriting your command) is:gawk -F, -v s="$start_marker" -v e="$end_marker" '$6 > e || $6 < s' myfile.csv >/tmp/newfileIn awk, it is not recommended practice to use...
View ArticleRemove lines from a csv file based on column value
I have a csv file with 12 million lines in the following format:mcu_i,INIT,200,iFlash, 11593925, 88347,,0x00092684,r,0x4606b570, ok,, 32,single,op-c,0,, 0, 0, 0,mcu_i,INIT,200,iFlash, 11593931,...
View ArticleAnswer by U. Windl for Remove lines from a csv file based on column value
If it does not have to be awk, then you might try Perl instead:#!/usr/bin/perluse 5.18.2;use warnings;use strict;my ($X, $Y) = (88347, 88347);while (<>) { next if (/(?:^[^,]*,){5}\s*([^,]+)/...
View ArticleAnswer by aborruso for Remove lines from a csv file based on column value
if you can use tools other than awk, duckdb is very quick and convenient.If your input ismcu_i,INIT,200,iFlash, 11593925, 88347,,0x00092684,r,0x4606b570, ok,, 32,single,op-c,0,, 0, 0,...
View ArticleAnswer by jubilatious1 for Remove lines from a csv file based on column value
Using Raku (formerly known as Perl_6)~$ raku -MText::CSV -e 'my @rows; my $csv = Text::CSV.new( sep => ","); \ while ($csv.getline($*IN)) -> $row { @rows.push: $row.map(*.trim) if 88000 <...
View Article
