DELETING FROM ENTRANCE FILES – HelpCenter Braincube

Deleting some files

If the filename contains "PO_Consumption_", it is deleted.

for f in *.csv; do
   if [ $(echo "$f" | grep -c PO_Consumption_) -gt 0 ]; then
      rm "$f" 
   fi
done

If the gz file is corrupted`, it is deleted.

#Be carreful, if files are gz.gz they will be deleted 
for f in *.gz; do
if [ -f "$f" ]; then
gunzip "$f"
if  [ $? -eq 0  ]; then
echo "fichier OK"
else
echo "fichier pas OK"
rm "$f"
fi
fi
done

Deleting lines from a file before a particular word

Here we delete lines before the "Schichttext" word appears (the line with the word will be kept)

for f in *.txt; do
   if  [ $(echo $f | grep -c pm2_sh) -gt 0  ]; then

      if [ $(egrep -a -h "Schichttext" "$f" | wc -l) -gt 0 ]; then
         new_file2=text_"$f"
         cp $f "$new_file2"
         sed -n -i '/Schichttext/,$p' "$new_file2"
         mv $new_file2 "$PathRepo"
      fi

      mv $f "$PathRepo"

   fi
done

Removing unreadable characters from filenames

Because everybody knows that in computer-speak a ç becomes ¤$?!!
```
for f in *.*;do detox "$f";done
```

Removing n lines from the start and m lines from the end of a file

Here we remove 3 lines from the start and 3 lines from the end

if [ $(echo $f | grep -c UT_production_journaliere_) -gt 0 ]; then

   newf=$(echo "$f" | sed "s/.txt/.csv/")
   nbline=$(awk 'END {print NR}' $f)
   deb=$((nbline-3))
   fin=$((-3))
   tail -n $deb $f | head -n $fin >> $newf
   rm $f
fi

Removing single quotes from just one line in a file (the header line)

You can make sed much faster if you only work on a single line:

for f in *.csv; do
   if [ $(echo "$f" | grep -c "HX_Ligne_POUDRE") -gt 0 ]; then
      sed -i "1 s|'||g" "$f"  # The "1" at the beginning  of the pattern makes sure that sed only reads that line
      mv "$f" "$PathRepo"
   fi
done

Removing double quotes in the all file content

for f in *.csv; do 
    if [ $(echo "$f" | grep -c "Monats") -gt 0 ]; then
    sed -i 's|"||g' "$f"
    fi
mv "$f" "$PathRepo";
done

Removing MacOS bloat files

If you've been messing around with files on your Mac before uploading them, MacOS will upload hidden files nobody cares about. This removes them:

# Removing MacOS bloat files
find -path './._*' -delete
rm .DS_Store

Deleting some columns

Here we keep only columns n°1, 2, 4, 5, 6 and 7

for f in *.csv; do
    if [ $(echo $f | egrep -c Edition_Arret_Conditionnement_Ligne_1L_ ) -gt 0 ]; then 
    newf=$(echo "$f""-new.csv")
    awk -F';' '{ OFS = ";" }{print $1,$2,$4,$5,$6,$7}' $f > $newf
    cp $newf $f
    rm $newf
    fi
done