ENCODING

Converting an iso (Windows) file to utf8 (Linux)

• Here we're working on the FchBraincube_toto.csv files

for f in *.csv; do
   if [ $(echo $f | grep -c FchBraincube_) -gt 0 ] && [ $(echo $f | grep -c toto) -gt 0 ]; then
      i=$(echo $f | sed "s/.csv/.txt/")
      iconv -f iso8859-15 -t utf8 "$f" > "$i"
   fi
done

Changing the encoding

Change the encoding from UTF16 to UTF8

for f in AL*CSV; do
   iconv -f UTF16 -t UTF8 $f > test.txt
   rm test.txt
done

Going from Windows new line characters to Unix new line characters

CR+LF --> LF.
To know the final caracter of each line, you have to open the file in NotePad++ ==> "Affichage"/"Symboles spéciaux"/"Afficher les symboles de fin de ligne" or "View"/"Show symbol"/"Show end of line" (depending if you have NotePad++ in French or English). At the end of the line you should see LF or CR+LF or something else

if [ $(echo "$f" | grep -c HAV_NAME) -gt 0 ]; then
     newfile=$(echo "$f" | sed "s/.csv/.txt/")
     tr -d '\0' < $f > $newfile
     sed -i "s/\r$//" "$newfile"
     mv "$newfile" "$PathRepo"
     rm $f
fi

Going from Unix new line characters to Windows new line characters

LF --> CR+LF.
To know the final caracter of each line, you have to open the file in NotePad++ ==> "Affichage"/"Symboles spéciaux"/"Afficher les symboles de fin de ligne" or "View"/"Show symbol"/"Show end of line" (depending if you have NotePad++ in French or English). At the end of the line you should see LF or CR+LF or something else

if [ $(echo "$f" | grep -c HAV_NAME) -gt 0 ]; then
         newfile=$(echo "$f" | sed "s/.csv/.txt/")
         tr -d '\0' < $f > $newfile
         sed -i "s/$/\r/" "$newfile"
         mv "$newfile" "$PathRepo"
         rm $f
    fi

Going from MacOS new line characters to Windows new line characters

CR --> CR+LF.
To know the final caracter of each line, you have to open the file in NotePad++ ==> "Affichage"/"Symboles spéciaux"/"Afficher les symboles de fin de ligne" or "View"/"Show symbol"/"Show end of line" (depending if you have NotePad++ in French or English). At the end of the line you should see LF or CR+LF or something else

if [ $(echo "$f" | grep -c Vision) -gt 0 ]; then
        newfile=$(echo "$f" | sed "s/.csv/.txt/")
        tr '\r' '\n' < "$f" > "$newfile"      # Replaces CR with LF  
        sed -i "s/$/\r/" "$newfile"           # Replaces LF with CRLF
        rm "$f"
    fi