Tuesday Tip: Converting Windows to Unix format

[Melinda Skelton]

The question was:
A developer wrote code on his Windows box and checked it into Harvest. He is compiling the code under Unix and the builds are failing because there is a hard return (ctrl-M) at the end of each line. A short term (and very labor intensive) solution is to check out all the code in Unix, remove the (ctrl-M) and check it back in. Is there a less painful way to get this done?

While there is no way to make this change to the files without first checking them out, there are a couple of ways to go to solve the problem.

You could convert the file to Unix format before checking it in on the Windows side.
- There are certain editors in Windows, such as Notepad++ that will actually let you specify whether you want the file saved as a Windows file or a Unix file, and will allow you to convert back and forth between the two formats.
- There are free utilities out there that will perform the conversion such as dos2unix. Here’s a link:
http://sourceforge.net/projects/dos2unix/

The other option would be to convert the files after checking them out on Unix and before executing the build. You could set up a script to run as a post link process to make the conversion on all the files that were checked out. There are several ways to accomplish this in Unix:
- dos2unix and unix2dos
The utilities dos2unix and unix2dos are available for converting files from the Unix command line.
To convert a Windows file to a Unix file, enter:
dos2unix winfile.txt unixfile.txt
To convert a Unix file to Windows, enter:
unix2dos unixfile.txt winfile.txt
- tr
You can use tr to remove all carriage returns and Ctrl-z ( ^Z ) characters from a Windows file:
tr -d '\15\32' < winfile.txt > unixfile.txt
However, you cannot use tr to convert a document from Unix format to Windows.
- awk
To use awk to convert a Windows file to Unix, enter:
awk '{ sub("\r$", ""); print }' winfile.txt > unixfile.txt
To convert a Unix file to Windows, enter:
awk 'sub("$", "\r")' unixfile.txt > winfile.txt
Older versions of awk do not include the sub function. In such cases, use the same command, but replace awk with gawk or nawk.
- Perl
To convert a Windows text file to a Unix text file using Perl, enter:
perl -p -e 's/\r$//' < winfile.txt > unixfile.txt
To convert from a Unix text file to a Windows text file, enter:
perl -p -e 's/\n/\r\n/' < unixfile.txt > winfile.txt
You must use single quotation marks in either command line. This prevents your shell from trying to evaluate anything inside.

[Anon Anon]

You can also flag the item(s) as ASCII within harvest to prevent the checkout with the CRLF. Harvest does a wonderful job of automatically converting line feeds for both window and unix when the item is flagged as ASCII instead of binary. The setting is created and stored at the item level so, version level updates will still cause issue for either win or unix depending on the last format checked-in.

To prevent the problem for new files, check the "File Extensions" tab for the repository within the admin interface. If the repository was set to "treat all as binary" then all the JAVA files would for example fail to compile. In that case just change to "Use Global text file extensions" for the repository.

However that only fixes new files. To fix existing files, you have to use hchgtype.exe or the context menu when selecting an item -> Change Storage Type -> Convert to Text. Once the existing item(s) are flagged as ASCII you can redo the checkout and the file will be checked-out with correct line feeds no matter which platform, windows or unix, is the target.