Do you have to know English to be a Programmer?

This one comes from a large chain of events, and since I like continuity, I am posting about it. This question, “Do you have to know English to be a Programmer?” was a subject on a blog post, http://hype-free.blogspot.com/2008/10/should-we-use-english.html. It was sppoted by me on Scott Hanselman’s blog. It is a big question, and a very important one at that.

As you should know (just by looking at my right sidebar), I am a Brazilian, which means my language is Portuguese (which should be called “brasilianese” or something), and I began learning English when I was only six years old. I have been interacting with the language ever since.

Therefore, my knowledge of English is prior to my knowledge of programming, which makes my opinion less relevant in the matter somehow. However, why did I learn English in the first place? I think it was about the same reason that would lead a programmer to learn it – I loved to play video games back in my childhood, and those only came in English. Programmers, I believe, also learned because of a similar fact, software usually come in English. There are exceptions – mainly for those who live in more developed countries or regions, like Europe. There are better efforts at software localization nowadays, and that means a lesser necessity in dealing with non-native language.

Nevertheless, I must come back to the phrase – “Do you have to know English to be a Programmer?” I think that such question is very gray, since we have not stated what is “know” and what is “Programmer”. If anyone who writes a hello world or a simple script is to be considered a programmer, then, in my opinion, the answer is no, you do not have to know English for that. But if I rephrase the question:

Do you have to be able to read some English to be a good programmer?

The answer is a very big YES! Why:

·         Main technologies researches that happen worldwide are conducted in English. Researches make this choice, because their product needs to have a reach as large as possible. Sure, you could wait for someone to translate a given book or article, but good programmers should know about newer technologies as soon as possible, to be ahead of competition.

·         Translations are, often, very poor (all your bases are belong to us) and small variances can lead to big mistakes.

·         As an engineer, I love standardization, and I’d choose English as the one. Some could argue about the fact that Spanish is one of the most widespread languages in the world, but I guess they’re missing the fact that many Spanish programmers already know English, while the inverse isn’t true.

·         Most computer languages are in English (actually, I don’t know of any in another language). 

Communication is very import in this age, and we need a language that enables a standard effective one. On computer-side, this one seems to be XML (which, by-the-way, is in English). The human counterpart is English.

However, I do not mean to say that there are not good programmers who cannot understand English. Every rule has exceptions, and this one is no exception (unless there is a exception… wait…).

What do you think?

Excel Tips & Tricks I - Removing Duplicates

And one more article is blooming now. For a time I've been thinking about this specific article: how to do some [basic] stuff in Excel. (I call it 'basic' because you probably will use it very constantly.) Stuff that is actually helpful, and simple to do -- not like the previous article that was a bit more complex than first intended -- but actually so simple that you need not be an Excel expert to do it.

First of all, why I should be talking about Excel? I'm specialized in software engeneering, so maybe I should keeping talking about scripting or maybe even programming. But the truth is that Excel is the most useful tool in my computer, way above any IDE that I have here or scripting tool or whatever.

Why? Excel is capable of scripting, for one thing. Second, there's no more intuitive way of organizing data for a human beam than using a bidime

nsional table. Finally, Excel can help us format this data in a very professional (which means "pretty") way, automatically.

I. Removing Duplicates

That said, one of the bests uses for excel is to gath

er data... maybe records from a report, maybe names from a list, whatever. This data often contains duplicates that need to be removed so that the analisys can be made. There are many different ways to remove those duplicates -- and in Excel 2007 Microsoft has finally included a functionality that does this for us.

But since most of us still use Excel 2003 or less, it's very important to know how to best do this. So, here's a very simple tutorial:

1. Define the Key Column

The first step is to define which column(s) will be used in the duplicated comparison. Let's ilustrate: you have a employee list. On column A is the employee name, on B his last name, on C his carrer level name and on D his montly wage. You know for a fact that someone screwed up the report and there are duplicates records -- the same employee appears more than once on the list. It's your purpose to create SERIAL NUMBER

S for your employees, but first you will need to clean-up the list.

However, you also expect to have duplicates first names in your organization, after all, we are talking about a lot of people. So to determine that th

e record is a duplicate of a previous one? You will have to join the Last Name and the First Name to create a third column. This column will be called your KEY column, and it's where we check for duplicates.

Example Sheet

In our example, the "KEY" column is the concatenation of both NAME and SERIAL NUMBER. We create this in the E column. To concatenate both values, you can either use the CONCATENATE function or just the join operator (&); I preffer the latter, so your cells should have the following formula: 

=B2&A2

...which is self-explanatory. This column should not have duplicates -- and this is how you ensure that:

• Sort your KEY column (E) in Ascending Order. 
• Create another column, labelled DUPLICATED?, which should compare if a given item from column E is equal to the previous one, using the formula:
  
  =E2=E1
  
  Resulting in true for whatever lines are duplicates from the first one.
  
• No you can filter you sheet to select only duplicated lines:

Before filtering...

...and after filtering.

• Now, all you have to do is exclude the duplicated lines. To do this, select the first line (in the example, click on "Henry"), then, holding both CTRL and SHIFT, press "down" once, which should select all cells in A. Now, hold SHIFT only and press "space". This will select the whole line (even empty cells). Right-click the selecion and choose "Delete Row". 
• Clear the filter and the duplicated will have disappeared.

That's all for this tip. It's quite easy to do this when after you did it two or three times, and it's handy. Users of Excel 2007 might even preffer to remove duplicates this way since you will see exactly what you are removing.

Next week I will post another tip, this time on coloring odd lines in a document.

Microsoft Always Think We Default To "Stupid"

While I write my next real blog post, a minor rant (and something to remember when you are developing an application):

Microsoft newer products, since the complete widespreading of Internet, which would begin by the end of the last century, are PARANOID. First of all, they assume the user is a complety newbie that will do everything wrong. Secondly, they assume everything outside (and, sometimes, inside) the computer that can be harmful will be. In other words, MS has closed all doors to your computer that they could imagine, so that no hacker could exploit them. 

Of course, all this paranoia accomplished nothing. MS browsers, OSes and Applications are still vunerable to exploits. But it also made the life of the user (us!) much more difficult -- we have to click a lot of pop-ups to let our applications make anything deemed harmful, like downloading a file from the internet or modifying the system register. But today, I've come to the best example of how this untrusting paranoia can be paradoxal:

I use MS Project server on my projects and today I decided to try to integrate it with my Outlook 2007. So I navigate to the Project portal and browse to the web page where I can download the plugin. Naturally, this is a ActiveX plugin that is unsafe accordingly to Microsoft, so when I click the button, I get this message box:

Now, MS asks me to disable all that security paranoia, and just be careful. See the deadlock they've created?

How To Count Lines Of Text (LOC) in Documents

Suppose you have a (large) number of files and you want to know how many lines of text each contains. In computer programming that number is know as Lines-Of-Code or LOC and is a measurement of code complexity. Or maybe you are a writer or editor and separates chapters of a book in different files. Or maybe you have this data sample that came in plain text files and you need to know how many entries (records) each sample has.

There are, of course, many ways to do it. The most straightforward one is to simply open each file on your preferred text-editor, be it notepad, notepad++, UltraEdit or whatever and then use CTRL-END to go to the end of the text, then simply look at the current line number.

This option is ok for a small set of files, say, ten or fifteen. But if you have a larger set, fifty or more, it probably will take too much time to do it. So, let's look at alternatives:

I. Using a DOS Batch

This option is simple and very quick to build. Experienced VBScript writers will prefer the next one since it's more flexible, but common users should default to this. They are very similar and have about the same basic result.

The Basics: The DOS Script is a scripting format for DOS (which now is called the Windows Command Prompt). Old timers will remember dealing with .bat files in the old DOS days, usually to pass parameters during program call. But, in fact, DOS bat files could do much more even back then, and now they are a very handy tool to any windows user that doesn't want to deal with the intrinsics of VBS programming.

Restrictions: Can only be applied to non-formatted files.
Downsides: It can only count the lines. VBScript solution might apply filters, etc.
Recommended: If all you need is the total number of lines for each file.
How to:

A DOS bat is a text file with the .bat extension (for instance, example.bat) that contains DOS native commands. The bat is purely interpreted -- it's as if each command is being typed separately by the user on the command line shell. To use it, open a text file on your favorite text editor (notepad, UltraEdit, notepad++, etc) and save it with the .bat extension (countLOC.bat). It's easier saving the file on c:\ so that we can access it easily from the command prompt. The first example will only count the lines of a single text file:

countLOC.bat (1)

1. @echo off
2. REM The following line sets delayed expansion, which is used to
   
3. REM make sure variables have dynamic value.
4. 
   
5. SETLOCAL ENABLEDELAYEDEXPANSION
6. 
   
7. REM The following FOR loop reads the file line by line
8. FOR /F "tokens=*" %%j in (data\text.txt) do (
9. set /a numLines=!numLines!+1
10. )
11. echo data\text.txt !numLines!

To ran the script above, you will need to open the DOS command prompt (now simply called command prompt), and type the name of the script:

1. Open the DOS prompt (Start → Run, "cmd").
   
2. First switch to the disk partition where the files are by typing the letter of the partition followed by a colon, like "c:", then enter.
3. If your shell tells you you're on a different folder, just type "cd \", which will bring you to the root.
   
4. Run the program by typing its name on the prompt:
   
   c:\>countLOC

Now, let's examine the code. The first line disables echo, which means that commands itselves won't be displayed on the screen, only their output. Remove this line if you want to debug the bat, but keep it afterwards so that display is kept clean.

The lines 2 and 3 begin with REM, which is short for REMARK. Use it as a commentary mark (anything following REM is ignored). SETLOCAL in line 5 does what the commentary section explains: on DOS bat, if you declare a variable (which is done through a SET command), the variable is static. It means that it cannot be modified during execution of commands such as the FOR loop.

Finally, it's necessary to understand the FOR command. FOR is a very handy tool that enables script writers to do repeating tasks a certain number of times. It's help is very comprehensive (type c:\>for /?) if you want a description of its uses. The most interesting format is /f, which uses files (or text, actually) as input variables. The command will scan the file, "data\text.txt", line-by-line. Usually, this would be used in composition with another command, to extract some information from the file. In our case, though, we use the code on line 9 that will increment the numLines variable when executed.

The /a option in SET tells the command to interpret its input as a aritmethic expression, so line 9 adds one to numLines. If you ran the above script, you will see a output like this:

C:\>countLOC.bat
data\text.txt 9

The next goal is to do this with all the files inside a certain folder. Take a look at the next version:

countLOC.bat (2)

1. @echo off
2. 
   
3. REM The following line sets delayed expansion, which is used to make sure variables
4. REM have real dynamic value.
5. 
   
6. SETLOCAL ENABLEDELAYEDEXPANSION
7. 
   
8. REM Get DIR from input
9. SET DIR=%1
   
10. 
    
11. for /f "tokens=*" %%i in ('dir /b /a-d %DIR%') do (
12. for /f "tokens=*" %%j in (%DIR%\%%i) do (
13. set /a numLines=!numLines!+1
14. )
15. echo %%i !numLines!
16. )

I've highlighted the differences. As you can see, we've nested the FOR loop in another FOR loop. What I'm going to do here is to repeat our previous loop for each file in a given folder. I use as an argument to FOR the command dir /b %DIR%, which pass the output from dir command as a text file to FOR, so each file in the folder is read. The /b switch tells DIR to run in bare mode, which just output files name, without header or trailer and the /a-d tells it to not display directories.

Finally, I've "parametrized" the folder name. DOS bath can receive arguments from the command line through %1, %2, etc. So, the user must pass the folder name as an argument to the DOS bat the folder name, which is c:\data in our example:

C:\>countLOC data
asd.txt 7
cassi.txt 10
DBD_All.txt 8926
SourceSafe.txt 8927

The script is ready. All you have to do is copy the solution from countLOC.bat (2) and save it with this name. In case you want to learn more about DOS scripting, point your browser to Rob's van der Woude Scripting Page.

II. Use a VBScript

Using a scripted language to do it is probably the best way to go, since you can customize this solution to whatever you want. This solution is based on a article by Hey Scripting Guy, from MSDN. I've used it more than once, to count LOC from source files and records from data files.

The basics: You will need to write a very simple VBScript to do it. VBScript isn't the best scripting language out there, sure, but has a great advantage -- any Windows installation comes with it. To 'program', just open your prefferred text editor (we love text editors, don't we?) and sabe the file as something.vbs (as usual, "something" is anything you want). Then double click this file in Windows Explorer -- it will be compiled and ran.

"Compiled?", you ask. Yes, VBScript is, oddly enough, a compiled language. This means that your code is first fed into a compiler, generating a 'binary' of some sort (not necessarily native machine code) before it runs.

Restrictions: Can only be applied to non-formatted files.
Downsides: VBScript programming is a bit complex for newbies.
Recommend: If you want to apply some filters on the file, like verifying if a line is a comment, etc.

How to:

So, let's write this program. First you must understand that, to access files and other Operational System functions, VBScript will use a "component" from windows. It doesn't really matter what that object is, but if you are interested, just google it. To create this object, write this on the file:

countLOC.vbs (1):

Set fso = CreateObject("Scripting.FileSystemObject")

This will create a object named fso (short for file system object) which we'll use to access files and stuff. Let's say now that you want to open a file named "text.txt", which resides in "c:\data". All you have to do is:

countLOC.vbs (2):

Set fso = CreateObject("Scripting.FileSystemObject")
Set objTextFile = fso.OpenTextFile("c:\data\text.txt", 1)

This will give you access to the objTextFile, which you will use to manipulate the text file itself. Looking at the arguments (the stuff enclosed in parenthesis), you will notice a '1' being passed. This tells fso that the file is to be openend as read-only, preventing the script from messing up with your text. Now we need to count the lines from text.txt. We do that by reading until the end of the file and then checking how many lines were read:

countLOC.vbs (3):

Const ForReading = 1

Set fso = CreateObject("Scripting.FileSystemObject")
Set objTextFile = fso.OpenTextFile("c:\data\text.txt", ForReading)

objTextFile.ReadAll
Wscript.Echo(objTextFile.Line)

On the code above, we made two modifications. First, the number that tells fso that the file is for reading only was replaced by a constant, so that our code becomes more readable. Second, we read the whole file using the method ReadAll and then printed, through the use of Wscript.Echo, the number of the current line of the file. Since we read the whole file, the line printed will be the last one. The code above is the skeleton of what we will be using to create the report containing all the files number of line. The first difference is that we'll get a list of all the files in the folder and then we will analyzed each one of then. To do so, we will use interaction loops in our script. The second difference is that we will output the result to a file instead of pop-ups. It's easy but, if you are not really interested in learning the hows and whys, skip until the last code listing.

The way to retrieve the file listing from the folder is to use the method fso.GetFolder("folderPath"), which returns a folder object, and the use this object to retrieve the listing. We iterate through the listing by using For Each loop:

countLOC.vbs (4)

1. On Error Resume Next
2. Const ForReading = 1
3. Set fso = CreateObject("Scripting.FileSystemObject")
4. 
   
5. ' Open an output file to write results. Existing files will be overwritten.
6. Set reportFile = fso.CreateTextFile("c:\FileList.txt", True)
7. 
   
8. ' Get the file listing for the given folder
9. Set folder = fso.GetFolder("c:\data")
   
10. 
    
11. ' iterate through the the files
12. For Each fileIdx In folder.Files
13. ' Open the file using the name from folderIdx (folder index)
14. Set objTextFile = fso.OpenTextFile("c:\data\" & fileIdx.Name, ForReading)
15. objTextFile.ReadAll
16. 
    
17. ' Output to the reportfile
18. reportFile.WriteLine(fileIdx.Name & ";" & objTextFile.Line)
19. 
    
20. Next
21. ' Close the report file
22. reportFile.Close

As you can see, we made many modifications. Line 1 informs the script that, in case of an error during the For Each loop, it should resume on the next item. Line 6 creates the output file and line 9 creates a folder object that can be used to access the file listing. Line 12 begins the loop, which sets a variable fileIdx to each file in the folder. As you can see in lines 12 and 18, fileIdx.Name returns the name of the file.

This is the gist of it. The script access a folder, "c:\data", iterate through all files in this folder and outputs each file LOC to "c:\FileList.txt". But we want to make this into a real tool, right? So what's wrong? Well, the script need to be edited everytime you want to change the folder that contains the files or the name to the report. So let's parametize those:

countLOC.vbs (4)

1. On Error Resume Next
2. Const ForReading = 1
3. 
   
4. ' Get first parameter as the foldername
5. sFolder = WScript.Arguments.Item(0)
6. ' Second parameter is the report name
7. sReport = WScript.Arguments.Item(1)
8. 
   
9. If sFolder = "" Or sReport = "" Then
10. Wscript.Echo "Invalid Syntax. Expected: countLOC.vbs folderName reportName"
11. Wscript.Quit
12. End If
13. 
    
14. ' Create a file system object that will help us acessing files
15. Set fso = CreateObject("Scripting.FileSystemObject")
16. 
    
17. ' Open an output file to write results. Existing files will be overwritten.
18. Set reportFile = fso.CreateTextFile(sReport, True)
19. ' Get the file listing for the given folder
20. Set folder = fso.GetFolder(sFolder)
21. 
    
22. ' iterate through the the files
23. For Each fileIdx In folder.Files
24. ' Open the file using the name from folderIdx (folder index)
25. Set objTextFile = fso.OpenTextFile(sFolder & "\" & fileIdx.Name, ForReading)
26. objTextFile.ReadAll
27. 
    
28. ' Output to the reportfile
29. reportFile.WriteLine(fileIdx.Name & ";" & objTextFile.Line)
30. 
    
31. Next
32. 
    
33. ' Close the report file
34. reportFile.Close

Lines 5 and 7 load the arguments from the command line. Lines 9 through 12 test to see if the parameters are passed correctly. To use the code above, you will have to open the command prompt (Start → Run, "cmd") and type

c:\>countLOC folder output

Where folder is the path of the folder that contains the files you want to analyze and output is the name of the report.

Now, a seasoned VBScript writer can re-adapt the code above and instead of using a ReadAll, read it line-by-line and check if the line is empty or not, if it's a comment or not, etc.

Presentation 1.0

I must say that I've been stalling beginning this web site for a few months. Keeping a real web site or a blog up-to-date is a lot of work. You HAVE to post weekly or more, or the few visitors you get won't return.

Secondly, you have to work on a good layout. This means knowing a lot of HTML (which incidentally, I do), choosing the right place to host and have at your disposition a number of free hours which none of us, computer geeks, have. That is, unless we decide we can stop playing that addictive game every night -- a thing that probably won't happen.

Third, and most important of all, you have to choose subject(s) carefully. This is difficult. It's easy to stray from what you should be talking about (your blog presentation) and begin a discussion over what type of coffee you like or your favorite supermodel.

It's no supermodel, but is my favorite

So, what can a Blog be about that there aren't a hundred more in the Web? That is a impossible question, mostly because if there's a subject that no one has ever thought of doing a Blog over, it must be rare. So I've decided to improve over some blogs that already exist and hope that mine is Google listed (otherwise, nobody will ever find me out).

And I guess that will only happen after I have fifty posts or more, but still I'm going to try. So, to answer the question, what will this blog be about?

This blog will be a place where I'm going to post SOLUTIONS to everyday problems when using a computer. Those problems should be simple stuff that most people have to do everyday, like search for stuff on files, creating reports etc. Mainly, I'll be dealing with automation, by creating a number of MACROS or using scripts to solve stuff.

The target audience will be the not-so-common computer user: someone who knows what a Word macro is or has heard of Linux shell scripting. However, no prior knowledge over programming, scripting or macros will be necessary.

The blog won't be focused on tweaking or troubleshooting Windows, actual programming (even though some programming might surge as part of the solutions), or high-technology discussion (SOA, .Net or whatever). If it's not useful for a more common user, it won't be here.

But don't worry if you don't find what you want in this blog. I'm deadly sure there will be thousands other web sites with whatever you seek.