API's that Suck

January 4, 2010

Learning IL – The Echo Program

Filed under: CIL — Grauenwolf @ 4:27 pm

The requirements for our echo program is pretty simple.

  1. Take any number of command line arguments.
  2. Print each argument on separate lines.
  3. Return the number of command line arguments.

In this lesson you will learn how to define parameters, work with arrays, work with loops, and return values from functions. This lesson assumes that you already understand the hello world code.

Version 1

Version 1 will deal with requirements 1 and 3. This will be enough to prove that the command line parameters are actually making it all the way to the program and the return value is making it back to the shell. In Windows you can use the command “ECHO %ERRORLEVEL%” immediately after running the program to see the return value. For this to work you need to run the program itself from the command line as well.

  1. .assembly extern mscorlib {}
  2. .assembly EchoProg {}
  3. .method static int32  Main(string[] args)
  4. {
  5.     .entrypoint
  6.     ldarg.0
  7.     ldlen
  8.     ret
  9. }

Line 1 is the core assembly reference. Again, I’m not asking for a specific version.

Line 2 is our program’s name. I’m calling it “EchoProg” because “echo” is reserved by the Windows command shell.

Line 3 is our entry point. Lets compare is to the hello world version.

  1. .method static void Main()

As you can see void has been replaced by the return type. In addition, we now have one argument in a C#-like style with the type preceding the argument name. Note that we don’t actually use the argument name anywhere, it is just their for reference.

Line 6 is the “load argument N onto the stack” command. In this case we are loading the “args” parameter, which is a pointer to an array of string.

Line 7 is the “replace the array on the stack with it’s length” command, also known as load length.

As I mentioned last time, the return command in line 8 takes the last value on the stack and places it on the calling stack, then exits the current function.

To make experimenting with this code a little easier, I wrote this little batch file. If you run it using version 1, you should get the number 4.

  1. ilasm EchoProg.il
  2. PEVerify EchoProg.exe
  3. EchoProg.exe "la la la" dee di doh
  4. ECHO %ERRORLEVEL%

Version 2

Before we jump into loops, lets just try printing the first command line argument. The code for this is pretty straight forward.

  1. .assembly extern mscorlib {}
  2. .assembly EchoProg {}
  3. .method static int32  Main(string[] args)
  4. {
  5.     .entrypoint
  6.  
  7.     //printline code
  8.     ldarg.0
  9.     ldc.i4 0
  10.     ldelem.ref
  11.     call       void [mscorlib]System.Console::WriteLine(string)
  12.     
  13.     //return code
  14.     ldarg.0
  15.     ldlen
  16.     ret
  17. }

 

Line 7 is a comment. I believe you can use // at the end of any line, as I often see it in compiler-generated IL.

In line 8 we are loading the array onto the stack just as before.

In line 9 we see the “load a constant” command. The “.i4” suffix means that the constant is a 4-byte integer, a.k.a. a System.Int32. This is followed by the number 0, which we want. This gives us the following stack:

0

pointer to args

[not yours]

Line 10 starts with the “load element from array” command. It appears that the “.ref” suffix is used to indicate the array contains a non-numeric type. After this command is executed both the array and the constant will be removed from the stack and a pointer to the first string will be in its place. This is shown in the spec under the “stack transition” heading.

 

image

Version 3

Ok, now things get tricky. We are going to have to start keeping track of an array index. And since there are no loops in CIL, you have to fake it using if statements and gotos. Before we run off and start coding, lets write some pseudo code showing what we are trying to accomplish.

  1. Variables: index, arrayLength
  2. arrayLength = args.Length
  3. index = 0
  4. If index = arrayLength Then goto done
  5. Print args[index]
  6. index = index+1
  7. goto if
  8. done

Reminds me of BASICA back from the days of DOS, except you get labels instead of line numbers. Anyways, on to the code.

  1. .assembly extern mscorlib {}
  2. .assembly EchoProg {}
  3. .method static int32  Main(string[] args)
  4. {
  5.     .entrypoint
  6.     //Variables: index, arrayLength    
  7.     .locals init (
  8.         [0] int32 index,
  9.         [1] int32 arrayLength)
  10.  
  11.     //arrayLength = args.Length
  12.     ldarg.0
  13.     ldlen
  14.     stloc 1
  15.     
  16.     //index = 0
  17.     ldc.i4 0
  18.     stloc 0
  19.  
  20.     //If index = arrayLength Then goto done
  21. if: ldloc 0
  22.     ldloc 1
  23.     beq done
  24.  
  25.     //Print args[index]
  26.     ldarg.0
  27.     ldloc 0
  28.     ldelem.ref
  29.     call       void [mscorlib]System.Console::WriteLine(string)
  30.     
  31.     //index = index+1
  32.     ldloc 0
  33.     ldc.i4 1
  34.     add.ovf
  35.     stloc 0
  36.  
  37.     br if
  38.  
  39. done:
  40.  
  41.     //return code
  42.     ldarg.0
  43.     ldlen
  44.     ret
  45. }

On line 6 thru 9 we see our variables being defined. It starts with the “.locals” directive. This is followed by “init”, which sets all the local variables to 0/null. According to the spec, all verifiable code must use the init keyword.

Each well formed local variable consists of three values. First is the slot index, next is the variable type, and finally there is the variable name. Of these only the type is actually required. You could write, “.locals init (int32, int32)”, but that is going to be a royal pain to understand later on.

Lines 11 thru 14 load the array length into a local variable. The only thing new here is the “store local” command, which takes a slot number as a parameter.

Lines 16 thru 18 load the value 0 into the index variable. Strictly speaking this isn’t necessary because we are using init, but I’ll leave it here for reference.

Lines 20 to 23 represent our if statement. It starts with the label “if:” so we can jump to it later. We could have used just about anything for our label, it doesn’t really matter once the code is assembled.

Then we use the “load local” command twice, once for each variable.

Finally we use the “branch if equal” command on line 23. If the top-two items on the stack are equal, we go to line labeled “done”.

Lines 25 thru 29 is our code to actually print the parameter. The only difference between this and the one in version 2 is that we are loading the index using ldloc instead of a constant value.

Lines 31 to 35 will increment our index variable. It does this by loading both the index and the constant 1 onto the stack. Then it calls the “add with overflow check” command. Finally it stores the result value back into the index variable.

Line 37 takes us back to the top of the loop using the “branch” command. It is important that this line is skipped if our if statement back on line 20 is true.

Conclusions

I was surprised how much code it took to write this. Sure I knew I would have to write my own loops, but even basic stuff like incrementing local variables took much harder than I expected. I was expecting to find a “increment local” command, but it looks like we can’t do anything with values until they are on the stack.

Advertisements

Learning IL – Hello World in CIL

Filed under: CIL — Grauenwolf @ 6:38 am

If you are going to be writing your own programming language you need to spend a lot of time learning the target. In my case, the target is .NET’s Common Intermediate Language, also known as IL or CIL.  As per tradition, here is my hello world app written in CIL.

  1. .assembly extern mscorlib {}
  2. .assembly Test1 {}
  3. .method static void Main()
  4. {
  5.     .entrypoint
  6.     ldstr      “Hello World”
  7.     call       void [mscorlib]System.Console::WriteLine(string)
  8.     ret
  9. }

Line 1 says that I need mscorlib, but I don’t care which version. If it was important, then I would include some extra information in the brackets.

Line 2 is the name of my assembly.  I think I’m supposed to put assembly-level attributes in the brackets, but I’m not certain.

Line 3 starts my first function, or in IL parlance a “.method”. I bet you were expecting to see some sort of class or module, I sure as heck was. But no, that is not required.  The ECMA documentation clearly says, “Methods can be defined at the global level (outside of any type)”.  Well now, if that’s the case why not just go ahead and try it out.

Warning: Languages like VB and C# cannot call free-floating functions.

Ok, so what else do we know about line 3. Well it is an entry point function, so either it returns an integer or nothing. I choose nothing hence the “void” keyword. As for “static”, it seems kinda silly. Turns out the assembler will infer that it is a static method form the context, but since we don’t want warnings we will go ahead and put that in there.

Line 5 says that this is the entry point for the executable. For executables you must have one and only one entry point, the assembler won’t even try to sort it out if you break that rule.

Line 6 is translated as “load a string and push it onto the stack”. What follows is a QSTRING. From the spec:

QSTRING is a string surrounded by double quote (″) marks. Within the quoted string the character “\” can be
used as an escape character, with “\t” representing a tab character, “\n” representing a newline character, and
“\” followed by three octal digits representing a byte with that value. The “+” operator can be used to
concatenate string literals. This way, a long string can be broken across multiple lines by using “+” and a new
string on each line. An alternative is to use “\” as the last character in a line, in which case, that character and
the line break following it are not entered into the generated string. Any white space characters (space, line-
feed, carriage-return, and tab) between the “\” and the first non-white space character on the next line are
ignored. [Note: To include a double quote character in a QSTRING, use an octal escape sequence. end note]

You don’t have to use a QSTRING. You could instead use a SQSTRING. I don’t know why you would though, because literally the only difference is that it is wrapped in single quotes instead of double quotes.

Line 7 is a method call. (Not a function call; this is CIL and everything is a method.) Note that we have to be very explicit about what assembly the method lives in, what type wraps it, and which overload we desire.

Lets take a moment to talk about the stack. When you want to call a method you need to push the arguments onto the stack. When the method returns, if it has a non-void return type, the result will be pushed onto the stack. This means at run time your stacks are going to look something like this:

Before calling WriteLine After calling WriteLine
   
Pointer to “Hello World”  
[not yours] [not yours]

If you stray into the “not yours” part of the stack, say by calling WriteLine twice in a row, then bad things will happen. The assembler doesn’t care about double-checking the stack, so that won’t save you. There is a program called PEVerify that will warn you about potential stack underflows. If you don’t use it, then you find your error when your program crashes hard with an InvalidProgramException.

Line 8 is the return statement. If the function has a return value, it is copied from the top of the current stack to the top of the calling function’s stack. Since we aren’t returning anything, the stacks are not affected.

January 2, 2010

Foundry – Types of Variables and Variable Types

Filed under: Foundry — Grauenwolf @ 9:30 pm

It seems I’ve gotten a bit ahead of myself. Why talk about passing variables to functions before you figure out how to define a variable.

One of the things that really sucks about .NET is that variables are nullable and mutable by default. While I have never seen any convincing evidence for getting rid of either, there is plenty of reasons to limit them as much as possible. So whatever is decided needs to tackle both aspects.

Between VB and C#, I much prefer VB’s syntax. It is easier to parse, with its leading keyword, and looks the same whether or not you are using type inference. But it still has flaws such as the totally unnecessary “As” keyword. Using “dim” versus “const” sounds good, so lets start there.

Define

Define variables are variables that can only be set once and not changed from there on out. Foundry doesn’t expect programmers distinguish between statically compiled, CLR-style constants and variables that are simply assign-once, that’s what compilers are for. Likewise, type is inferred from the expression.

<Define-Statement> := “Define” <Identifier> “=” <Expression>

Declare

Declare variables are variables whose value can be changed. They require a type, which can be made nullable using the ‘?’ operator. An initial value is required if the type is left off or the variable is not nullable.

<Declare-Statement> := “Declare” <Identifier> [<Type> [ “?” ]]  “=” <Expression>
<Declare-Statement> := “Declare” <Identifier> <Type> [ “=” <Expression> ]

At some point that should be some escape analysis to make it possible to leave of the expression for non-nullable variables, but it will have to wait.

String Literals

There is another post on this one.

Date/Time Literals

Unless someone else has a reason otherwise, I’m going to adopt VB’s style of date literals for both dates and times. T-SQL just uses the string literal syntax, but that means you don’t get type inference.

<Time-Literal> := #hh:MM:ss.fffff#
<Date-Literal> := #yyyy-mm-dd#
<DateTime-Literal> := #yyyy-mm-ddThh:MM:ss.fffff#
<DateTimeOffset-Literal> := #yyyy-mm-ddThh:MM:ss.fffffZ# | #yyyy-mm-ddThh:MM:ss.fffff [+/-] hh:mm#

Numeric Literals

If the number contains a decimal place, then it is treated as Decimal. Integers default to Int32. If they don’t fit, they will be automatically treated as an Int64. If they don’t fit within that either, a Decimal is used.

Foundry: What kind of string literals should be used by default?

Filed under: Foundry — Grauenwolf @ 9:21 pm

There are a lot of options for string literals, but which to choose as the default? I’m inclined to choose verbatim strings because I’m primarily a VB and T-SQL programmer, but the other forms do have merits.

Verbatim Strings

The easiest is the one used by CSV, VB, and SQL, where in there is no escape sequences except for quotes. C# uses this as a secondary form, accessed using @" instead of the usual ".

Escaped Strings

Escaped strings are what languages like Java and C# use by default.  They use the backslash to start a escape sequence. This makes strings that actually use backslashes like file paths annoying.

Interpolated Strings

String interpolation, where in variables and expressions can be inlined right into the string literal. Under the covers this would be implemented using the String.Format function call. This could be combined with either of the two above formats.

Why not all of the above?

C# already offers two of the three by tacking an extra symbol to the front of a string to indicate it should be handled differently. But which to choose as the default?

Foundry – Unattached Functions

Filed under: Foundry — Grauenwolf @ 1:19 am

An unattached functions is a function that isn’t tied to any specific class or module. It just floats around in the global namespace. In addition to a executable’s entry point, this will be used primarily for scripting.

Action

An action is a function that doesn’t have a return value.  It has both a inline and a block form. The block terminator is very forgiving, allowing one to choose between braces or the “end” keyword. If you choose to use end, then you can optionally specify either the word “Action” or the name of the function.

<Action-Block> := <Action-Header> [ “{“ ] <EOL> <Imperative-Block> <Action-Footer>
<Action-Header> := “Action” <Identifier> <Params>
<Action-Footer> := “End” | “End Action” | End <Identifier> | “}”

The inline variant of this is detonated by  either the keyword “Then” or a colon. It allows for a single statement followed by and end-of-line token.

<Action-Inline> := <Action-Header> <Inline-Marker> <Statement> <EOL>
<Inline-Marker> := “Then” | “:”

Func

A Func is a function that has a return value.

<Func-Block> := <Func-Header> <Type> [ “{“ ] <EOL> <Imperative-Block> <Func-Footer>
<Func-Header> := “Func” <Identifier> <Params>
<Func-Footer> := “End” | “End Func” | End <Identifier> | “}”

The inline version of the Func doesn’t require a type. If not provided, it is inferred from the expression that follows.

<Func-Inline> := <Func-Header> [<Type>] <Inline-Marker> <Expression> <EOL>

Main

Main is a function that serves as the entry point for an application. By default it has an implied parameter called “args String[]” and an implied return type Int32.

<Main-Block> := “Main” [ “{“ ] <EOL> <Imperative-Block> <Main-Footer>
<Main-Footer> := “End” | “End Main” | “}”

Parameters

Obviously parameters are going to be encapsulated by parens. It is a tried and true method of handling such matters, though what goes inside the parens can vary greatly from language to language. More on this another day, its late and my brain is fuzzy.

January 1, 2010

Foundry – Block Terminators II

Filed under: Foundry — Grauenwolf @ 8:03 pm

Ok, so here are our candidates from round 1.

Candidate A

Candidate B

Candidate C

Candidate D

  1. Program Test1 {
  2.     References {
  3.         mscorlib
  4.     }
  5. }
  6.  
  7. Main {
  8.     Print "Hello World"
  9. }

  1. Program Test1
  2.     References
  3.         mscorlib
  4.     }
  5. }
  6.  
  7. Main
  8.     Print "Hello World"
  9. }

  1. Program Test1
  2.     References
  3.         mscorlib
  4.     End References
  5. End Program
  6.  
  7. Main
  8.     Print "Hello World"
  9. End Main

  1. Program Test1
  2.     References
  3.         mscorlib
  4.     End
  5. End
  6.  
  7. Main
  8.     Print "Hello World"
  9. End

Candidate A

A is better than B because…

  • There is a clear syntax for arbitrary blocks.
  • It doesn’t have unmatched braces.

A is better than C because…

  • It takes less characters to express the same concept.
  • There is a clear syntax for arbitrary blocks.

A is better than D because…

  • It takes less characters to express the same concept.
  • There is a clear syntax for arbitrary blocks.

Candidate B

B is better than A because…

  • It takes less characters to express the same concept.
  • It takes less tokens to express the same concept.

B is better than C because…

  • It takes less characters to express the same concept.

B is better than D because…

  • It takes less characters to express the same concept.
  • It takes less tokens to express the same concept.

Candidate C

C is better than A because…

  • Easier for users to know which block is being closed.
  • Easier for the compiler to give meaningful feedback.
  • Keywords are easier to spot than semi-colons.

C is better than B because…

  • Easier for users to know which block is being closed.
  • Easier for the compiler to give meaningful feedback.
  • Keywords are easier to spot than semi-colons.
  • It doesn’t have unmatched braces.

C is better than D because…

  • Easier for users to know which block is being closed.
  • Easier for the compiler to give meaningful feedback.

Candidate D

D is better than A because…

  • It takes less tokens to express the same concept.
  • Keywords are easier to spot than semi-colons.

D is better than B because…

  • Keywords are easier to spot than semi-colons.
  • It doesn’t have unmatched braces.

D is better than C because…

  • It takes less tokens to express the same concept.
  • It takes less characters to express the same concept.

Conclusion… not yet

It is too soon to pick a winner, though my personal bias tends towards one. I’m looking for some thoughts from others before I commit myself.

Foundry – Block Terminators I

Filed under: Foundry — Grauenwolf @ 6:50 pm

First an introduction. Foundry is meant to be a general purpose programming language that targets the CLR. I call it Foundry because there will be a heavy emphasis on making intelligent data objects. Building data objects tends to be very tedious. I especially when you start looking at wiring in update events for WPF, creating immutable variants of each class, tagging everything for serialization, etc.

Block Terminators

The construction of blocks is vital for most programming languages.  Popular options include keywords, braces, and whitespace. In order to test the options, I am looking at a hello world program. Using three block we see the program name, the list of assemblies that the program references, and the main function.

Whitespace 

  1. Program Test1
  2.     References
  3.         mscorlib
  4. Main
  5.     Print “Hello World”

I’m discarding this option because every time I try to copy and paste it the white spaces gets messed up.

Braces and Parens

There are three commonly used format for braces and parens.

Variant 1 – Suffix

  1. Program Test1 {
  2.     References {
  3.         mscorlib
  4.     }
  5. }
  6.  
  7. Main {
  8.     Print “Hello World”
  9. }

Variant 1 is in the running for now.

Variant 2 – Suffix on new line

  1. Program Test1
  2. {
  3.     References
  4.     {
  5.         mscorlib
  6.     }
  7. }
  8.  
  9. Main
  10. {
  11.     Print “Hello World”
  12. }

Burning two lines for each block header just seems stupid to me, so this one it out.

Variant 3 – Prefix

  1. {Program Test1
  2.     {References
  3.         mscorlib
  4.     }
  5. }
  6.  
  7. {Main
  8.     Print “Hello World”
  9. }

Placing the braces before the the block keyword makes it obvious that they pretty redundant. So how about a fourth variant instead.

Variant 4 – Closing only

  1. Program Test1
  2.     References
  3.         mscorlib
  4.     }
  5. }
  6.  
  7. Main
  8.     Print “Hello World”
  9. }

Very succinct, though perhaps a different closing character is in order. Definitely in the running.

Keywords

Variant 1 – End keyword with block identifier

  1. Program Test1
  2.     References
  3.         mscorlib
  4.     End References
  5. End Program
  6.  
  7. Main
  8.     Print “Hello World”
  9. End Main

This is redundant, but it does allow for a smarter compiler. If you see “end program” before “end references”, then you can tell the user what’s going on. Or you could even implicitly end the previous block. So this one is in the running.

Variant 2 – End keyword

  1. Program Test1
  2.     References
  3.         mscorlib
  4.     End
  5. End
  6.  
  7. Main
  8.     Print “Hello World”
  9. End

This isn’t quite as clear as the other one, but it is nice and terse.

Create a free website or blog at WordPress.com.