Surgery

Just a quick update.

In the spirit of the “iterate early and iterate often” mantra, I had to go back and clean up some shortcuts I had taken to get some features out the door and tested and used. Most of these tidy ups would not be noticeable performance-wise. I just tied up some loose ends in the code parsing and did some refactoring and removed duplicated code.

A bonus of this would be quicker delivery of new language features, and the directives (or “main” as I think I will rename it to) uses the same compilation/parsing routine as the templates; meaning code-completion in the main(), looping and conditional constructs in the main(), and you cal call templates within templates for recursive, stack-blowing goodness. An example of why you would want this would be for when you wish to loop through tables linked to a current table via foreign key constraints. These loops will be provided soon – including loops through indexes (and columns within indexes). And even further down the track loop through model entities themselves, not just a database schema.

This lead to the inevitable implementation of a symbol table for variables. Hopefully this will better support the code-completion which still has some niggles needing to be ironed out.

It was a surgical week. I don’t really like them. It’s that long stretch in hacker desert where the guts have been ripped open and tossed about all over the place .My AnkhSVN plug-in glowing red boxes in the solution explorer all over like a rash. But finally I reached that oasis where I can hit the commit button.

Performance improvements have been negligible. It’s plateaued so far. Compilation speed isn’t too bad, at a rate of about 5,000 lines per second. The syntax is pretty simple, it could be improved if I was brilliant but there probably won’t need to be any for the near future. Unless someone comes up with a chunky template of 40k lines. But I think the main concern from there would be generated output.

So generated output averages around 260k of data per second, or 3.84 seconds per megabyte of data . To test this I ran a template to generate typed datasets on a database with 1733 stored procedures and that took 34 seconds on average resulting in an 8.85mb code file. (I’d hate to think about the agony of dragging and dropping those 1733 stored procedures from the server tab into the dataset designer).

The performance will do for now. There is one more trick not available to the outside world and that is to not obfuscate the executable. A vanilla build took 22 seconds on average to generate the same 8.85mb of data; approx 402k per sec, or 2.5 secs for mb. That’s quite a performance hit with obfuscation, about 35% slower. I don’t know whether it’s the obfuscater itself, or actual obfuscation. Open source it you cry? One day perhaps. Depending of course on how clever (or successful) my revenue strategy is for this project.

The real solution I guess would be to generate managed code as part of the compilation process and compile and run that actual code. The Metadrone syntax will essentially then be a kind of a macro to create c#/vb code generation code (or a T4 template).