Surgery

Just a quick update.

In the spirit of the “iterate early and iterate often” mantra, I had to go back and clean up some shortcuts I had taken to get some features out the door and tested and used. Most of these tidy ups would not be noticeable performance-wise. I just tied up some loose ends in the code parsing and did some refactoring and removed duplicated code.

A bonus of this would be quicker delivery of new language features, and the directives (or “main” as I think I will rename it to) uses the same compilation/parsing routine as the templates; meaning code-completion in the main(), looping and conditional constructs in the main(), and you cal call templates within templates for recursive, stack-blowing goodness. An example of why you would want this would be for when you wish to loop through tables linked to a current table via foreign key constraints. These loops will be provided soon – including loops through indexes (and columns within indexes). And even further down the track loop through model entities themselves, not just a database schema.

This lead to the inevitable implementation of a symbol table for variables. Hopefully this will better support the code-completion which still has some niggles needing to be ironed out.

It was a surgical week. I don’t really like them. It’s that long stretch in hacker desert where the guts have been ripped open and tossed about all over the place .My AnkhSVN plug-in glowing red boxes in the solution explorer all over like a rash. But finally I reached that oasis where I can hit the commit button.

Performance improvements have been negligible. It’s plateaued so far. Compilation speed isn’t too bad, at a rate of about 5,000 lines per second. The syntax is pretty simple, it could be improved if I was brilliant but there probably won’t need to be any for the near future. Unless someone comes up with a chunky template of 40k lines. But I think the main concern from there would be generated output.

So generated output averages around 260k of data per second, or 3.84 seconds per megabyte of data . To test this I ran a template to generate typed datasets on a database with 1733 stored procedures and that took 34 seconds on average resulting in an 8.85mb code file. (I’d hate to think about the agony of dragging and dropping those 1733 stored procedures from the server tab into the dataset designer).

The performance will do for now. There is one more trick not available to the outside world and that is to not obfuscate the executable. A vanilla build took 22 seconds on average to generate the same 8.85mb of data; approx 402k per sec, or 2.5 secs for mb. That’s quite a performance hit with obfuscation, about 35% slower. I don’t know whether it’s the obfuscater itself, or actual obfuscation. Open source it you cry? One day perhaps. Depending of course on how clever (or successful) my revenue strategy is for this project.

The real solution I guess would be to generate managed code as part of the compilation process and compile and run that actual code. The Metadrone syntax will essentially then be a kind of a macro to create c#/vb code generation code (or a T4 template).

Beta 0.11

Beta 0.11 has been released.

The new features are:

1. CodeDOM implementation.
2. Expression evaluations have been revised and are much more robust and intuitive. This forms a kind of bedrock for the language.
3. Code completion fixes.
4. Searching has been revised and works more predictably.

There are some remaining features I have mentioned previously that will be implemented in the 1.0 launch. The next phase is now the launch with hopefully a stable and fully functional and feature rich code generation tool. Accompanying this will be the revamped website, documentation, and community facility.

Once we’re launched, we’ll try our luck with diving into the mysterious void of marketing and promotion. We’ll really find out if Metadrone is actually “wanted” by the developer community.

Temptation

I have been swimming in recursion soup the past fortnight to improve expression evaluations. For a moment I thought it would be “smarter” to build VB/C# code in codeDOM representing an expression in a template and substituting the primitives (variables) on the fly. This would mean having actual expressions compiled using the .NET framework and identified in a symbol table, and when used in the template the code is referenced and the variable values passed into their placeholders. This would probably would have resulted in a different type of spaghetti and be restricted to a language designed for programming, not model-driven code generation.

So it was head down, bum up, and(((lots of these) to sort), out) and precedences^to painfully(figure, out).

The hard work has paid off, and the result has been surprisingly neat. Also the syntax is not therefore bound to VB/C# to improve portability if need be.

That being said executing VB/C# code is ready to go. Various other bugs have been fixed and optimisations have been made to the compilation process (the tokenisation, building the syntax tree, etc). Text searching has been fixed up properly, and the code-completion has had some more annoying features done away with.

There are some more things to add onto for the final polish up before launch but these fixes are important enough for another beta release, which will come very soon.