Convert any text to a string literal

The last couple of months I found myself trying to convert pieces of code or html to a string allot.  Some of the stuff I was trying to do includes:

• Compilingsource code with the csharp code provider
• Creating my own HtmlHelper extensions that generate a Html string
• Writing unit test for a json serializer

All of the above required me to convert text with allot of quotes and abnormal characters into a string.  I searched the internet for some conversion site that would do this for me but couldn’t find one.  The only option that I had left was to write one of my own.

So if you are ever in the need to convert some pesky piece of text to a string and you’re not masochistic enough to do it manually, then be free to go to http://texttostringliteral.lucbos.net/ and this utility website will do it for you :-)

Till next time!

Small note: I’ve noticed the site takes a couple of seconds to load when my host recycled it.

CSharpCodeProvider + MEF = Match made in heaven

Okay so the following post probably has zero business value (like some other posts do) but I quit enjoyed myself working on it.   What I wanted to try out was make a system that would be extensible by code the user provides… at runtime … in a console application.  So I started fiddling around with the CSharpCodeProvider and was quite impressed by how easy to use it was!
 

So what the CSharpCodeProvider allows you to do is compile code at runtime.  Not something the everyday developer has to do but very fun for hobby projects :-)  So I then thought it would be cool if I could load in that code at runtime and saw the behavior of my code change while I was running it!

Shinny diagram…oooooh

Off course loading code at runtime is a breeze with MEF, so the only thing I had to do is tell MEF in which directory to load up the assemblies.  Just for the sake of putting pretty diagrams on my blog, here is one explaining what I’m trying to do:

Implentation explanation

The idea I came up with was a simple application that responds to the user based on previous expressions that were provided.  Seems pretty abstracting saying it like that so I’ll show you some screenshots to make it more obvious:

So when the user logs in he can either insert a new expression or let the program evaluate userinput.

When the user chooses to Insert an expression he has to provide a lambda statement of the type func<string,bool>.  This statement will later on be used to evaluate user input. 

Then he is supposed to input an answer that the application will provide when the statement returns true.

The application will then inject the lambda and the answer into some source code and compile it to a new assembly.

If we then choose to evaluate user input the application will load all at runtime compiled assemblies and use them to evaluate the users input.

The user can then choose to provide as much expressions and answers as he likes and change the behavior of how the system will respond to certain user input. 

Now the geek part begins… code!

So when the application asks for the expression and answer it will inline the requested input in the following string:

private string CreateSourceCode(string lambda, string answer)

        {

            return

    @"using System;

    using System.ComponentModel.Composition;

    using TestCompilation;

    [Export(typeof(IExpression))]

    class NewExpression : IExpression

    {

        public Func<string, bool> Expression

        {

            get

            {

                return " + lambda + @"

            }

        }

        public string Answer

        {

            get

            {

                return " +

    "\"" + answer + "\"" + @";

            }

        }

    }";

        }

Essentially the user input in combination with the string should provide a valid class in source code.  Maybe you’ve noticed the Export attribute on top of the class?  This is where MEF comes in.  When this source code is compiled we will use MEF to load the compiled assembly at runtime and inject it in our code.  The IExpression interface is just a simple interface with the two properties in it.

But how do we compile this at runtime?  We use the CSharpCodeProvider:

 var location = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

var csc = new CSharpCodeProvider(new Dictionary<string, string>() { { "CompilerVersion", "v4.0" } });

var parameters = new CompilerParameters(new[] { "mscorlib.dll", "System.Core.dll", "System.ComponentModel.Composition.dll", "ConsumerContracts.dll" },

                                                    location + "\\Extensions\\" + Path.GetRandomFileName() + ".dll",

                                                    true);

            CompilerResults results = csc.CompileAssemblyFromSource(parameters, sourceCode);

In the CompilerParameters we have to tell the CSharpCodeProvider which assemblies to use to be able to build our class.  If we leave these out it will return the TypeNotFound exception.

The result of the CompileAssemblyFromSource method will be a dll file located in the Extensions folder of my running assembly.

 

So the next step is to load in the assemblies located in the Extensions folder:

private void LoadExtensions()

{

    var location = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

    var catalog = new DirectoryCatalog(location + "\\Extensions\\");

    var container = new CompositionContainer(catalog);

    var batch = new CompositionBatch();

    batch.AddPart(this);

    container.Compose(batch);

}

All classes that implement IExtension will then be injected into the following member.

[ImportMany(typeof(IExpression))]

public ICollection<IExpression> Expressions = new List<IExpression>();

The only thing we need to do now is evaluate the users input against the loaded expressions.

private void EvaluateExpression()

{

    Console.WriteLine("Please say something and I will try to answer.");

    var userInput = Console.ReadLine();

    var matchingExpression = Expressions.FirstOrDefault(e => e.Expression(userInput));

    if (matchingExpression == null)

    {

        Console.WriteLine("Could not answer you.");

    }

    else

    {

        Console.WriteLine(matchingExpression.Answer);

    }

    Console.ReadKey();

    Console.Clear();

}

90% of readers won’t read this part

I probably lost 90% of you by now and my feedburner statistics will be dropping to an all-time low but, for the few diehard people that are interested in the full source code, you can find it on github.  Be aware that this was created in an hour or so, so don’t expect the source code to be all that.

Have fun!

Calling a stored procedure with Entity Framework Code First

Currently I’m doing a project that uses Entity Framework Code First for data access.  Though I’m not 100% convinced that this technology is mature enough, a coworker of mine recommended it.  For the read side of our project we want flattened dto’s that are made to fit our view’s need.  Since some of the views were a bit too complex to write with linq to entities we thought it would be best to write stored procedures for them.

The problem here is that Entity Framework Code First does not support Stored Procedures out of the box.  Well… it actually does, but in such a way that it is indistinguishable from plain ADO.NET.  Yet I wanted a more type safe solution.  I’m still not sure if this is the best solution for this problem but I thought I would share the code with you.

Imagine we have a stored procedure:

DECLARE @Iets int

DECLARE @NogIets bit

SET @Iets = 5

SET @NogIets = 1

EXECUTE [dbo].[sp_TestProcedure]

   @Iets

  ,@NogIets

GO

The standard way of calling the stored procedure in EF Code First would be something like this:

var ietsParameter = new SqlParameter("@Iets",5);

var nogIetsParameter = new SqlParameter("@NogIets",true);

DbContext.Database.SqlQuery<ProcedureResult>("sp_TestProcedure @Iets @Nogiets", ietsParameter, nogIetsParameter);

After creating the database extension you can write a more typesafe solution:

var testProcedureStoredProcedure = new TestProcedureStoredProcedure() { Iets = 5, NogIets = true };

var result = DbContext.Database.ExecuteStoredProcedure(testProcedureStoredProcedure);

The typed stored procedure looks like the following:

public class TestProcedureStoredProcedure : IStoredProcedure<ProcedureResult>

{
    public int Iets { get; set; }
    public bool NogIets { get; set; }
}

Notice that you have to inherit from a generic IStoredProcedure and supply the type of the result of the stored procedure.  The stored procedure class contains properties that are used as the parameters for the execution of the stored procedure.  The extension will use the name of the class to determine which stored procedure to execute.  The default behavior is ‘sp_’+classname but you can edit the source to your own naming convention.

You can download the database extension class here, or get it from github.  Just add a reference to the project and import the namespace to start using the extension method.

There is a drawback however.  The execution time is slower because before it can call the query it has to reflect the properties of the stored procedure class.  It’s still a work in progress but it can be undoubtedly be executed faster by adding some kind of cache for the stored procedure names into it.  If you have any improvements then I would be happy to hear about them!

A small note: if you have any questions about my blog posts or if there are any links broken then please add a comment to the blog post in question.  Mails often get lost in my mailbox and it can take a couple of weeks before I can answer them.

I've created a new extensions project for Entity Framework Code First.  You can read all about it here!

Reconstructing an OpenXml Element from Xml

First of all let me wish everyone out there a happy new year!  Keep on coding and off course keep on reading my blog :-P

So now back to business.  A couple of months ago, while I was writing the WordProcessingSerialization project, I've bumped into a little problem.   The OpenXml SDK 2.0 lets you read the OuterXml of an OpenXmlElement but it doesn’t allow you to construct an OpenXml element from XML.  Yet I think that this is an issue for allot of developers that want to save the raw XML for later reuse.  Last week I seem to have bumped into this problem again so I thought let’s fix this!



It proved rather difficult to construct the OpenXml element from XML.  The main reason why it was that difficult is because there doesn’t seem to be a factory class in the SDK that does the conversion.  Instead when I looked at the implementation of the SDK I was kind of chocked that the development team opted for such a weird implementation (and gigantic flaw in separation of concerns).  It seems that each individual OpenXml is responsible for constructing their children.  Here is a small example of their code:

     internal override OpenXmlElement ElementFactory(byte namespaceId, string name)

    {

      if (23 == (int) namespaceId && "rPr" == name)

        return (OpenXmlElement) new RunProperties();

      if (23 == (int) namespaceId && "br" == name)

        return (OpenXmlElement) new Break();

      if (23 == (int) namespaceId && "t" == name)

        return (OpenXmlElement) new Text();

      if (23 == (int) namespaceId && "delText" == name)

        return (OpenXmlElement) new DeletedText();

      if (23 == (int) namespaceId && "instrText" == name)

        return (OpenXmlElement) new FieldCode();

…

So after some more peeking it was obvious to me that if I wanted to create some kind of factory class, I was going to have to do it myself.  Without boring you too much with pages of code I’ll give you the link to the Git Repository where you can download the project yourself.

The usage of the code is very easy:

var element1 = new OpenXmlElementReconstructor().Reconstruct(outerXml)

So you just create an instance of the OpenXmlElementReconstructor and call the Reconstruct method which accepts a parameter that contains the Xml value.

At the time of writing the library is 80% completed.  I reassemble the OpenXmlElements by searching for their implementation with reflection but I’m planning on creating a preassembled map of all OpenXmlElements so the execution time will be faster.  Current execution times are 439 milliseconds at the first execution time and from then on 47 milliseconds for subsequent calls.  The first call is pretty heavy but that should be solved when I have finished the preassembled map.

Enjoy!

Download link: https://github.com/LucBos/OpenXmlFactory

Legacy code retreat

This Saturday I had the privilege of attending another code retreat, only it wasn’t just a normal everyday code retreat but a legacy code retreat! This code retreat was hosted by J. B. Rainsberger so big thanks to him!

Whats the difference?

The basic setup is pretty much the same.  You pair with another coder and start an intensive 45 minutes session of coding.  After each session there is a retrospective of what kind of stuff you tried and how well it went.  The big difference in between a normal code retreat and a legacy code retreat is that you don’t start from scratch but you begin with an existing code base.  It is then your task to refactor the code as best as you can in 45 minutes without breaking existing functionality.

Test as documentation and failsafe

The first session wasn’t that special.  The code was handed to us and the purpose was to try and figure out what it does.  I started out writing tests for the class with the most behavior and just started from top to bottom to figure out each individual behavior.  I enjoyed this allot because not only do you get to cover the existing code with tests so you wouldn’t break the functionality when refactoring later, the tests in itself is an excellent way of discovering how the code base works and behaves.

After the first session a retrospective was held and it was surprising to see how many pairs actually didn’t even run the code!  To be honest I didn’t run it too.  Another surprising feat was that some couples even managed to find bugs in the code from the first couple of unit tests!

The golden master

So then came the second session.  Here we were asked to write end to end tests with a technique called the golden master.  The idea behind this is that you save the output of the program to a separate file.  There should be enough variation in the input of your parameters to try and cover as much of your program as possible.  As a rule for the code handed by us, we wrote away 10.000 different variations of the parameters. The output you’ve written away should be confirmed as the correct behavior of the program as it will be used as a reference for the next tests.

When you refactor the code you can then run the golden master to save the output of the refactored code.  The correctly verified output will then be compared to the output of the refactored code and any differences between the two will be displayed.

This technique is very useful for when you have an application with zero tests in it and the code is too coupled to test.  You can then slightly change the code to make unit testing easier and still have a small safe-zone in which you can verify the correctness of your application.

Some couples (including me again -_-‘ ) tried to save all the possible values the parameters could form.  This turned out to be a mistake as the variations ran up to a couple of millions.

I did have one issue though and that is that if you weren’t able to get the end to end tests done in this exercise you didn’t have a backup testing mechanism in the other sessions.  I addressed this to JB and he said he was going to plan to upload the Golden Master so that next code retreat this wouldn’t be a problem any more.

Mocking behavior by sub classing

This technique was already known to me but I didn’t know it would prove as useful as it did when dealing with legacy code.  Subclass To Test… basically the name says it all.  When to code has a strong coupling to it and mocking behavior of the class proves difficult we can make the methods of that class virtual (overridable/@override).  We can then subclass this class and override the methods that we want to mock in order to test our initial method.  I’ll demonstrate this by some code:

Say we have the following legacy code:

public class BootyShaker

{

    private List<string> _booties;

    public int NrOfBooties

    {

        get { return _booties.Count(); }

    }

    public void CreateSomeBooties()

    {

        _booties = new List<string>();

        // Legacy code should contain magic numbers!!!

        for (int i = 0; i < 12; i++)

        {

            _booties.Add(CreateBooty());

        }

    }

    /// <summary>

    /// Disclaimer: I do not take any responsibility for suicides as a consequence of utilizing this method

    /// </summary>

    /// <returns></returns>

    public string CreateBooty()

    {

        string thaBootie = string.Empty;

        // Some complex behavior

        return thaBootie;

    }

}

So we want to test the CreateSomeBooties method but the developer didn’t make it easy for us.  To make matters even worse the CreateBootie method is called by a couple of dozen other methods that we haven’t tested yet!  So in order to mock the behavior without interfering with the rest of the untested (!) code we’ll subclass the BootyShaker.

public class BootyShaker

{

    private List<string> _booties;

    public int NrOfBooties

    {

        get { return _booties.Count(); }

    }

    public void CreateSomeBooties()

    {

        _booties = new List<string>();

        // Legacy code should contain magic numbers!!!

        for (int i = 0; i < 12; i++)

        {

            _booties.Add(CreateBooty());

        }

    }

    /// <summary>

    /// Disclaimer: I do not take any responsibility for suicides as a consequence of utilizing this method

    /// </summary>

    /// <returns></returns>

    public virtual string CreateBooty()

    {

        string thaBootie = string.Empty;

        // Some complex behavior

        return thaBootie;

    }

}

[TestFixture]

public class BootyShakerTest

{

    [Test]

    public void CreateSomeBooties_should_create_12_booties()

    {

        var bootyShaker = new BootyShakerSubclassTest();

        bootyShaker.CreateSomeBooties();

        Assert.AreEqual(bootyShaker.NrOfBooties, 12);

    }

}

public class BootyShakerSubclassTest : BootyShaker

{

    public override string CreateBooty()

    {

        return "Mah booty";

    }

}

We need to make the method that we want to mock virtual (or overridable) so we can change the behavior in the subclass.  Then when we have successfully mocked the behavior we can write a test for the original method.  When the test is written we can refactor CreateSomeBooties.

Delegate the call by using Subclass To Test

In the session after that we went a step further.  Now that we’ve created the base class that mocks the behavior of the CreateBooty method we can refactor this method to the appropriate type and delegate the call.  Phew… I think I’ve lost you there :-P.  I’ll try to speak the universal language then… code:

So simply put the CreateBooty behavior doesn’t belong inside the BootyShaker so we’re going to try and extract the behavior out of the class.  So the first thing we need to do is inject the new type into the original BootyShaker:

public interface IBootyCreator

{

    string CreateBooty();

}

…

private readonly IBootyCreator _bootyCreator;

public BootyShaker(IBootyCreator bootyCreator)

{

    _bootyCreator = bootyCreator;

}

Don’t forget to adjust your base class to so you can still run your test:

public BootyShakerSubclassTest(IBootyCreator creator):base(creator)

{

       

}

Then refactor away the code in our base class inside a mock:

public class MockBootyCreator : IBootyCreator

{

    public string CreateBooty()

    {

        return "Mah booty";

    }

}

Then delegate the call in our BootyShaker:

public virtual string CreateBooty()

{

    return _bootyCreator.CreateBooty();

}

And then remove the base class and use the original Bootyshaker with the injected 

MockBootyCreator:

[Test]

public void CreateSomeBooties_should_create_12_booties()

{

    var bootyShaker = new BootyShaker(new MockBootyCreator());

    bootyShaker.CreateSomeBooties();

    Assert.AreEqual(bootyShaker.NrOfBooties, 12);

}

So now you’ve coupled away the responsibility of creating booties to a new type!

Extracting pure functions

The next session was a bit different.  This time we had to refactor by extracting pure functions.  The good thing about doing this is that you do not alter the state of the object in which you’re working thus this makes it very easy to test.

This wasn’t really new to me and I try to do this as much as possible since working without state reduces allot of the risk and makes testing allot easier.

Actually do stuff

The last 2 sessions JB decided to do something different.  This time we weren’t required to throw away our code so we could actually get some work done! To be honest, at the end of the day my head felt like it was going to explode so I needed the two extra sessions to be a bit more productive.

The problem I had with the two sessions is that you still had to switch pairs.  You would then have to adjust to the code of your new partner.  I would think it would be better if it was just one extra-long session since it takes a couple of minutes to get into each other’s mindset.

So that was it for the legacy code retreat.  Big thanks to Erik for the organization and to JB for taking the time to guide us :-).

Join us next time!  Till then...