Coding time - Martin Konicek

How to implement rule engine in C#

2011-07-03T04:10:00.000-07:00

Recently there was an interesting question on StackOverflow about creating a rule engine in C#.

Say we have a collection of Users (with Name, Age, ...) and rule definitions like this:

static List<Rule> rules = new List<Rule> {
new Rule ("Age", "GreaterThan", "20"),

new Rule ( "Name", "Equal", "John"),
new Rule ( "Tags", "Contains", "C#" )
};

and we want to be able to evaluate the rules:

// Returns true if User satisfies given rule (e.g. 'user.Age > 20')

bool Matches(User user, Rule rule)

    // how to implement this?

One obvious solution is to use reflection.
Here we will show a solution which uses Expression trees to
compile the rules into fast executable delegates. We can then evaluate the rules as if they were normal boolean functions.

public static Func<T, bool> CompileRule<T>(Rule r)

    var paramUser = Expression.Parameter(typeof(User));

    Expression expr = BuildExpr<T>(r, paramUser);

    // build a lambda function User->bool and compile it

    return Expression.Lambda<Func<T, bool>>(expr, paramUser).Compile();

static Expression BuildExpr<T>(Rule r, ParameterExpression param)

    var left = MemberExpression.Property(param, r.MemberName);

    var tProp = typeof(T).GetProperty(r.MemberName).PropertyType;

    ExpressionType tBinary;

    // is the operator a known .NET operator?

    if (ExpressionType.TryParse(r.Operator, out tBinary))    {

        var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));

        // use a binary operation, e.g. 'Equal' -> 'u.Age == 15'

        return Expression.MakeBinary(tBinary, left, right);

    } else {

        var method = tProp.GetMethod(r.Operator);

        var tParam = method.GetParameters()[0].ParameterType;

        var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tParam));

        // use a method call, e.g. 'Contains' -> 'u.Tags.Contains(some_tag)'

        return Expression.Call(left, method, right);

Now we can implement the rule validation by compling the rules and then simply invoking them:

var rule = new Rule ("Age", "GreaterThan", "20");

Func<User, bool> compiledRule = CompileRule<User>(rule);

// true if someUser.Age > 20

bool isMatch = compiledRule(someUser);

Of course we compile all the rules just once and then use the compiled delegates:

// Compile all the rules once.

compiledRules = rules.Select(r => CompileRule<User>(r)).ToList();

// Returns true if user satisfies all rules.

public bool MatchesAllRules(User user)

    return compiledRules.All(rule => rule(user));

Note that the “compiler” is so simple because we are using 'GreaterThan' in the rule definition, and 'GreaterThan' is a known .NET name for the operator, so the string can be directly parsed. The same goes for 'Contains' – it is a method of List. If we need custom names we can build a very simple dictionary that just translates all operators before compiling the rules:

Dictionary<string, string> nameMap = new Dictionary<string, string> {

    { "greater_than", "GreaterThan" },

    { "hasAtLeastOne", "Contains" }

};

That’s it. As an improvement we could add error messages to the rule definitions and print the error messages for the unsatisfied rules.

If you found this useful, feel free to rate the answer on StackOverflow.

Covariance and contravariance - simple explanation

2011-05-17T14:39:00.000-07:00

I tried to be as concise as possible. In 10 minutes you should understand what
covariance/contravariance is and how to take advantage of it. The examples are in Scala.

Covariance

Covariance lets you treat say List[Apple] as List[Fruit].

val apples = List(new Apple(), new Apple())

processList(apples)

def processList(list:List[Fruit]) = {

// read the list

}

This seems obvious - indeed, a list of apples is a list of fruit, right?

The surprise comes when we find out this does not work for arrays. Why is that so? Because you could do the following:

val a = Array(new Apple(), new Apple())

processArray(a)

def processArray(array:Array[Fruit]) = {

array(1) = new Orange() // putting an Orange into array of Apples!

}

The main difference between List and Array here is that the List is immutable (you cannot change its contents) while the Array is mutable. As long as we are dealing with immutable types, everything is OK (as in the first example).

So how does the compiler know that List is immutable? Here is the declaration of List:

sealed abstract class List[+A]

The +A type parameter says "List is covariant in A". That means the compiler checks that there is no way to change contents of the List, which eliminates the problem we had with arrays.

Simply put, a covariant class is a class from which you can read stuff out, but you can't put stuff in.

Contravariance

Now when you already understand covariance, contravariance will be easier - it is exactly the opposite in every sense.

You can put stuff in a contravariant class, but you can never get it out (imagine a Logger[-A] - you put stuff in to be logged). That doesn't sound useful, but there is one very useful application: functions. Say you've got a function taking Fruit:

// isGoodFruit is a func of type Fruit=>Boolean
def isGoodFruit(f:Fruit) = f.ageDays < 3

and filter a list of Apples using this function:

val list:List[Apple] = List(new Apple(), new Apple())

list.filter(isGoodFruit) // filter takes a func Apple=>Boolean

So a function of Fruits can be treated as function of Apples - the filter will throw Apples in and isGoodFruit will know how to handle them.

The type of isGoodFruit is actually Function[Fruit, Boolean] - yes, in Scala even functions are traits, declared as:

trait Function[-A,+B]

So functions are contravariant in their parameter types and covariant in their return types.

OK, that's it; this is the minimal explanation I wanted to cover.

Did you find this understandable? Please provide feedback so I can make things even clearer. Thanks!

Software engineering radio - best episodes

2011-05-05T05:00:00.000-07:00

Software engineering radio is an excellent podcast full of in-depth information for developers; contains high quality content different from what you usually find online.

General

NoSQL and MongoDB with Dwight Merriman
Top 10 Architecture Mistakes with Eoin Woods
Being a consultant - honest, informal and funny
Software Craftsmanship with Bob Martin - concentrated motivation
Stefan Tilkov on REST - quite practical
Singularity research OS - microkernels, safety, static code analysis

Scala

Martin Odersky on Scala - great interview with the author of Scala
Scala Update with Martin Odersky - second half provides insights into possible future of programming

OmegaTau

Btw, Markus Völter (the guy behind se-radio) also does a podcast on technology and science. Software engineer talking to a Nuclear fusion expert - what could we want more? ;) I really enjoyed the following episodes:

Astrobiology at the NASA Astrobiology Institute
Quantum computing - kudos for mentioning the "next technical revolution"
Nuclear Fusion at MPI für Plasmaphysik

Please help me find great episodes - if you know about an episode that you really enjoyed, post it into comments. Thanks!

TFS addin for Outlook TaskConnect released!

2011-02-06T15:08:00.000-08:00

TaskConnect connects Outlook with issue tracking systems in a very cool way. So far there is support for TFS (2008, 2010) and Redmine.

This is how TaskConect looks inside Outlook:

It has many useful concepts making you deal with your tasks in no time. Be sure to check out the entirely new website.

Scripting in Scala

2011-01-21T07:02:00.000-08:00

Imagine we have a long text file, for example:

'CHINESE' : 'zh',
'DUTCH': 'nl',  
'ENGLISH' : 'en',
'SPANISH' : 'es',

And we want to turn it into this:

'zh' : 'Chinese',
'nl' : 'Dutch',  
'en' : 'English',
'es' : 'Spanish',

What we want to do is very simple: "Swap the two parts on every line!", and I believe it should be this simple also in code. Let's see how it looks in Scala:

import scala.io.Source

val lineRegex = "\\s+'(.+)'.*'(.+)'.*".r

for(line <- Source.stdin.getLines) {
   val lineRegex(name, code) = line
   println("'%s': '%s'," format (code, name.toLowerCase.capitalize))
}

That's 5 lines including the import, and the code is readable. Now we can run:

cat languages.txt | scala ourScript.scala > output.txt

That's it.

Btw, let's look at at the options we had for solving the problem:

We don't want to be writing code in Java, C++ or C# and actually compiling an executable. Also you can imagine that the code in any of these languages would be overly complicated.
OK, so we definitely should use a scripting language. What options do we have?
- sed, awk, perl - Learn a (cryptic) language just for the purposes of scripting? No thanks.
- Python, F#, Scala - All these languages have an amazing property that while you are already building your applications in them, you can use them for writing quick scripts. Scala is just elegant and more innovative than F# or Python. For a deeper look how the magic with lineRegex works, see this. If unfamiliar with Scala, here is a quick overview.

Programming not fun anymore?

2010-11-17T09:46:00.000-08:00

I'm writing this because recently I'm seeing more and more friends moving from programming to something "high level" such as management or consulting.

They say they had enough of building crap database applications in a cellar with a bunch of coworkers.

To all people like this I say:
Come on guys! What's the matter with you? We used to be programmers and we kicked ass. It was your child dream. Have you forgotten the feeling of learning your first languages and building stuff at home just for fun?

I'll tell you what - programming IS fun. What is not fun is building information systems for small businesses over and over again (actually that's one of the most boring things in the world). But building such things is not programming. Why don't you find a good programming job instead of complaining about being a servant to people who "would like one more button on the customers page"?

I once had the feeling that programming is not the right thing anymore as well. I have a theory about where it comes from: It is all about this feeling of social isolation. And it goes away when you have a girlfriend you care about. It is hard to be working only with guys while wishing to meet a girl - that can be challenging. While not having the girlfriend, hanging out with non-IT friends helps a lot and when you finally have the girlfriend, you can start to love programming again.

Oh, and one last thing. Did you have a feeling that programmers are not respected by non-technical people? That's not true at all. I realized when no-IT friends asked me about what my summer job was so I said "Google summer of code". The reaction was always the same - "wow!". And people tell each other and keep asking about it. More and more friends ask me "So how do all these programs and websites really work?" Ordinary people realize the importance of software and they do find awesome software awesome.

Happy coding!
Martin

New blogposts about programming Outlook add-ins

2010-10-25T15:00:00.000-07:00

I published two new blogposts on the website of our add-in for Outlook called TaskConnect:
Adding custom data to Outlook emails
Searching emails from an Outlook add-in

Knowing the information published there before developing TaskConnect would definitely help me a lot so I'm sharing it. I hope you will also find it useful.

TFS addin for Outlook TaskConnect goes live!

2010-04-17T07:50:00.000-07:00

You can download the first beta of TaskConnect now. I've been working on this for the last year with a team of cool guys in Prague.
TaskConnect is quite unique as it integrates TFS into Outlook in a new, clever way:

TaskConnect is a full-fledged TFS client with fulltext search field at your fingertips right in Outlook
Work items can be attached to emails: when you are discussing a work item with someone, the work item is visible right next to the email throughout your whole conversation. You can of course edit it right from Outlook.
Saving your time is absolutely the primary goal - speed of use is just incomparable to any existing TFS client

We think this is such a good idea that it would be silly just to support TFS. So in the future you can expect Outlook integration of many popular issue tracking systems. We also have a ton of ideas for the future and we are looking forward to your feedback and ideas.

C# 4 delegates contravariance

2010-01-18T19:58:00.000-08:00

A little quiz.

Given the following two delegates:

delegate void EventHandler(object sender, EventArgs e);
 
delegate void PropertyChangedEventHandler(object sender, PropertyChangedEventArgs e);

Does the following code compile?

void propertyHandler(object sender, PropertyChangedEventArgs e)
{ }
 
EventHandler handler = propertyHandler;

Did you answer "YES, because PropertyChangedEventHandler IS an EventHandler"?

Unfortunately, the answer is no. Imagine how would the following code work:

handler(this, new EventArgs());

How could just EventArgs be passed to propertyHandler, which expects PropertyChangedEventArgs?

Actually, the exact opposite is correct in C# 4:

void handler(object sender, EventArgs e)
{ }
 
PropertyChangedEventHandler propertyHandler = handler;

Then, calling

propertyHandler(this, new PropertyChangedEventArgs("Name"))

is perfectly ok, because the handler just sees passed PropertyChangedEventArgs as EventArgs.

So the conclusion is: you can handle specialized events using less specialized handlers (you can handle eg. PropertyChanged event using just an EventHandler).

For more info, see C# delegates on msdn.

Implementation of anonymous delegates in C#

2009-11-09T07:18:00.000-08:00

What is the output of the following program?

int x = 5;

Action action = (() => { Console.WriteLine(x); });

x = 6;

action();

The output is of course 6, because delegates capture references to outer variables.

So how is this implemented in the compiler?
The compiler rewrites the code above to something like this:

Scope s = new Scope();
scope.x = 5;
Action action = (scope);
scope.x = 6;
scope.Call();

and defines the Scope class like this:

private sealed class Scope
{
    public int x;
 
    public Scope();
    public void Call()
    {
        Console.WriteLine(this.x);
    }
}

As we would expect, the is no way to create a reference to eg. an int in IL, so this is solved naturally - the int is wrapped inside Scope class and all accesses are done through the wrapper.
The wrapper also defines the Call method, which is exactly the body of the anonymous delegate.

Actually, the compiler generated name for our Scope class is "<>__DisplayClass1", and the Call method is named "<Main>b__0".

For a little more complex example, see this post on stackoverflow.

HTTPS and DNS poisoning attack

2009-06-14T17:08:00.001-07:00

The best explanation of HTTPS I have seen was written by Jeff Moser, highly recommended!

After reading the article, there was only one thing left unclear, so I asked the author, Jeff Moser, and he responded:

Me:

Just one thing is really unclear to me - DNS poisoning: The attacker obtains certificate from amazon.com, I enter "amazon.com" to browser, browser goes to attacker's site, which responds by valid amazon.com certificate signed by Verisign. How does the browser tell this is an attack?

Jeff:

Great question! Note that if an attacker did this, they'd run into trouble in the "Trading Secrets" section that I described. Without knowing Amazon.com's private key, they couldn't decrypt the pre-master secret that the client sends out because the certificate from Verisign has Amazon's public key. Thus, the client would use that public key (and not one an attacker generated).

DNS poisoning is an attack when attacker fools DNS server. You type "amazon.com" in the browser, the browser asks the DNS server to resolve the URL = to translate the URL to IP address. Since the DNS server is poisoned, it returns attacker's IP address and browser connects to attacker's server, while address bar reads "amazon.com" - quite nasty.

Now everything is 100% clear, thanks Jeff!

Custom IFormatProvider for doubles

2009-06-12T05:02:00.000-07:00

The following example shows how to write a custom IFormatProvider which you can use in String.Format(IFormatProvider, ...).

public class DoubleFormatter : IFormatProvider, ICustomFormatter

{

// always use dot separator for doubles

private CultureInfo enUsCulture =

CultureInfo.GetCultureInfo("en-US");

public string Format(string format, object arg,

IFormatProvider formatProvider)

{

// format doubles to 3 decimal places

return string.Format(enUsCulture, "{0:0.000}", arg);

}

public object GetFormat(Type formatType)

{

return (formatType == typeof(ICustomFormatter))

? this : null;

}

Having this formatter, we can use it like this:

double width = 15.77555;

double height = 12.8497979;

Console.WriteLine(

string.Format(new DoubleFormatter(), "w={0} h={1}", width, height));

Output:

w=15.776 h=12.850

So now we have a reusable format for doubles - 3 decimal places with dot separator. That is nice, but this formatter is very simple - it formats everything (eg. DateTime) as "0:000". This is a fast version if you know that you will only use it for formatting lots of doubles.

The real version should look like this:

public class DoubleFormatter : IFormatProvider, ICustomFormatter

{

// always use dot separator for doubles

private CultureInfo enUsCulture =

CultureInfo.GetCultureInfo("en-US");

public string Format(string format, object arg,

IFormatProvider formatProvider)

{

if (arg is double)

{

if (string.IsNullOrEmpty(format))

{

// by default, format doubles to 3 decimal places

return string.Format(enUsCulture, "{0:0.000}", arg);

}

else

{

// if user supplied own format use it

return ((double)arg).ToString(format, enUsCulture);

}

// format everything else normally

if (arg is IFormattable)

return ((IFormattable)arg).ToString(format, formatProvider);

else return arg.ToString();

}

public object GetFormat(Type formatType)

{

return (formatType == typeof(ICustomFormatter)) ? this : null;

}

Example:

Console.WriteLine(string.Format(new DoubleFormatter(),

"Numbers {0} and {1:0.0}." +

"Now a string {2}, a number {3}, date {4} and object: {5}",

1.234567, -0.57123456, "Hi!", 5, DateTime.Now, new object()));

Output:

Numbers 1.235 and -0.6. Now a string Hi!, a number 5, date 12.6.2009 17:11:35 and object: System.Object

This article should give you an overview of implementing custom IFormatProvider. Now you should be able to modify the code to suit your specific needs.

Other examples with custom formatters can be found in MSDN. See example with formatter for 12-digit account numbers (12345–678–9012).

Associating data with an event

2009-06-04T05:55:00.000-07:00

I just solved the following problem - best explained by concrete example:

WPF Animation.Completed is an event. I need to register this event and when it fires, access custom data associated with the event:

void animate(Graph graph)

{

PointAnimation anim = new PointAnimation();

anim.Completed += new EventHandler(anim_Completed);

// in anim_Completed, I want to call graph.Fix() - how?

}

void anim_Completed(object sender, EventArgs e)

{

// how do I access 'graph' here?

}

Seems like a tough problem. Remembering 'graph' in eg. static variable is not an option, especially if we have more graphs animated at the same time.
Fortunately, C#'s lambdas come to the rescue:

void animate(Graph graph)

{

PointAnimation anim = new PointAnimation();

anim.Completed += new EventHandler((s, e) => { graph.Fix(); });

}

This works because anonymous methods bind to outer variables.

Starting with Scala - slides from presentation

2009-05-16T15:21:00.000-07:00

I started learning Scala programming language.

Scala rocks! It is statically typed, has functional features, and aims for developer productivity. In Scala you can express what you want with less code than in C#. Scala runs on JVM and any Java code can be called from Scala.

Here are the slides from my presentation at Charles University in Prague: pdf, powerpoint, examples (.zip)

Some of Scala's clever concepts suprised me:

Actors - a totally different thinking about concurrency than you are used to. No threads + shared state + locks. Actors live independently, share no state, and communicate by sending each other messages.
Adding new keywords to the language just by writing ordinary methods. Consider adding C#'s using or lock keyword to Scala, without compiler support!
Traits - behaviors can be mixed in freely at instatiation time, sort of built-in Strategy design pattern.
Absence of the static keyword, turns out it is not needed! Instead of having static state and methods of a class, you declare a singleton instance and call its methods.

If you want to see concrete examples, check out the slides.. it's coding time ;)

Links to get started:
First steps to Scala - a great tutorial.
Twitter on Scala - detailed interview - did you know that Twitter runs on Scala?

Thanks to my friends Joe and Pz for support!

I'm in!

2009-04-22T02:52:00.000-07:00

Great news! I've been accepted to Google summer of code. I'll be working on debugger visualizers for SharpDevelop.
This is a great opportunity to continue my work on the Object graph visualizer, and even get it integrated in an open source IDE.
I'll be also working on other visualizers and things (particularly one of them totally cool in my opinion).
Cool thing about this is that none of the visualizers is present in Visual Studio.
Hope the visualizers will make your debugging experience even better :)

Debugger Visualizer for Visual Studio

2009-03-31T06:01:00.000-07:00

Recently, I have been playing with implementing a Visual Studio add-in that displays your data structures as graphs live as you debug.

The graph on the right shows current state of the data structure being watched. The graph updates as you step through the code.
Here is an example of more complex structure:

The layout is done using GLEE graph layout engine, which could be possibly replaced by Graphviz or something else.

As I was suspecting prior to the implementation, there is an important issue that needs to be solved: when the user does a step in the debugger, the object graph is rebuilt from scratch and the layout engine calculates new layout. However, even if the graph changes only slightly, it can sometimes affect the layout significantly and therefore confuse the user.
I am currently thinking about matching nodes from current step to the nodes from the previous step (provided the graph doesn't change that much between steps, which is typically true - and when it chages too much, we don't mind layout changes anyway). The matching of the two graphs could be used to preserve the layout until the nodes really have to move. And if they have to move, move them in a way that they preserve relative positions to each other. There is a space for some interesting algorithms here.

Current source code of the add-in is available here.

Subversion visually explained in 30 sec - svn tutorial

2008-04-16T13:46:00.001-07:00

This video tutorial tries to explain the basics of how SVN is used. The buttons at the bottom are movie controls. Scroll down for conflict explanation.

svn.swf

This explains a conflict while working with Subversion:

svn_conflict.swf

Summary of SVN commands (do these commands with TortoiseSVN by right clicking on a file or directory):

SVN checkout
The first command to use Subversion. Downloads repository contents to a folder on your PC, makes this folder special SVN folder with SVN icon.
Update
Merges new changes from repository to your working copy. Update before every commit is important because of possible conflicts, that must be resolved by conversation with your teammate, because concurrent changes cannot be resolved automatically.
Commit
Megres changes from your yorking copy to SVN repository.
Resolved
Tells SVN that the conflicted file is now OK.
Add
Schedules new file to be saved to repository with next commit.
Delete
Shedules file to be deleted from repository with next commit. SVN keeps all history, so accidental deletes are not a problem.
Revert
Undoes all changes to your working copy since last commit. Usefull if you really mess up and want to throw away the changes and start over.
Log
Shows nice log of changes sorted by date, even with simple statistics.

Why does XML have attributes?

2008-03-28T05:26:00.001-07:00

Have you ever wondered why there are also attributes in XML and not just elements, causing many never-ending "element vs attribute" discussions?
And why is XML so redundant - repeating the tag name in the ending tag, thus making the document much larger?
I asked Charles Goldfarb, the author of SGML - the original format from the 1960s (already had <tags> and attributes) from which XML and HTML were derived.
He was very kind to reply to me. Enjoy:


- In <elem>text</elem>, why repeat "elem" for the second time?

In fact, there is an option in SGML that allows you to omit the element-type
name from an end-tag, but it is rarely supported. (In XML the option does not
exist at all.) The name is there primarily for the convenience of humans
debugging the markup, as there are large document types, such as aircraft
maintenance manuals, where elements can nest very deeply.


- Why are there attributes in SGML? One could express the same just with elements.

In fact, one could just as easily express the same just with attributes!

The basic notion is that elements represent objects while attributes represent
their properties. However, the property "content" is given a convenient syntax
-- everything between the start-tag and matching end-tag -- which makes it
simpler to express content hierarchy by simply nesting elements.

In a nutshell, if information is part of the content of the document, represent
it as an element. If it is some other property, represent it as an attribute.

I wish you the best of luck with your studies.

Charles Goldfarb

--
©2008 Charles F. Goldfarb * www.xmlhandbook.com * www.xmlbooks.com
The XML Handbook?* 5th Edition ISBN 0-13-049765-7 * 100,000 in print!
--

Implement your own Parallel.For in C#

2008-03-17T14:18:00.000-07:00

This article is intended for .NET 2 or 3.5. If you are on .NET 4, use System.Threading.Parallel class.

I was thinking about how Parallel.For could be implemented. I wrote my own, I am using it in my own project and it scales very well.

Here it is:

public class Parallel

{

/// <summary>

/// Parallel for loop. Invokes given action, passing arguments

/// fromInclusive - toExclusive on multiple threads.

/// Returns when loop finished.

/// </summary>

public static void For(int fromInclusive, int toExclusive, Action action)

{

// chunkSize = 1 makes items to be processed in order.

// Bigger chunk size should reduce lock waiting time and thus

// increase paralelism.

int chunkSize = 4;

// number of process() threads

int threadCount = Environment.ProcessorCount;

int index = fromInclusive - chunkSize;
// locker object shared by all the process() delegates
var locker = new object();

// processing function

// takes next chunk and processes it using action

var process = delegate()

{

while (true)

{

int chunkStart = 0;

lock (locker)

{

// take next chunk

index += chunkSize;

chunkStart = index;

}

// process the chunk

// (another thread is processing another chunk
// so the real order of items will be out-of-order)

for (int i = chunkStart; i < chunkStart + chunkSize; i++)

{

if (i >= toExclusive) return;

action(i);

}

};

// launch process() threads

IAsyncResult[] asyncResults = new IAsyncResult[threadCount];

for (int i = 0; i < threadCount; ++i)

{

asyncResults[i] = process.BeginInvoke(null, null);

}

// wait for all threads to complete

for (int i = 0; i < threadCount; ++i)

{

process.EndInvoke(asyncResults[i]);

}

As noted in the code, by setting chunkSize to 1 we can make the items be processed in order. With bigger chunk size, items can be processed in mixed non-deterministic order, which is ok in some application, like when you want to modify all items of a collection or render lines of a picture.
Bigger chunkSize should reduce lock waiting time and thus increase overall speed. But too big chunkSize is bad too, because if the work is split into only a few big parts, there is not enough parallelism exposed - we could find ourselves waiting for the last single thread to finish its large chunk.

Using Parallel.For is simple:

Parallel.For(0, 1000, delegate(int i)

{

// your parallel code

Thread.Sleep(100);

Console.WriteLine(i);

});

Serialize object graph to XML in .Net

2008-03-14T14:59:00.000-07:00

How to serialize any data structure to XML? My first idea was XmlSerializer. But then I found out it had some serious drawbacks. Luckily, there is a better option - NetDataContractSerializer.

In .Net, there are a few classes for (de)serialization. This is an overview of their features:

XmlSerializer

Cannot serialize circular references.
If more objects point to the same object, its copies are created in the xml for each of these references.
Has to know all types that could be encountered during serialization in advance - throws an exception on unknown type. Known types are passed in the constructor to XmlSerializer or marked by XmlIncludeAttribute.
Generates simple readable xml.

DataContractSerializer

Has to know types in advance - like XmlSerializer.
Can serialize circular references - construct with preserveObjectReferences = true
Used by WCF.

NetDataContractSerializer - better!

Serializes object graph properly including circular references.
Doesn't have to know types in advance.
However, MSDN states that it can be only used in .Net <-> .Net communication, which is ok also for storing object in a file.
Generates simple readable xml.

BinarryFormatter

Works well, like NetDataContractSerializer.
Disadvantage is that it serializes to binary format, which make its unusable e.g. for saving to a file that user could later edit.
The output is smallest thanks to the binary format.

SoapFormatter

Deprecated. Cannot serialize generic collections

- All serializers need the type to be serialized marked by SerializableAttribute.

What does it mean that XmlSerializer has to know all types that could be encountered during serialization in advance?
Imagine that we have two classes: Base and Derived.

[Serializable]
    public class Base
    {
        public string name;
        public Base()
        {
            name = "base instance";
        }
    }
 
    [Serializable]
    public class Derived : Base
    {
        public Derived left;
        public Derived right;
 
        public Derived()
        {
        }
    }

What if we have a reference to Base and we actually don't want to care about the actual type?

Base b = new Derived();
// we only know we are holding reference to Base
// and we don't want to care about the actual type
XmlSerializer ser = new XmlSerializer(typeof(Base));
// serialize
using (FileStream fs = File.Create(AppDomain.CurrentDomain.BaseDirectory + "data.xml"))
{
    // XmlSerializer throws an Exception
    ser.Serialize(fs, b);
}
// deserialize
using (FileStream fs = File.OpenRead(AppDomain.CurrentDomain.BaseDirectory + "data.xml"))
{
    Base baseDeserialized = (Base)ser.Deserialize(fs);     Derived deserialize = baseDeserialized as Derived;
}

XmlSerializer throws an exception, because it encounters an "unknown" type - Derived. We could solve this by passing all the possible derived types in constructor of XmlSerializer or tagging all by XmlIncludeAttribute. This is of course inconvenient if you have a lot of classes. The worst thing is that when you add a derived class, you have to change code elsewhere.
NetDataContractSerializer doesn't have this problem.

The second issue with XmlSerializer is that it cannot serialize complex object graph. What does it mean "to serialize object graph"?

Derived top = new Derived();
 
top.left = new Derived();
top.left.name = "left son";
top.right = new Derived();
top.right.name = "right son"; 
 
top.left.right = new Derived();
//         top
//       /     \
//  left       right
//       \     /
//        bottom
top.right.left = top.left.right;
 
XmlSerializer ser = new XmlSerializer(typeof(Derived));
using (FileStream fs = File.Create(AppDomain.CurrentDomain.BaseDirectory + "data.xml"))
{
    ser.Serialize(fs, top);
}
using (FileStream fs = File.OpenRead(AppDomain.CurrentDomain.BaseDirectory + "data.xml"))
{
    Derived deserialized = (Derived)ser.Deserialize(fs);
    // false - we want true
    bool ok = deserialized.left.right == deserialized.right.left;
}

After deserialization, deserialized.left.right == deserialized.right.left is false, that means the object graph is different. Worse - XmlSerializer cannot serialize circular references at all.
Again, NetDataContractSerializer doesn't have any of these problems.