How to implement a rule engine in C#


Recently there was an interesting question on StackOverflow about creating a rule engine in C#.

Say we have a collection of Users (with Name, Age, ...) and rule definitions like this:

static List<Rule> rules = new List<Rule> {
     new Rule ("Age""GreaterThan""20"),
     new Rule ( "Name""Equal""John"),
   new Rule ( "Tags""Contains""C#" )
};

and we want to be able to evaluate the rules:

// Returns true if User satisfies given rule (e.g. 'user.Age > 20')
bool Matches(User user, Rule rule)
{
    // how to implement this?
}
 
One  obvious solution is to use reflection.
Here we will show a solution which uses Expression trees to
compile the rules into fast executable delegates. We can then evaluate the rules as if they were normal boolean functions.

public static Func<T, bool> CompileRule<T>(Rule r)
{
    var paramUser = Expression.Parameter(typeof(User));
    Expression expr = BuildExpr<T>(r, paramUser);
    // build a lambda function User->bool and compile it
    return Expression.Lambda<Func<T, bool>>(expr, paramUser).Compile();
}
 
static Expression BuildExpr<T>(Rule r, ParameterExpression param)
{
    var left = MemberExpression.Property(param, r.MemberName);
    var tProp = typeof(T).GetProperty(r.MemberName).PropertyType;
    ExpressionType tBinary;
    // is the operator a known .NET operator?
    if (ExpressionType.TryParse(r.Operator, out tBinary))    {
        var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));
        // use a binary operation, e.g. 'Equal' -> 'u.Age == 15'
        return Expression.MakeBinary(tBinary, left, right);
    } else {
        var method = tProp.GetMethod(r.Operator);
        var tParam = method.GetParameters()[0].ParameterType;
        var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tParam));
        // use a method call, e.g. 'Contains' -> 'u.Tags.Contains(some_tag)'
        return Expression.Call(left, method, right);
    }
}
 
Now we can implement the rule validation by compling the rules and then simply invoking them:

var rule = new Rule ("Age""GreaterThan""20");
Func<Userbool> compiledRule = CompileRule<User>(rule);
 
// true if someUser.Age > 20
bool isMatch = compiledRule(someUser);

Of course we compile all the rules just once and then use the compiled delegates:

// Compile all the rules once.
var compiledRules = rules.Select(r => CompileRule<User>(r)).ToList();
 
// Returns true if user satisfies all rules.
public bool MatchesAllRules(User user)
{
    return compiledRules.All(rule => rule(user));
}
 
Note that the “compiler” is so simple because we are using 'GreaterThan' in the rule definition, and 'GreaterThan' is a known .NET name for the operator, so the string can be directly parsed. The same goes for 'Contains' – it is a method of List. If we need custom names we can build a very simple dictionary that just translates all operators before compiling the rules:

Dictionary<stringstring> nameMap = new Dictionary<stringstring> {
"greater_than""GreaterThan" },
"hasAtLeastOne""Contains" }
};

That’s it. As an improvement we could add error messages to the rule definitions and print the error messages for the unsatisfied rules.

If you found this useful, feel free to rate the answer on StackOverflow.

Posted by Martin Konicek on 1:10 PM

64 comments:

Ricardo Rodrigues said...

Hey, great stuff!
I never really gotten into Expression Trees but it seems that I must, since they're so powerful.
I'd suggest using an Enum for the operator List rather than a Dictionary, with the obvious commitment of not having a dynamic List of operators, if that is a requirement, the Dictionary is the correct and obvious choice :)

Anonymous said...

Hi, why in the expression "ExpressionType.TryParse" tells me that cannot resolve symbol tryparse?
It seems like ExpressionType has no TryParse method...and it should nat because ExpressionType is an enum type, isn't it?

Inv said...

Enum.TryParse was added in .NET 4. On older .NET versions, use Enum.Parse: http://msdn.microsoft.com/en-us/library/essfb559.aspx

Hamza Essounni said...

Hi Martin,
thanks for this great article.
Currently I'm developping a tool that take as input a list of rules (in db),and based on the result of tthe rule(if it's true) it takes an action et optionally raise a warning/error/message.
An action can lead to an another rule.
The answer for the first part of my question is already in this article.
Still,I would like to know how to take Action based on the result of a rule and how to optionally raise event (Alert/Warning/Message) ?
Thank you.

Inv said...
This comment has been removed by the author.
Inv said...

Hi Hamza,

with each rule in the database, you could store whether it should produce an Alert/Warning/Message when it matches. Then your rule object wouldn't be just a simple function, but something like:

class Rule {
  Func<User, boolean> Matches { get; set; }
  Level Level { get; set; } // Level is an enum
  String Message { get; set; }
}

Then you can do

if (r.Matches(user)) {
  if (r.Level == Level.Error) {
    Console.WriteLine(r.Message);
  }
}

As for the second part about matching rules leading to additional rules: a simple solution would be to also give each rule a Name and store it in the database. Also with every rule, you would store a list of rule names that should be also evaluated when the rule matches.

class Rule {
  String Name { get; set; }
  List<String> AdditionalRules { get; set; }
  Func<User, boolean> Match { get; set; }
  Level Level { get; set; } // Level is an enum
  String Message { get; set; }
}

Just be careful about cyclic references between rules, so that evaluating a rule does not lead to a stack overflow.

Also note that all this work of storing rules in the database is only useful if you actually want your users to define the rules. If the rules are fixed, then you are better off encoding them in your code yourself.

Hamza Essounni said...

Hi Martin,Thank you very much for the great answer !
Unfortunately life is not so easy :) :

If the PropertyMember is a list of item (say generic list of options L = List)
IndexOption has an enum property.
I we want to check that an IndexOption exist and is equal to a targetvalue (ie:check that ONLINE option exist and is equal to OFF),Could you figure out how to do this ?
Thanks

Inv said...

Well how would the code look if you did it manually? Did you manage to solve it yet?

Anonymous said...

Hi Martin,

Can I ask if it would be possible, if the set of rules would need to be evaluated with AND/OR operators defined in the rule to determine if the set of rules match a user?
So like:

Rule ("Age", "GreaterThan", "20", "OR")
Rule ("Age", "GreaterThan", "20", "AND")
Rule ( "Tags", "Contains", "C#", "N/A" )

Would that be possible too with Expression trees?

With compiledRules.All(rule => rule(user)); it evaluates all the rules with AND operator in between

Thanks,
Andras

Inv said...

Hi, evaluating the rules using AND/OR is possible. Once you have the list of compiled rules, you can do (rules[1] && (rules[2] || rules[3])) for example.

Note that AND/OR are binary operators though, so by just putting them on the rules themselves, it is not clear how the rules should be combined together:

(rule1 && rule2 && ... ruleN) && (rule1 || rule2 || ... ruleM)
or:
(rule1 && rule2 && ... ruleN) || (rule1 || rule2 || ... ruleM)

If you want to do that, just split the list into 3 lists (AND, OR, N/A) and handle each of the lists differently. You can use orRules.Any(rule => rule(user)) instead of All to get OR semantics.

I think it makes more sense to be able define how rules combine into composite rules, such as:

"longTimeCustomer", "YearsSinceJoin GreaterThan 3"
"frequentUser", "FrequentUser Equals True"
"popularUser", "FacebookFriendsCount GreaterThan 300"

"offerDiscount = (longTimeCustomer AND (frequentUser OR popularUser))"

Jacques said...
This comment has been removed by the author.
Jacques said...

Interesting! i've been pulling my hair out the last two weeks trying to find a way to implement rule engines without having to get a PhD in design patterns. Thanks for the article.

I am, however, having problems compiling the solution, the following error keeps popping up:

System.Workflow.Activities.Rules.Rule' does not contain a definition for 'MemberName' and no extension method 'MemberName' accepting a first argument of type 'System.Workflow.Activities.Rules.Rule'

and

'System.Workflow.Activities.Rules.Rule' does not contain a definition for 'Operator' and no extension method 'Operator' accepting a first argument of type 'System.Workflow.Activities.Rules.Rule' could be found (are you missing a using directive or an assembly reference?)


Also It seems that There is no Rule Constructor with only three arguments available

new Rule ("Age", "GreaterThan", "20"); --- > compile error

I'm using 4.0.and these are my using directives:

using System.Workflow.ComponentModel.Compiler;
using System.Workflow.ComponentModel.Serialization;
using System.Workflow.ComponentModel;
using System.Workflow.ComponentModel.Design;
using System.Workflow.Runtime;
using System.Workflow.Activities;
using System.Workflow.Activities.Rules;
using System.Linq.Expressions;

What am I missing? Any help would be appreciated.
Thank you

Inv said...

Hi Jacques,

this post is quite a simple example unrelated to the Workflow foundation. I'm not very familiar with Workflow foundation but a google search for "workflow foundation rule engine" gave me some interesting results.

Martin

Jacques said...

Noted! Thanks for the prompt reply Martin.

Lindsay said...

Hi Martin Konicek, Could you please assist. I'm interest in understanding the above code as I'm interested in building a rule engine. I've used the above code but when I execute my test solution, I get an error in the "BuildExpression" function. See error message "The value passed in must be an enum base or an underlying type for an enum, such as an Int32. Parameter name: value". Am I missing something here. Thanks for you help. Lindsay

Inv said...

Hi Lindsay, I will attach a working solution today. Note that this is a very basic implementation of a rule engine though.

Lindsay said...

Hi Martin, thank for responding. Much appreciated. The basic implementation is in order. I just need to get a feel for the new technology as I spend most of my days coding in sql and I'm looking to implement rule based application. Regards Lindsay

Lindsay said...

Hi Martin, have you attached a working solution of the basic rule engine? Where you I find it. Thanks

offline said...

Hi Martin,
you said that -
'Contains' can be directly parsed because it is a method of List
But ExpressionType.TryParse( return null for it?

can you please explain ?

offline said...

i meant ExpressionType.TryParse return False (not null)

Nasir Amin said...

Hi Martin,

Great post. It really helped me getting started on the Expression Trees.

Would you kindly advise on how to make the above code work if we replaced the type 'User' with a DataRow. I tried but it says that 'Age' is not a member of the DataRow.

Is there something really basic I am missing?

Regards,

Nas

Inv said...

Hi,

can you please provide a larger piece code showing where TryParse is returning null?

Inv said...

Hi Lindsay, I didn't get to it sorry. The code posted here should compile on .NET 4.

Inv said...

Hi Nas,

Thanks!
You would have to replace the line that use Type.GetProperty with a lookup of a Datarow column based on its name,

Inv said...

Nas,

you would also have to change the part that builds the 'left' expression. Instead of u.Age == 25 you would have sth like dataRow['Age'] == 25. Note that indexers are called 'Item'

Nasir Amin said...

Thanks Martin for the prompt reply.

How would I get the DataRow object from param expression? or do I have to pass it on differently. Sorry I am new to the Expression Trees.

Inv said...

Hi Nas,

the param expression is essentially a parameter representing your DataRow. Note that the code doesn't work with a DataRow directly, it's building an Expression that will do 'dataRow'["Age"] == 15.

You will need to change the line
var left = MemberExpression.Property(param, r.MemberName);

to something like:
var left = MemberExpression.Property(param, "Item", Expression.Constant(r.MemberName));

which is essentially the 'dataRow'["Age"].

This is definitely a good exercise for the Expression trees API. However, since DataRows are already dynamic (the compiled code will still do column lookups by name at runtime), I was thinking maybe expression trees won't give you that much of a performance benefit in this case. You would have to measure of course. You would still get the speedup of the '== 25' or '.Contains("foo")' being done without any IFs or reflection at runtime, but still.. a few switch statements plus some reflection might be easier. The expression trees are purely a performance optimization.

Tell me how it went or if you need more advice just ask.

Nasir Amin said...

Thanks very much Martin. I would try this and let you know. I am basically trying to design a Rule Engine where the user can define rules on a .net dataset and tables within it. Basically the user would see columns of a table and decide what 'rules' they want to apply. For Example, IsNull, IsNumeric, IsISOFormat, IsInList, Equals, GreaterThan etc.

I want to build expression trees for the rules and then store them in a database as a template. Then I have service that will read this expression from the database and apply it to the data the service is currently processing.

Is this the right approach or do you know of a better and easier approach. As you can see I won't be just using the .Net's operators but would be calling functions/delegates for my specialised operators. For example, IsInList would contain a name of the list that would be somewhere in the database. When a column value is to be checked In a list using IsInList - it will essentially call a function to check that the value in the column exists in the named list passed as IsInList operator.
Then I would have more complicated rules. For example checking Value in table1.column1 exists in table2.Column3 and Is not null and IsInList etc.

I thought of using CodeDom but it seemed really complicated. I also had a look at the Windows Workflow Foundation's Rules but they also seemed really cumbersome for what I am trying to do.


I read somewhere that the expression trees are very powerful and usually more optimized than other technique.

If you think this is not the right approach then I would really appreciate if you could direct me to better alternatives.

Regards,

Nas

Nasir Amin said...

Martin,

I tried what you said but I am stuck with getting the type of the column.
The line:
var tProp = typeof(T).GetProperty(r.MemberName).PropertyType;

Because the above statement for r.MemberName would return nothing for a datarow so accessing PropertyType at the end throws Object reference not set exception.

I tried getting the data table and col through the expression but with no success.
I have done the following so far (VB.net I hope you don't mind):
Dim left = MemberExpression.Property(param, "Item", expression.Constant(r.FieldName))
Dim Table = MemberExpression.Property(param, "Table")
Dim Cols = MemberExpression.Property(Table, "Columns")
Dim Col = MemberExpression.Property(Cols, "Item", Expression.Constant(r.FieldName))

Dim DataType = MemberExpression.Property(Col, "DataType")

But the above does not give me the underlaying data type of the DataColumn. It seems like this should be something really simple but I can't find the right property/indexer to call.

Regards,

Nas

Ivan Duric said...

Hi Martin,
is there a way to see if the Expression with specified operator (ExpressionType) is valid or do i have to implement it by my self. For example GreaterThan with two string parameters throws exception when expression is compiled.
Ivan

Inv said...

Hi Nas,

sorry for replying late this time. Could you please post a piece of VB code that does what you want without using Expression trees and then we can try to rewrite it to an expression tree?

Inv said...

Hi Ivan,

I think that's because in C# you can't write 's1 > s2'. which is what GreaterThan does. Try creating an expression that uses String.CompareTo or String.Compare.

The list of all ExpressionTypes is here: http://msdn.microsoft.com/en-us/library/bb361179.aspx

VB said...

Hi Martin,

It is really great post. I am able to get simple rules working. I need some help with AND/OR operators.

Actually I need solution where rules definition can be read from a XML/text file.

How can I create one rule with multiple conditions with this configuration requirement? Because when these rules are defined in a text file, that these simple rules are not created yet.

Any help/code would be highly appreciated.

LillianMorris Blog said...

Hello! I am excited to find out one thing, of course if I'm not asking too much could you please tell us the place where you spent your childhood?

VB said...

I have figured out the complex rule issue by forming nested And/Or Data structure.

But I am facing issues with Nested class.

e.g.

public class User
{
public int Age { get; set; }
public string Name { get; set; }
public string Tags { get; set; }
NestedUser NestedUser { get; set; }

public User(int age, string name, string tags)
{
Age = age;
Name = name;
Tags = tags;
NestedUser = new NestedUser();
NestedUser.Count = 10;

}
}

public class NestedUser
{
public int Count { get; set; }
}

How do I get this code Rule working
"NestedUser.Count", "GreaterThan", "5"

Inv said...

Hi VB,

that's pretty cool you figured out how to store AND / OR rules - you probably represent them using an AST, right? Also, an AST is easy to serialize to XML, since it is a tree.

To create an expression for 'NestedUser.Count', have a look at the Expression.Property method: http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.property.aspx

There is even a PropertyOrField method.

Anonymous said...

Hi VB,

How did you manage And/Or operator?

Thanks,
Bob

VB said...

I have Created my own classes for And and OR Rules, which can contain List of rules, and each Rule is an Expression.

I AND or OR these rules based on type of the class.

I have also solved the nested properties issue using PropertyInfo Class.

But now I have came across another hurdle.

These Objects could be collection themselves or they could be Properties which could be collection.

Not sure how to resolve these issues.



Ram Sharma said...

Hi Martin,

When i am passing GreaterThan then i am getting error as below
"The binary operator GreaterThan is not defined for the types 'System.String' and 'System.String'."

Please help me to rsolve

cinuser said...

Very interesting. It's possible to have your full source code solution? I would be very thankful.

Bey Melamed said...
This comment has been removed by the author.
Bey Melamed said...

Great article thanks.
Not being an experienced ("sharp") C#er, I wonder if the comment / suggestion below is valid.
Seems to me that by compiling the rule the operations and values (e.g. u.Age, "Equals", "20") get 'baked-in'.

If so, the technique could be much more powerful if these could be parameterized - if I understand the technique correctly - at the price of having to construct / compile the rule upon each invocation.

Any feedback or suggestion for implementing such idea is appreciated - even if my question / comment reveal total ignorance.

Thanks for the clean design / thought process.

Am looking for the posted working code sample (my interest lies mainly in using this model for implementation of a QA Automation tool - the object(s) against which the rules will be invoked will be mainly UITestControl instances.)

Martin Konicek said...

VB,

sorry for responding late.
To support nested properties / collections you would need something like

new Rule ("Parent.Age", "GreaterThan", "40")
new Rule ("Parents[*].Age", "Contains", "40")

and adapt BuildExpr to support this syntax. It's not trivial but definitely doable.

You might also want to support rules with paths such as "Parents[*].Parents[*].Age", but there it's not clear what the semantics should be - is it a single collection of integers, or a collection of collections of integers, and how do you match on the inner collections?

I would probably start with supporting just

"Parent.Age"
and
"Parent.Tags"

and add more features step by step if needed.

Martin Konicek said...

Bey,

also sorry for replying late.

Parameterized rules are an excellent idea!
You could declare rules like:

val rule = new ParameterizedRule("Age", Operators.GreaterThan);

and then change CompileRule to build a function that takes one additional parameter besides the object itself:

Func compiledRule = CompileRule(rule);

Essentially, you would create one additional Expression.Parameter and use it in BuildExpr instead of Expression.Constant.

Let me know if this helped. Thanks a lot!

John Harry said...

Hi,
I am really struggling to get nested properties to work. I have looked at the Expression.Property method: http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.property.aspx but still can't get it to work. Any help would be much appreciated (such as a code snippet?).

Martin Konicek said...

Hi John,

by nested properties, do you mean e.g. foo.Bar.Baz?
If you have an expression "e" representing "foo", can you do the following?

Expression.Property(Expression.Property(e, "Bar"), "Baz")

John Harry said...
This comment has been removed by the author.
John Harry said...

Hi Martin,
Thanks for the reply, I did exactly that. In fact I Split the r.MemberName on the "." and loop through building the left Expression recursively. i.e. left = MemberExpression.Property(left, leftOperands[i]);

However, I am having trouble with lists, and/or indexed lists.

e.g. new Rule ("Collection[0].Code", "Equal", "C137.2")
Or
new Rule ("Collection[*].Code", "Contains", "C137")

Would appreciate a pointer on how to handle such a scenario.

Martin Konicek said...

Hi John,

to create an expression for "Collection[0].Code" you'll have to extend your parser that splits the string by dots to also look for integers inside square brackets. The left expression of the "Code" MemberExpression will be an IndexExpression which you can create e.g. using one of the overloads of Expression.Property: http://msdn.microsoft.com/en-us/library/dd630266.aspx

I found this by simply full-text searching the Expression API doc for "index":
http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression_methods.aspx

"Collection[*].Code" is more complicated. You could translate this into an expression tree equivalent to Collection.Any(item => item.Code.Contains("C137")).

Note that if you find out your parser of string containing C# expressions is getting complex, you can check out the open-source NRefactory library that contains a C# parser:
http://www.codeproject.com/Articles/408663/Using-NRefactory-for-analyzing-Csharp-code

Sankalp said...

Hi Martin,

Great Article, I am planning to use Expressions in one of the projects that I am working on. I have a situation where the user will create a set of rules dynamically and I have to store them in the database. They could be such as [rule1 && (rule2 || rule3)] combunation of AND and OR. Noticed that there are solutions to have seperate and list and or list but what is the best way to approach if I have to persist the expression above.

Many Thanks
Sankalp

leMoussel said...

Is it possible to build a "between" rule to check if a value is included in a range of 2 values ?
eg : "Age", "Between", "20", "40" -> 20 < Age < 40
eg : "Age", "BetweenOrEqual", "20", "40" ->> 20 <= Age <= 40

leMoussel said...

To have to have a generic Rule compiler for any types of objects, I changed the line
var paramUser = Expression.Parameter(typeof(User));
To
var paramUser = Expression.Parameter(typeof(T));

Anonymous said...

How would I go about handling nullable values? I have a property that is a nullable datetime and I get an error when the MakeBinary is called :

The binary operator GreaterThan is not defined for the types 'System.Nullable`1[System.DateTime]' and 'System.DateTime'.

Is there a way to change the type of the Expression.Property in cases like this?

Basel said...

Hi Maritn,
its great article and i used this concept to build custom rule engine, and i want to use new method which is not available in Binary Expressions, similar to Contains, Say StartsWith. but i couldn't find it in the String type, so i tried to use extensions it didn't work though, any idea how to use custom method to work with system types?

cricket betting tips free said...

very useful post...thanks for sharing

Jorge Luis Pérez Medina said...

Hi Martin,

Great document. I posted a case in stactoverflow about rule engine (http://stackoverflow.com/questions/33500953/how-evaluate-an-object-in-a-collection-with-microruleengine). Basically i need testing how access collections. Have you an idea of it? Thanks

Pawan agrawal said...

Hi Martin,

Great artical, I was looking for creating some code samples to create my own business rule engine and this one is really good any easy to understand.

I am not an expert of expression tree, could you please let me know if I can use this solution for creating the rule engine which can process thousands of rule per minute. I want to from the performance perspective using Expression tree based solution will be good over the other available solution like NRules.

Ferrans Nogués said...

Hi Martin. I have lots of these type of conditions stored in the database:

StartReal.IsEmpty() && EstimatedStart.Date < DateTime.Today.Date
StartReal.IsEmpty() && EstimatedStart.Date > DateTime.Today.Date
StartReal.IsEmpty() && EstimatedStart.Date == DateTime.Today.Date

(Note: IsEmpty() is an extension method to check if a Date is null)

Right now, I'm parsing and splitting each one of the conditions and checking manually to the object properties (is a Task class). Is there any way to do it with a expression builder?
I'm asking, because I've tried to adapt your BuildExpr, but the lack of knowledge is not driving me anywhere.
Thanks in advance. Saludos desde Uruguay

Martin Konicek said...

Ferrans Nogués doing it manually is fine and probably performant enough. This blog post is only useful if performance is really critical, should have stated it more clearly.

nekrajB1 said...

We provide the hastmethun tips and ilaaj in hindi in worlds .Find us here

nekrajB1 said...

We provide the best free cricket betting tips in worlds .Find us here
We provide the best free cricket betting tips in worlds .Find us today cricket match predction You need not go anywhere for other cbtf tips as we provide best cricket betting tips

sss said...

Hi Guys,
Can these rules be created on the fly? An admin would create these validtion/rules

Free Cricket Betting Tips said...

its great article....thanks for sharing.
For free and 100% Guaranteed cricket betting tips visit the Website, remember you can earn by trading not by betting.

Post a Comment