How to implement a rule engine in C#
Recently there was an interesting question on StackOverflow about creating a rule engine in C#.
Say we have a collection of Users (with Name, Age, ...) and rule definitions like this:
static List<Rule> rules = new List<Rule> {
new Rule ("Age", "GreaterThan", "20"),
new Rule ("Age", "GreaterThan", "20"),
new Rule ( "Name", "Equal", "John"),
new Rule ( "Tags", "Contains", "C#" )
};
new Rule ( "Tags", "Contains", "C#" )
};
and we want to be able to evaluate the rules:
// Returns true if User satisfies given rule (e.g. 'user.Age > 20')
bool Matches(User user, Rule rule)
{
// how to implement this?
}
One obvious solution is to use reflection.
Here we will show a solution which uses Expression trees to
compile the rules into fast executable delegates. We can then evaluate the rules as if they were normal boolean functions.
Here we will show a solution which uses Expression trees to
compile the rules into fast executable delegates. We can then evaluate the rules as if they were normal boolean functions.
public static Func<T, bool> CompileRule<T>(Rule r)
{
var paramUser = Expression.Parameter(typeof(User));
Expression expr = BuildExpr<T>(r, paramUser);
// build a lambda function User->bool and compile it
return Expression.Lambda<Func<T, bool>>(expr, paramUser).Compile();
}
static Expression BuildExpr<T>(Rule r, ParameterExpression param)
{
var left = MemberExpression.Property(param, r.MemberName);
var tProp = typeof(T).GetProperty(r.MemberName).PropertyType;
ExpressionType tBinary;
// is the operator a known .NET operator?
if (ExpressionType.TryParse(r.Operator, out tBinary)) {
var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));
// use a binary operation, e.g. 'Equal' -> 'u.Age == 15'
return Expression.MakeBinary(tBinary, left, right);
} else {
var method = tProp.GetMethod(r.Operator);
var tParam = method.GetParameters()[0].ParameterType;
var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tParam));
// use a method call, e.g. 'Contains' -> 'u.Tags.Contains(some_tag)'
return Expression.Call(left, method, right);
}
}
Now we can implement the rule validation by compling the rules and then simply invoking them:
var rule = new Rule ("Age", "GreaterThan", "20");
Func<User, bool> compiledRule = CompileRule<User>(rule);
// true if someUser.Age > 20
bool isMatch = compiledRule(someUser);
Of course we compile all the rules just once and then use the compiled delegates:
// Compile all the rules once.
var compiledRules = rules.Select(r => CompileRule<User>(r)).ToList();
// Returns true if user satisfies all rules.
public bool MatchesAllRules(User user)
{
return compiledRules.All(rule => rule(user));
}
Note that the “compiler” is so simple because we are using 'GreaterThan' in the rule definition, and 'GreaterThan' is a known .NET name for the operator, so the string can be directly parsed. The same goes for 'Contains' – it is a method of List. If we need custom names we can build a very simple dictionary that just translates all operators before compiling the rules:
Dictionary<string, string> nameMap = new Dictionary<string, string> {
{ "greater_than", "GreaterThan" },
{ "hasAtLeastOne", "Contains" }
};
That’s it. As an improvement we could add error messages to the rule definitions and print the error messages for the unsatisfied rules.
If you found this useful, feel free to rate the answer on StackOverflow.
Posted by Martin Konicek
on 1:10 PM
73 comments:
Hey, great stuff!
I never really gotten into Expression Trees but it seems that I must, since they're so powerful.
I'd suggest using an Enum for the operator List rather than a Dictionary, with the obvious commitment of not having a dynamic List of operators, if that is a requirement, the Dictionary is the correct and obvious choice :)
Hi, why in the expression "ExpressionType.TryParse" tells me that cannot resolve symbol tryparse?
It seems like ExpressionType has no TryParse method...and it should nat because ExpressionType is an enum type, isn't it?
Great code!
Enum.TryParse was added in .NET 4. On older .NET versions, use Enum.Parse: http://msdn.microsoft.com/en-us/library/essfb559.aspx
Hi Martin,
thanks for this great article.
Currently I'm developping a tool that take as input a list of rules (in db),and based on the result of tthe rule(if it's true) it takes an action et optionally raise a warning/error/message.
An action can lead to an another rule.
The answer for the first part of my question is already in this article.
Still,I would like to know how to take Action based on the result of a rule and how to optionally raise event (Alert/Warning/Message) ?
Thank you.
Hi Hamza,
with each rule in the database, you could store whether it should produce an Alert/Warning/Message when it matches. Then your rule object wouldn't be just a simple function, but something like:
class Rule {
Func<User, boolean> Matches { get; set; }
Level Level { get; set; } // Level is an enum
String Message { get; set; }
}
Then you can do
if (r.Matches(user)) {
if (r.Level == Level.Error) {
Console.WriteLine(r.Message);
}
}
As for the second part about matching rules leading to additional rules: a simple solution would be to also give each rule a Name and store it in the database. Also with every rule, you would store a list of rule names that should be also evaluated when the rule matches.
class Rule {
String Name { get; set; }
List<String> AdditionalRules { get; set; }
Func<User, boolean> Match { get; set; }
Level Level { get; set; } // Level is an enum
String Message { get; set; }
}
Just be careful about cyclic references between rules, so that evaluating a rule does not lead to a stack overflow.
Also note that all this work of storing rules in the database is only useful if you actually want your users to define the rules. If the rules are fixed, then you are better off encoding them in your code yourself.
Hi Martin,Thank you very much for the great answer !
Unfortunately life is not so easy :) :
If the PropertyMember is a list of item (say generic list of options L = List)
IndexOption has an enum property.
I we want to check that an IndexOption exist and is equal to a targetvalue (ie:check that ONLINE option exist and is equal to OFF),Could you figure out how to do this ?
Thanks
Well how would the code look if you did it manually? Did you manage to solve it yet?
Hi Martin,
Can I ask if it would be possible, if the set of rules would need to be evaluated with AND/OR operators defined in the rule to determine if the set of rules match a user?
So like:
Rule ("Age", "GreaterThan", "20", "OR")
Rule ("Age", "GreaterThan", "20", "AND")
Rule ( "Tags", "Contains", "C#", "N/A" )
Would that be possible too with Expression trees?
With compiledRules.All(rule => rule(user)); it evaluates all the rules with AND operator in between
Thanks,
Andras
Hi, evaluating the rules using AND/OR is possible. Once you have the list of compiled rules, you can do (rules[1] && (rules[2] || rules[3])) for example.
Note that AND/OR are binary operators though, so by just putting them on the rules themselves, it is not clear how the rules should be combined together:
(rule1 && rule2 && ... ruleN) && (rule1 || rule2 || ... ruleM)
or:
(rule1 && rule2 && ... ruleN) || (rule1 || rule2 || ... ruleM)
If you want to do that, just split the list into 3 lists (AND, OR, N/A) and handle each of the lists differently. You can use orRules.Any(rule => rule(user)) instead of All to get OR semantics.
I think it makes more sense to be able define how rules combine into composite rules, such as:
"longTimeCustomer", "YearsSinceJoin GreaterThan 3"
"frequentUser", "FrequentUser Equals True"
"popularUser", "FacebookFriendsCount GreaterThan 300"
"offerDiscount = (longTimeCustomer AND (frequentUser OR popularUser))"
Interesting! i've been pulling my hair out the last two weeks trying to find a way to implement rule engines without having to get a PhD in design patterns. Thanks for the article.
I am, however, having problems compiling the solution, the following error keeps popping up:
System.Workflow.Activities.Rules.Rule' does not contain a definition for 'MemberName' and no extension method 'MemberName' accepting a first argument of type 'System.Workflow.Activities.Rules.Rule'
and
'System.Workflow.Activities.Rules.Rule' does not contain a definition for 'Operator' and no extension method 'Operator' accepting a first argument of type 'System.Workflow.Activities.Rules.Rule' could be found (are you missing a using directive or an assembly reference?)
Also It seems that There is no Rule Constructor with only three arguments available
new Rule ("Age", "GreaterThan", "20"); --- > compile error
I'm using 4.0.and these are my using directives:
using System.Workflow.ComponentModel.Compiler;
using System.Workflow.ComponentModel.Serialization;
using System.Workflow.ComponentModel;
using System.Workflow.ComponentModel.Design;
using System.Workflow.Runtime;
using System.Workflow.Activities;
using System.Workflow.Activities.Rules;
using System.Linq.Expressions;
What am I missing? Any help would be appreciated.
Thank you
Hi Jacques,
this post is quite a simple example unrelated to the Workflow foundation. I'm not very familiar with Workflow foundation but a google search for "workflow foundation rule engine" gave me some interesting results.
Martin
Noted! Thanks for the prompt reply Martin.
Hi Martin Konicek, Could you please assist. I'm interest in understanding the above code as I'm interested in building a rule engine. I've used the above code but when I execute my test solution, I get an error in the "BuildExpression" function. See error message "The value passed in must be an enum base or an underlying type for an enum, such as an Int32. Parameter name: value". Am I missing something here. Thanks for you help. Lindsay
Hi Lindsay, I will attach a working solution today. Note that this is a very basic implementation of a rule engine though.
Hi Martin, thank for responding. Much appreciated. The basic implementation is in order. I just need to get a feel for the new technology as I spend most of my days coding in sql and I'm looking to implement rule based application. Regards Lindsay
Hi Martin, have you attached a working solution of the basic rule engine? Where you I find it. Thanks
Hi Martin,
you said that -
'Contains' can be directly parsed because it is a method of List
But ExpressionType.TryParse( return null for it?
can you please explain ?
i meant ExpressionType.TryParse return False (not null)
Hi Martin,
Great post. It really helped me getting started on the Expression Trees.
Would you kindly advise on how to make the above code work if we replaced the type 'User' with a DataRow. I tried but it says that 'Age' is not a member of the DataRow.
Is there something really basic I am missing?
Regards,
Nas
Hi,
can you please provide a larger piece code showing where TryParse is returning null?
Hi Lindsay, I didn't get to it sorry. The code posted here should compile on .NET 4.
Hi Nas,
Thanks!
You would have to replace the line that use Type.GetProperty with a lookup of a Datarow column based on its name,
Nas,
you would also have to change the part that builds the 'left' expression. Instead of u.Age == 25 you would have sth like dataRow['Age'] == 25. Note that indexers are called 'Item'
Thanks Martin for the prompt reply.
How would I get the DataRow object from param expression? or do I have to pass it on differently. Sorry I am new to the Expression Trees.
Hi Nas,
the param expression is essentially a parameter representing your DataRow. Note that the code doesn't work with a DataRow directly, it's building an Expression that will do 'dataRow'["Age"] == 15.
You will need to change the line
var left = MemberExpression.Property(param, r.MemberName);
to something like:
var left = MemberExpression.Property(param, "Item", Expression.Constant(r.MemberName));
which is essentially the 'dataRow'["Age"].
This is definitely a good exercise for the Expression trees API. However, since DataRows are already dynamic (the compiled code will still do column lookups by name at runtime), I was thinking maybe expression trees won't give you that much of a performance benefit in this case. You would have to measure of course. You would still get the speedup of the '== 25' or '.Contains("foo")' being done without any IFs or reflection at runtime, but still.. a few switch statements plus some reflection might be easier. The expression trees are purely a performance optimization.
Tell me how it went or if you need more advice just ask.
Thanks very much Martin. I would try this and let you know. I am basically trying to design a Rule Engine where the user can define rules on a .net dataset and tables within it. Basically the user would see columns of a table and decide what 'rules' they want to apply. For Example, IsNull, IsNumeric, IsISOFormat, IsInList, Equals, GreaterThan etc.
I want to build expression trees for the rules and then store them in a database as a template. Then I have service that will read this expression from the database and apply it to the data the service is currently processing.
Is this the right approach or do you know of a better and easier approach. As you can see I won't be just using the .Net's operators but would be calling functions/delegates for my specialised operators. For example, IsInList would contain a name of the list that would be somewhere in the database. When a column value is to be checked In a list using IsInList - it will essentially call a function to check that the value in the column exists in the named list passed as IsInList operator.
Then I would have more complicated rules. For example checking Value in table1.column1 exists in table2.Column3 and Is not null and IsInList etc.
I thought of using CodeDom but it seemed really complicated. I also had a look at the Windows Workflow Foundation's Rules but they also seemed really cumbersome for what I am trying to do.
I read somewhere that the expression trees are very powerful and usually more optimized than other technique.
If you think this is not the right approach then I would really appreciate if you could direct me to better alternatives.
Regards,
Nas
Martin,
I tried what you said but I am stuck with getting the type of the column.
The line:
var tProp = typeof(T).GetProperty(r.MemberName).PropertyType;
Because the above statement for r.MemberName would return nothing for a datarow so accessing PropertyType at the end throws Object reference not set exception.
I tried getting the data table and col through the expression but with no success.
I have done the following so far (VB.net I hope you don't mind):
Dim left = MemberExpression.Property(param, "Item", expression.Constant(r.FieldName))
Dim Table = MemberExpression.Property(param, "Table")
Dim Cols = MemberExpression.Property(Table, "Columns")
Dim Col = MemberExpression.Property(Cols, "Item", Expression.Constant(r.FieldName))
Dim DataType = MemberExpression.Property(Col, "DataType")
But the above does not give me the underlaying data type of the DataColumn. It seems like this should be something really simple but I can't find the right property/indexer to call.
Regards,
Nas
Hi Martin,
is there a way to see if the Expression with specified operator (ExpressionType) is valid or do i have to implement it by my self. For example GreaterThan with two string parameters throws exception when expression is compiled.
Ivan
Hi Nas,
sorry for replying late this time. Could you please post a piece of VB code that does what you want without using Expression trees and then we can try to rewrite it to an expression tree?
Hi Ivan,
I think that's because in C# you can't write 's1 > s2'. which is what GreaterThan does. Try creating an expression that uses String.CompareTo or String.Compare.
The list of all ExpressionTypes is here: http://msdn.microsoft.com/en-us/library/bb361179.aspx
Hi Martin,
It is really great post. I am able to get simple rules working. I need some help with AND/OR operators.
Actually I need solution where rules definition can be read from a XML/text file.
How can I create one rule with multiple conditions with this configuration requirement? Because when these rules are defined in a text file, that these simple rules are not created yet.
Any help/code would be highly appreciated.
Hello! I am excited to find out one thing, of course if I'm not asking too much could you please tell us the place where you spent your childhood?
I have figured out the complex rule issue by forming nested And/Or Data structure.
But I am facing issues with Nested class.
e.g.
public class User
{
public int Age { get; set; }
public string Name { get; set; }
public string Tags { get; set; }
NestedUser NestedUser { get; set; }
public User(int age, string name, string tags)
{
Age = age;
Name = name;
Tags = tags;
NestedUser = new NestedUser();
NestedUser.Count = 10;
}
}
public class NestedUser
{
public int Count { get; set; }
}
How do I get this code Rule working
"NestedUser.Count", "GreaterThan", "5"
Hi VB,
that's pretty cool you figured out how to store AND / OR rules - you probably represent them using an AST, right? Also, an AST is easy to serialize to XML, since it is a tree.
To create an expression for 'NestedUser.Count', have a look at the Expression.Property method: http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.property.aspx
There is even a PropertyOrField method.
Hi VB,
How did you manage And/Or operator?
Thanks,
Bob
I have Created my own classes for And and OR Rules, which can contain List of rules, and each Rule is an Expression.
I AND or OR these rules based on type of the class.
I have also solved the nested properties issue using PropertyInfo Class.
But now I have came across another hurdle.
These Objects could be collection themselves or they could be Properties which could be collection.
Not sure how to resolve these issues.
Hi Martin,
When i am passing GreaterThan then i am getting error as below
"The binary operator GreaterThan is not defined for the types 'System.String' and 'System.String'."
Please help me to rsolve
Very interesting. It's possible to have your full source code solution? I would be very thankful.
Great article thanks.
Not being an experienced ("sharp") C#er, I wonder if the comment / suggestion below is valid.
Seems to me that by compiling the rule the operations and values (e.g. u.Age, "Equals", "20") get 'baked-in'.
If so, the technique could be much more powerful if these could be parameterized - if I understand the technique correctly - at the price of having to construct / compile the rule upon each invocation.
Any feedback or suggestion for implementing such idea is appreciated - even if my question / comment reveal total ignorance.
Thanks for the clean design / thought process.
Am looking for the posted working code sample (my interest lies mainly in using this model for implementation of a QA Automation tool - the object(s) against which the rules will be invoked will be mainly UITestControl instances.)
VB,
sorry for responding late.
To support nested properties / collections you would need something like
new Rule ("Parent.Age", "GreaterThan", "40")
new Rule ("Parents[*].Age", "Contains", "40")
and adapt BuildExpr to support this syntax. It's not trivial but definitely doable.
You might also want to support rules with paths such as "Parents[*].Parents[*].Age", but there it's not clear what the semantics should be - is it a single collection of integers, or a collection of collections of integers, and how do you match on the inner collections?
I would probably start with supporting just
"Parent.Age"
and
"Parent.Tags"
and add more features step by step if needed.
Bey,
also sorry for replying late.
Parameterized rules are an excellent idea!
You could declare rules like:
val rule = new ParameterizedRule("Age", Operators.GreaterThan);
and then change CompileRule to build a function that takes one additional parameter besides the object itself:
Func compiledRule = CompileRule(rule);
Essentially, you would create one additional Expression.Parameter and use it in BuildExpr instead of Expression.Constant.
Let me know if this helped. Thanks a lot!
Hi,
I am really struggling to get nested properties to work. I have looked at the Expression.Property method: http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression.property.aspx but still can't get it to work. Any help would be much appreciated (such as a code snippet?).
Hi John,
by nested properties, do you mean e.g. foo.Bar.Baz?
If you have an expression "e" representing "foo", can you do the following?
Expression.Property(Expression.Property(e, "Bar"), "Baz")
Hi Martin,
Thanks for the reply, I did exactly that. In fact I Split the r.MemberName on the "." and loop through building the left Expression recursively. i.e. left = MemberExpression.Property(left, leftOperands[i]);
However, I am having trouble with lists, and/or indexed lists.
e.g. new Rule ("Collection[0].Code", "Equal", "C137.2")
Or
new Rule ("Collection[*].Code", "Contains", "C137")
Would appreciate a pointer on how to handle such a scenario.
Hi John,
to create an expression for "Collection[0].Code" you'll have to extend your parser that splits the string by dots to also look for integers inside square brackets. The left expression of the "Code" MemberExpression will be an IndexExpression which you can create e.g. using one of the overloads of Expression.Property: http://msdn.microsoft.com/en-us/library/dd630266.aspx
I found this by simply full-text searching the Expression API doc for "index":
http://msdn.microsoft.com/en-us/library/system.linq.expressions.expression_methods.aspx
"Collection[*].Code" is more complicated. You could translate this into an expression tree equivalent to Collection.Any(item => item.Code.Contains("C137")).
Note that if you find out your parser of string containing C# expressions is getting complex, you can check out the open-source NRefactory library that contains a C# parser:
http://www.codeproject.com/Articles/408663/Using-NRefactory-for-analyzing-Csharp-code
Hi Martin,
Great Article, I am planning to use Expressions in one of the projects that I am working on. I have a situation where the user will create a set of rules dynamically and I have to store them in the database. They could be such as [rule1 && (rule2 || rule3)] combunation of AND and OR. Noticed that there are solutions to have seperate and list and or list but what is the best way to approach if I have to persist the expression above.
Many Thanks
Sankalp
Is it possible to build a "between" rule to check if a value is included in a range of 2 values ?
eg : "Age", "Between", "20", "40" -> 20 < Age < 40
eg : "Age", "BetweenOrEqual", "20", "40" ->> 20 <= Age <= 40
To have to have a generic Rule compiler for any types of objects, I changed the line
var paramUser = Expression.Parameter(typeof(User));
To
var paramUser = Expression.Parameter(typeof(T));
How would I go about handling nullable values? I have a property that is a nullable datetime and I get an error when the MakeBinary is called :
The binary operator GreaterThan is not defined for the types 'System.Nullable`1[System.DateTime]' and 'System.DateTime'.
Is there a way to change the type of the Expression.Property in cases like this?
Hi Maritn,
its great article and i used this concept to build custom rule engine, and i want to use new method which is not available in Binary Expressions, similar to Contains, Say StartsWith. but i couldn't find it in the String type, so i tried to use extensions it didn't work though, any idea how to use custom method to work with system types?
very useful post...thanks for sharing
Hi Martin,
Great document. I posted a case in stactoverflow about rule engine (http://stackoverflow.com/questions/33500953/how-evaluate-an-object-in-a-collection-with-microruleengine). Basically i need testing how access collections. Have you an idea of it? Thanks
Hi Martin,
Great artical, I was looking for creating some code samples to create my own business rule engine and this one is really good any easy to understand.
I am not an expert of expression tree, could you please let me know if I can use this solution for creating the rule engine which can process thousands of rule per minute. I want to from the performance perspective using Expression tree based solution will be good over the other available solution like NRules.
Hi Martin. I have lots of these type of conditions stored in the database:
StartReal.IsEmpty() && EstimatedStart.Date < DateTime.Today.Date
StartReal.IsEmpty() && EstimatedStart.Date > DateTime.Today.Date
StartReal.IsEmpty() && EstimatedStart.Date == DateTime.Today.Date
(Note: IsEmpty() is an extension method to check if a Date is null)
Right now, I'm parsing and splitting each one of the conditions and checking manually to the object properties (is a Task class). Is there any way to do it with a expression builder?
I'm asking, because I've tried to adapt your BuildExpr, but the lack of knowledge is not driving me anywhere.
Thanks in advance. Saludos desde Uruguay
Ferrans Nogués doing it manually is fine and probably performant enough. This blog post is only useful if performance is really critical, should have stated it more clearly.
Hi Guys,
Can these rules be created on the fly? An admin would create these validtion/rules
Thanks for sharing .. Great content .. I learned a lot from this code.
Thanks for sharing .. Great content .. I learned a lot from this code. For more ipl betting tips
Free Cricket Betting Tips And Today Match Prediction For All Match Like IPL ,Big Bash,Cpl,Kpl,And More All Tips Free Provided On This Site JSK Betting TIPS.
Cricket Betting Tips
Nice & Informative Blog !
you may encounter various issues in QuickBooks that can create an unwanted interruption in your work. To alter such problems, call us at QuickBooks Customer Service Number 1-877-948-5867 and get immediate technical services for QuickBooks in less time.
Nice information about this games thanks for sharing keep it up
Cricket Betting Tips
Nice Blog !
Our experts at QuickBooks Customer Service are available to help you fix all your QuickBooks issues in this global situation.
Nice blog is it writing thanks for the sharing such types of blogs satta matka
Nice Blog !
One such annoying error is QuickBooks Error 15241. If you need immediate help with this particular issue, call us and get feasible solutions for QuickBooks problems.
Nice information about this games thanks for sharing keep it up
Cricket Betting Tips
Get 100% Free Cricket Betting Tips from Experts for T20, ODI, T10 and Test Matches. We cover all big games like IPL ,World Cup ,BBL and more.
Get 100% free IPL, bbl, t20 World cup, World cup, BPL, ODI and T20 series cricket match free betting tips and prediction.
Tips and prediction for online cricket betting, live odds on bet365.Deposit and start online betting on single click with 100% genuine trust.
Visit Free Cricket Betting Tips - Cricket Betting Tips from Expert Aerek Sir
Visit Free Online Cricket Betting Tips and Prediction| Online Betting Sites
Visit IPL Winner 2021
Visit free IPL betting tips| BBL betting tips | T20 World Cup betting tips|
excellent knowledge and really appreciate your content ,we know how tough this is and excellent everything thanks cheers
Visit Free Cricket Betting Tips | Cricket Betting Tips from Expert Aerek Sir
Visit Free Online Cricket Betting Tips and Prediction| Online Betting Sites
Visit IPL Winner 2021
Visit free IPL betting tips
Visit PSL betting tips
Hi Martin,
I wanted to know how to convert below method implementation into an async method implementation, so that its calling method (CompileRule()) can use await on it? :
static Expression BuildExpr(Rule r, ParameterExpression param)
Post a Comment