c# - Reusable Calculations For LINQ Projections In Entity Framework (Code First)

Question

My domain model has a lot of complex financial data that is the result of fairly complex calculations on multiple properties of various entities. I generally include these as [NotMapped] properties on the appropriate domain model (I know, I know - there's plenty of debate around putting business logic in your entities - being pragmatic, it just works well with AutoMapper and lets me define reusable DataAnnotations - a discussion of whether this is good or not is not my question).

This works fine as long as I want to materialize the entire entity (and any other dependent entities, either via .Include() LINQ calls or via additional queries after materialization) and then map these properties to the view model after the query. The problem comes in when trying to optimize problematic queries by projecting to a view model instead of materializing the entire entity.

Consider the following domain models (obviously simplified):

public class Customer
{
 public virtual ICollection<Holding> Holdings { get; private set; }

 [NotMapped]
 public decimal AccountValue
 {
  get { return Holdings.Sum(x => x.Value); }
 }
}

public class Holding
{
 public virtual Stock Stock { get; set; }
 public int Quantity { get; set; }

 [NotMapped]
 public decimal Value
 {
  get { return Quantity * Stock.Price; }
 }
}

public class Stock
{
 public string Symbol { get; set; }
 public decimal Price { get; set; }
}

And the following view model:

public class CustomerViewModel
{
 public decimal AccountValue { get; set; }
}

If I attempt to project directly like this:

List<CustomerViewModel> customers = MyContext.Customers
 .Select(x => new CustomerViewModel()
 {
  AccountValue = x.AccountValue
 })
 .ToList();

I end up with the following NotSupportedException: Additional information: The specified type member 'AccountValue' is not supported in LINQ to Entities. Only initializers, entity members, and entity navigation properties are supported.

Which is expected. I get it - Entity Framework can't convert the property getters into a valid LINQ expression. However, if I project using the exact same code but within the projection, it works fine:

List<CustomerViewModel> customers = MyContext.Customers
 .Select(x => new CustomerViewModel()
 {
  AccountValue = x.Holdings.Sum(y => y.Quantity * y.Stock.Price)
 })
 .ToList();

So we can conclude that the actual logic is convertible to a SQL query (I.e., there's nothing exotic like reading from disk, accessing external variables, etc.).

So here's the question: is there any way at all to make logic that should be convertible to SQL reusable within LINQ to entity projections?

Consider that this calculation may be used within many different view models. Copying it to the projection in each action is cumbersome and error prone. What if the calculation changes to include a multiplier? We'd have to manually locate and change it everywhere it's used.

One thing I have tried is encapsulating the logic within an IQueryable extension:

public static IQueryable<CustomerViewModel> WithAccountValue(
 this IQueryable<Customer> query)
{
 return query.Select(x => new CustomerViewModel()
 {
  AccountValue = x.Holdings.Sum(y => y.Quantity * y.Stock.Price)
 });
}

Which can be used like this:

List<CustomerViewModel> customers = MyContext.Customers
 .WithAccountValue()
 .ToList();

That works well enough in a simple contrived case like this, but it's not composable. Because the result of the extension is an IQueryable<CustomerViewModel> and not a IQueryable<Customer> you can't chain them together. If I had two such properties in one view model, one of them in another view model, and then the other in a third view model, I would have no way of using the same extension for all three view models - which would defeat the whole purpose. With this approach, it's all or nothing. Every view model has to have the exact same set of calculated properties (which is rarely the case).

Sorry for the long-winded question. I prefer to provide as much detail as possible to make sure folks understand the question and potentially help others down the road. I just feel like I'm missing something here that would make all of this snap into focus.

Answer 1

I did a lot of research on this the last several days because it's been a bit of a pain point in constructing efficient Entity Framework queries. I've found several different approaches that all essentially boil down to the same underlying concept. The key is to take the calculated property (or method), convert it into an Expression that the query provider knows how to translate into SQL, and then feed that into the EF query provider.

I found the following libraries/code that attempted to solve this problem:

LINQ Expression Projection

http://www.codeproject.com/Articles/402594/Black-Art-LINQ-expressions-reuse and http://linqexprprojection.codeplex.com/

This library allows you to write your reusable logic directly as an Expression and then provides the conversion to get that Expression into your LINQ query (since the query can't directly use an Expression). The funny thing is that it'll be translated back to an Expression by the query provider. The declaration of your reusable logic looks like this:

private static Expression<Func<Project, double>> projectAverageEffectiveAreaSelector =
 proj => proj.Subprojects.Where(sp => sp.Area < 1000).Average(sp => sp.Area);

And you use it like this:

var proj1AndAea =
 ctx.Projects
  .AsExpressionProjectable()
  .Where(p => p.ID == 1)
  .Select(p => new 
  {  
   AEA = Utilities.projectAverageEffectiveAreaSelector.Project<double>() 
  });

Notice the .AsExpressionProjectable() extension to set up projection support. Then you use the .Project<T>() extension on one of your Expression definitions to get the Expression into the query.

LINQ Translations

http://damieng.com/blog/2009/06/24/client-side-properties-and-any-remote-linq-provider and https://github.com/damieng/Linq.Translations

This approach is pretty similar to the LINQ Expression Projection concept except it's a little more flexible and has several points for extension. The trade off is that it's also a little more complex to use. Essentially you still define your reusable logic as an Expression and then rely on the library to convert that into something the query can use. See the blog post for more details.

DelegateDecompiler

http://lostechies.com/jimmybogard/2014/05/07/projecting-computed-properties-with-linq-and-automapper/ and https://github.com/hazzik/DelegateDecompiler

I found DelegateDecompiler via the blog post on Jimmy Bogard's blog. It has been a lifesaver. It works well, is well architected, and requires a lot less ceremony. It does not require you to define your reusable calculations as an Expression. Instead, it constructs the necessary Expression by using Mono.Reflection to decompile your code on the fly. It knows which properties, methods, etc. need to be decompiled by having you decorate them with ComputedAttribute or by using the .Computed() extension within the query:

class Employee
{
 [Computed]
 public string FullName
 {
  get { return FirstName + " " + LastName; }
 }
 public string LastName { get; set; }
 public string FirstName { get; set; }
}

This can also be easily extended, which is a nice touch. For example, I set it up to look for the NotMapped data annotation instead of having to explicitly use the ComputedAttribute.

Once you've set up your entity, you just trigger decompilation by using the .Decompile() extension:

var employees = ctx.Employees
 .Select(x => new
 {
  FullName = x.FullName
 })
 .Decompile()
 .ToList();