Benchmarking LINQ vs. DAAB vs. ADO.NET

Abstract: This article presents benchmarking results of performance of SP calls using LINQ, Data-access application block and ADO.NET. Some time ago I was doing LINQ performance benchmarking, looking for an option of replacing our existing data layer with it. One thing that was especially important for me to check - was performance of executing SP's. The reason is that most of our DAL logic resides in SP's at one hand and that performance-critical parts are implemented as SP's as well (for all the good reasons, like possibility to benefit from queries compilation etc). Below are performance benchmarking results of execution uspGetManagerEmployees SP from AdventureWorks DB. (Basically it does some sort of JOIN to retrieve records of employees data for the specified Manager ID). I tested three data-access models - raw ADO.NET vs. DAAB (our current DAL workhorse) vs. LINQ (May 2006 preview version). As a prerequisite for running this test you'll need AdventureWorks DB and installation of LINQ. Few side-notes: I've left opening of DB connection out of the context of benchmarked section. Only SP execution is measured. I use ExecuteReader rather than ExecuteDataSet, as it is much more lightweight. For LINQ benchmark I using GetEnumerator(). So here goes the test code: 1. Plain ADO.NET: public static void SqlSPBench() { string connectionString = "Data Source=localhost;Initial Catalog=AdventureWorks;Integrated Security=True"; using( SqlConnection conn = new SqlConnection( connectionString ) ) { conn.Open(); Stopwatch watch = new Stopwatch(); watch.Start(); int count = 10000; for( int i = 0; i < count; i++ ) { using( SqlCommand cmd = new SqlCommand() ) { cmd.Connection = conn; cmd.CommandText = "uspGetManagerEmployees"; cmd.CommandType = CommandType.StoredProcedure; cmd.Parameters.Add( new SqlParameter( "ManagerID", 3 ) ); using( SqlDataReader reader = cmd.ExecuteReader( CommandBehavior.Default ) ) { //while (reader.Read()) //{ // Console.WriteLine(reader[3]); //} } } } watch.Stop(); Console.WriteLine( "Running " + count + " SP's using SQL ExecuteReader took " + watch.ElapsedMilliseconds + " ms"); Console.ReadLine(); } } 2. DAAB (Data Access Application Block): public static void SqlSpDAAB() { Database adventureWorksDataBase = DatabaseFactory.CreateDatabase( "AdventureWorks" ); Stopwatch watch = new Stopwatch(); watch.Start(); int count = 10000; for( int i =0; i<count; i++ ) { using( DBCommandWrapper sp_uspGetManagerEmployees = adventureWorksDataBase.GetStoredProcCommandWrapper( "uspGetManagerEmployees" ) ) { sp_uspGetManagerEmployees.AddInParameter( "@ManagerID", DbType.Int32, 3 ); using( IDataReader dataReader = adventureWorksDataBase.ExecuteReader( sp_uspGetManagerEmployees ) ) { //DataSet results = // adventureWorksDataBase.ExecuteDataSet( sp_uspGetManagerEmployees ); } } } watch.Stop(); Console.WriteLine( "Running " + count + " SP's using DAAB took " + watch.ElapsedMilliseconds + " ms"); Console.ReadLine(); } 3. LINQ (using ORM layer produced by SQLMetal.exe - "c:\Program Files\LINQ Preview\Bin\SqlMetal.exe" /server:. /database:AdventureWorks /pluralize /sprocs /code:AdventureWorks.cs): public static void LinqSPBench() { string connectionString = "Data Source=localhost;Initial Catalog=AdventureWorks;Integrated Security=True"; AdventureWorks adventureWorks = new AdventureWorks( connectionString ); Stopwatch watch = new Stopwatch(); watch.Start(); int count = 10000; for( int i =0; i<count; i++ ) { StoredProcedureResult<UspGetManagerEmployeesResult> result = adventureWorks.UspGetManagerEmployees( 3 ); IEnumerator<UspGetManagerEmployeesResult> enumerator = result.GetEnumerator(); //while( enumerator.MoveNext() ) //{ // Console.WriteLine( enumerator.Current.LastName ); //} } watch.Stop(); Console.WriteLine( "Running " + count + " SP's using LINQ took " + watch.ElapsedMilliseconds + " ms"); Console.ReadLine(); } Here are timing results of this benchmarking test: Running 10000 SP's using SQL ExecuteReader took 42806 ms Running 10000 SP's using DAAB took 46443 ms Running 10000 SP's using LINQ took 55136 ms Results So, comparing to the raw ADO.NET - DAAB is 8% slower and LINQ is 28% slower. Comparing to DAAB - LINQ is 18% slower. CPU usage intensity is about 2% average for raw ADO.NET vs. about 8% for DAAB vs. about 20% for LINQ - order of magnitude worse than raw ADO.NET (CPU graphs captured with perfmon below). Fig. 1 ADO.NET - CPU usage Fig. 2 DAAB - CPU usage Fig. 3 LINQ - CPU usage In overall - this performance overhead is quite disappointing. I still consider to use LINQ, as we have DAL/BLL servers in our architecture, which we may merely upgrade to faster hardware, while isolating thin-client computers from this performance overhead. I think it's a big challenge for the LINQ team to address its performance.