Returning Large Volume of Records from SOAP based WCF Service

Objective

In this article I will explain; How to return large volume of data (around 50000 records) from SOAP based WCF service to any client. I will show; what service configuration setting and client configuration setting is needed to handle large volume of data returning for WCF service.

Follow the below steps

Step 1: Create a WCF service

To creates WCF service; select File -> New -> Project-> Web -> WCF Application.
Service will contain
1. One Operation contract. This function will pull large data from database using stored procedure.
2. One Data Contract. This class will act as Data Transfer object (DTO) between client and service.
3.basicHttpBinding is being used in the service. You are free to use any binding as of your requirement.

Data Transfer Class

[DataContract]
public class DTOClass
{
[DataMember]
public string SystemResourceId
{
get;
set;
}
[DataMember]
public string SystemResourceName
{
get;
set;
}
[DataMember]
public string Created
{
get;
set;
}
[DataMember]
public string Creater
{
get;
set;
}
[DataMember]
public string Updated
{
get;
set;
}
[DataMember]
public string Updater
{
get;
set;
}
}

1. Name of DTO class is DTOClass. You can give any name of your choice.
2. There are 6 string properties.
3. All properties are attributed with DataMember.

Contract

[ServiceContract]
[ServiceKnownType(typeof(DTOClass))]
public interface IService1
{
[OperationContract]
List GetData();
}

1. Service is returning List of DTOClass.
2. To get serialized at run time, making sure Data Contract is known to contract by using known type.

Service Implementation

public class Service1 : IService1
{
string cs = @"Data Source=xxxserver;Initial Catalog=Sampledatabase;User=dhananjay;Password=dhananjay";
List restDto = new List ();
DTOClass dto;

public List GetData()
{
SqlConnection con = new SqlConnection(cs);
SqlCommand cmd = new SqlCommand("GetAllSystemResourceDetails", con);
cmd.CommandType = CommandType.StoredProcedure;
DataTable dt = new DataTable();
SqlDataAdapter adp = new SqlDataAdapter(cmd);
adp.Fill(dt);
for (int i = 0; i < dt.Rows.Count; i++)
{
dto = new DTOClass();
dto.SystemResourceId = dt.Rows[i][0].ToString();
dto.SystemResourceName = dt.Rows[i][1].ToString();
dto.Created = dt.Rows[i][8].ToString();
dto.Updated = dt.Rows[i][10].ToString();
dto.Creater = dt.Rows[i][9].ToString();
dto.Updater = dt.Rows[i][11].ToString();
restDto.Add(dto);
}

return restDto; ;
}
} 
}

1. This is simple implementation. Where ADO.Net is being used to fetch data from Database.
2. GetAllSystemResourceDetails is name of the stored procedure.
Note: Purpose of this article is to show how to push large volume of data from WCF service. So, I am not emphasizing ADO.Net part here. See the other articles for detail explanation on ADO.Net 

Configuration setting at service side

<system.serviceModel><services><service name="TestingLargeData.Service1"behaviorConfiguration="TestingLargeData.Service1Behavior">                   
<endpoint address="" binding="basicHttpBinding" contract="TestingLargeData.IService1" bindingConfiguration="LargeBuffer"><identity><dns value="localhost"/></identity></endpoint><endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/></service></services><behaviors><serviceBehaviors><behavior name="TestingLargeData.Service1Behavior">                          
<serviceMetadata httpGetEnabled="true"/>                         
<serviceDebug includeExceptionDetailInFaults="false"/><dataContractSerializer maxItemsInObjectGraph="2147483647"/></behavior></serviceBehaviors></behaviors><bindings><basicHttpBinding><binding name="LargeBuffer" maxBufferSize="2147483647" maxReceivedMessageSize="2147483647"><readerQuotas maxDepth="2147483647" maxStringContentLength="2147483647"maxArrayLength="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="2147483647" /></binding></basicHttpBinding></bindings></system.serviceModel>

1. In service behavior, for dataContractSerializer I am increasing the number of object that can be serialized or de serialized at a time. By default it is set to 3565. Since our requirement is to push or return large volume of data, so I am giving value for maxItemsInObjectGraph to maximum integer number.
2. Since, I am using basicHttpBinding, so I am modifying values for this binding. I am setting all the attributes to maximum integer value.
3. Binding I am using is basicHttpBinding.
Compile the service and run to test, whether service is successfully created or not? 

Step 2: Creating a client and Consuming the service

Create any type of client. For my purpose I am creating a console client. After creating a console project, right click at Service Reference and add service reference. While adding service reference make sure, in advance setting as collection type System.Collection.Generic.List is selected 

Configuration setting at client side

<?xml version="1.0" encoding="utf-8" ?><configuration><system.serviceModel><bindings><basicHttpBinding><binding name="BasicHttpBinding_IService1" closeTimeout="00:01:00"openTimeout="00:01:00" receiveTimeout="00:10:00"
sendTimeout="00:01:00"allowCookies="false" bypassProxyOnLocal="false" hostNameComparisonMode="StrongWildcard"maxBufferSize="2147483647" maxBufferPoolSize="2147483647" maxReceivedMessageSize="2147483647"messageEncoding="Text" textEncoding="utf-8" transferMode="Buffered"useDefaultWebProxy="true"><security mode="None"><transport clientCredentialType="None" proxyCredentialType="None"realm="" /><message clientCredentialType="UserName" algorithmSuite="Default" /></security></binding></basicHttpBinding></bindings><client><endpoint address="http://localhost:55771/Service1.svc" binding="basicHttpBinding"bindingConfiguration="BasicHttpBinding_IService1" contract="ServiceReference1.IService1"name="BasicHttpBinding_IService1"behaviorConfiguration ="r1"/></client><behaviors><endpointBehaviors><behavior name ="r1"><dataContractSerializer maxItemsInObjectGraph ="2147483647"/></behavior></endpointBehaviors></behaviors></system.serviceModel></configuration>

1. Attributes of basicHttpBinding has been modified for the maximum integer values.
2. In endpoint behavior maxItemsInObjectGraph for dataContractSerializer has been modified to maximum integer value.

Program.cs

class Program
{
static void Main(string[] args)
{
Service1Client proxy = new Service1Client();

List res = new List ();
res = proxy.GetData();

foreach (DTOClass i in res)
{
Console.WriteLine(i.SystemResourceId + i.SystemResourceName + i.Updated + i.Updater);

}
Console.Read();
}
}
}

Conclusion

In this article, I explained how we can return large volume of data from WCF service and how at client side large volume of data can be consumed. This article explained on WCF SOAP based service. Next article I will explain how to achieve same for WCF REST service. Thanks for reading.