Channels ▼
RSS

.NET

Reducing Computing Time with Multithreading


How To Increase Performance with Multithreading

Applying multithreading techniques is useful when you have to deal with multiple requests and you do not want to slow down the process while waiting for results. The typical example is a system made of many requests to different web services. If you need to implement something similar, you will probably end up using a simple for iteration and calling each web service in this block.

This technique may work with few requests but to speed up execution you need to use multithreading.

Problem

Let's suppose we have to gather some data using different web services and display the results on the page, just like a flight-comparing engine would do.

We want to avoid latency and provide a better experience for the user while waiting for different external services to respond to our requests. Normally, if we opt to execute a single task at a time, the total time will be the sum of the entire operation. We can dramatically improve the speed by executing the tasks in parallel -- we will gain in total response time.

Solution

In heavy-load scenarios where you need to execute different tasks at the same time using a worker thread, you may reduce the total execution time. A worker thread is a secondary thread created by the primary one to accomplish a specific task.

The .NET Framework has a specific namespace, called System.Threading, to support threads and a specific class, named Thread, to represent the concept of a thread in managed code.

Thread has a special constructor that receives the code to be executed and a Start method to begin execution. When the thread is created, there is a fork in the execution flow: the primary thread will continue its normal execution, while the secondary will start its work. To provide a true multithreading experience, we are going to execute every request as a separate thread. Using this approach, the total time for the request to be executed is not the amount of different requests but the maximum one (plus the overhead of creating, destroying, and joining threads).

Even if possible, it is not a good idea to directly instantiate threads. For such a specific scenario, a specialized class, called ThreadPool, is used. This class represents a pool of threads managed by the CLR itself and can be used to coordinate them. When using a technique like this one, we need thread synchronization. Each call to the QueueUserWorkItem method will immediately return, so we need a way to notify our class that each thread has completed its work and that the results are ready to show. To accomplish this task, we need to manually use a WaitHandle class, shown in Figure 2.

Figure 2: Threads generation and synchronization need to be handheld to work with threads properly. When completed, a single thread will notify the ThreadPool.

The problem at this point is that, while accessing the List to add our results, there is no guarantee that there will not be collisions from different threads trying to modify the same collection at the same time. List is in fact not thread safe, so we need to synchronize the modifications using the lock keyword (in C#) or the Monitor class. The whole code is in Listing 1


<b>(a)</b>

public class PriceEngine
{
  public PriceEngine(string flightNumber)
  {
    FlightNumber = flightNumber;
    FlightResults = new List<FlightPriceResult>();
  }

  public void GetFlightPrices()
  {
    StartTime = DateTime.Now;
    try
    {
      List<WaitHandle> handles = new List<WaitHandle>();#1
      foreach (IFlightPriceProvider provider in GetProviders())#2
      {
        ManualResetEvent handle = new ManualResetEvent(false);
        handles.Add(handle);#3 
        Tuple<IFlightPriceProvider, ManualResetEvent> currentState =
          new Tuple<IFlightPriceProvider, ManualResetEvent>
                                (provider, handle);

        ThreadPool.QueueUserWorkItem(
          delegate(object state)
          {
           …
          }, currentState
        );
      }
      WaitHandle.WaitAll(handles.ToArray());

    }
    Finally#4
    {
      EndTime = DateTime.Now;
      Completed = true;
    }
  }  
}


<b>(b)</b>


Public Class PriceEngine
  Public Sub New(ByVal number As String)
    FlightNumber = number
    FlightResults = New List(Of FlightPriceResult)()
  End Sub
  
  Public Sub GetFlightPrices()
    StartTime = DateTime.Now
    Try
      Dim handleList As New List(Of WaitHandle)()#1
      For Each provider As IFlightPriceProvider In GetProviders()#2
        Dim handle As New ManualResetEvent(False)
        handleList.Add(handle)#3
        Dim currentState As
            New Tuple(Of IFlightPriceProvider, ManualResetEvent)
                                                    (provider, handle)

        ThreadPool.QueueUserWorkItem(AddressOf ExecuteProvider,
                                     currentState)
      Next

      WaitHandle.WaitAll(handleList.ToArray())
    Finally#4
      EndTime = DateTime.Now
      Completed = True
    End Try  End Sub
End Class

<b>
#1 WaitHandle list containing tasks
#2 Retrieve the providers list
#3 Create and register the handle
#4 Set the completed flags
</b>

Listing 1: The engine to initialize the multithreading requests: (a) C#; (b) VB

In Listing 1, you can find the general structure of the engine. Its inner working is contained in the code in Listing 2, where each result is retrieved from different providers and added to the results collection using a thread-safe approach.


<b>(a)</b>

delegate(object state)
{
  Tuple<IFlightPriceProvider, ManualResetEvent> invokeState =
  (Tuple<IFlightPriceProvider, ManualResetEvent>)state;#1

  FlightPriceResult result = null;
  IFlightPriceProvider currentProvider = invokeState.Item1;
  result = currentProvider.GetFlightPrice(FlightNumber);

  bool lockTaken = false;
  Monitor.Enter(Sync, ref lockTaken);#2
  try
  {
    FlightResults.Add(result);#3
  }
  finally
  {
   if (lockTaken)
      Monitor.Exit(Sync);#4
  }

  ManualResetEvent resetHandle = invokeState.Item2;
  resetHandle.Set();#5
}

<b>(b)</b>

  Private Sub ExecuteProvider(ByVal state As Object)
    Dim invokeState As Tuple(Of IFlightPriceProvider, ManualResetEvent) =
      DirectCast(state, Tuple(Of IFlightPriceProvider, ManualResetEvent))

    Dim result As FlightPriceResult = Nothing
    Dim currentProvider As IFlightPriceProvider = invokeState.Item1#1
    result = currentProvider.GetFlightPrice(FlightNumber)

    Dim lockTaken As Boolean = False
    Monitor.Enter(Sync, lockTaken)#2
    Try
      FlightResults.Add(result)#3
    Finally
      IF lockTaken Then
        Monitor.Exit(Sync)#4
      End If
    End Try

    Dim resetHandle As ManualResetEvent = invokeState.Item2
    resetHandle.Set()#5
  End Sub
End Class
<b>
#1 Use the provider
#2 Using a lock to be thread-safe
#3 Thead-safe add to a list
#4 Closing the lock
#5 Finally, re-join the thread...</b>

Listing 2: The engine implementation C# delegate(object state): (a) C#; (b) VB

The IFlightPriceProvider interface is guaranteeing that every provider has the GetFlightPrice() method to load results and is part of our design strategy, often referred to as Provider Model. The providers attached to this example are just for test and, in order to simulate latency, they have a call to Thread.Sleep to freeze execution for a couple of seconds. A simple implementation is in Listing 3.


<b>(a)</b>
public class SecondProvider: IFlightPriceProvider
{
  public FlightPriceResult GetFlightPrice(string FlightNumber)
  {
    Thread.Sleep(3000);#1

    return new FlightPriceResult() {
                        FlightNumber = FlightNumber, Price = 260 };#2
  }

}

<b>(b)</b>

Public Class SecondProvider
  Implements IFlightPriceProvider
  Public Function GetFlightPrice(ByVal FlightNumber As String) As FlightPriceResult
    Thread.Sleep(3000)#1
    
    Dim result as New FlightPriceResult()
    result.FlightNumber = FlightNumber
    result.Price = 260
    Return result#2
  End Function
  
End Class
<b>
#1 Retrieve the providers list
#2 Return a fixed value
</b>

Listing 3: A simple provider implementation: (a) C#; (b) VB

In real-life scenarios, you will insert your code in this method and populate a new instance of FlighPriceResult class to return the corresponding flight price. To effectively start the work, we need to create a page with a Textbox to enter the flight number and a Button to execute the code, as shown in Listing 4.


<b>(a)</b>

protected void StartWork_Click(object sender, EventArgs e)
{
  PriceEngine engine = new PriceEngine(FlightNumber.Text);

  Session["engine"] = engine;

  ThreadPool.QueueUserWorkItem(#2
    delegate(object state) 
    { 
      PriceEngine priceEngine = (PriceEngine)state;
      priceEngine.GetFlightPrices();
    }, engine);

  Response.Redirect("results.aspx");#3
}

<b>(b)</b>

Protected Sub StartWork_Click(ByVal sender As Object, ByVal e As EventArgs)
  Dim engine As New PriceEngine(FlightNumber.Text)#1
  
  Session("engine") = engine
  
<b>  ThreadPool.QueueUserWorkItem(AddressOf Execute, engine)#2</b>

  Response.Redirect("results.aspx")#3
End Sub

Protected Sub Execute(ByVal state As Object)
  Dim priceEngine As PriceEngine = DirectCast(state, PriceEngine)
  priceEngine.GetFlightPrices()
End Sub
<b>
#1 Start a new instance
#2 The next statement is executed
#3 Redirect on a waiting page
</b>

Listing 4: The page containing the code to start the work: (a) C#; (b) VB

The code in Listing 4 is very simple; the engine will start, saving its instance in Session so we can access it later. The results.aspx page includes a code to check in Session for the instance of PriceEngine class that originates the threads, checking at intervals for the execution to be completed. By using a simple reload of the page, as in Listing 5, we can check the state of the job and, in case it is done, just display the results to the user.


<b>(a)</b>

<asp:PlaceHolder ID="WaitingPanel" runat="server" Visible="false">#1
  Please wait...
</asp:PlaceHolder>

<asp:PlaceHolder ID="ResultsPanel" runat="server" Visible="false">#2
  <h1>Results</h1>
  
  <asp:literal ID="Feedback" runat="server" />
  
  <asp:GridView ID="ResultList" runat="server" />
</asp:PlaceHolder>

<b>(b)</b>

protected void Page_Load(object sender, EventArgs e)
{
  PriceEngine engine = Session["engine"] as PriceEngine;#3
  if (engine == null)
    Response.Redirect("./");

  if (<b>engine.Completed</b>)#4
  {
    ResultsPanel.Visible = true;
    ResultList.DataSource = engine.FlightResults;#5
    ResultList.DataBind();

    Feedback.Text = string.Format("Elapsed time: {0}",
            engine.EndTime.Subtract(engine.StartTime)); 

  }
  else
  {
    WaitingPanel.Visible = true;

    Header.Controls.Add(new HtmlMeta() { 
                            HttpEquiv = "refresh", Content = "2" }); #6
  }
}

<b>(c)</b>

Protected Sub Page_Load(ByVal sender As Object, ByVal e As EventArgs)
  Dim engine As PriceEngine = TryCast(Session("engine"), PriceEngine)#3
  If engine Is Nothing Then
    Response.Redirect("./")
  End If
  
  If engine.Completed Then#4
    ResultsPanel.Visible = True
    ResultList.DataSource = engine.FlightResults#5
    ResultList.DataBind()
    
      
    Feedback.Text = String.Format("Elapsed time: {0}", 
                  engine.EndTime.Subtract(engine.StartTime))
  Else
    WaitingPanel.Visible = True
    
    ' programmatically add a refresh meta tag
    Dim meta as New HtmlMeta()
    meta. HttpEquiv = "refresh"
    meta.Content = "2"
    Header.Controls.Add(meta)#6
  End If
End Sub
<b>
#1 Waiting panel
#2 Results panel
#3 Get the engine from session
#4 The work is completed
#5 Retrieve the result list
#6 Refresh the page and check again
</b>

Listing 5: The results.aspx page contains both the waiting panel and the results: (a) Markup; (b) C#; (c) VB

The sequence of the entire workflow is displayed in Figure 3, and it is very similar to what comparing price engines are doing to speed up their execution, while you are waiting for the results to come in the web page.

Figure 3: The sequence of our multithreading system. First of all, the providers are initialized and their work executed. When all the providers are done, the engine is notified so the results may be displayed.

Dealing with multithreading is not simple because, as previously shown, you have to take care of the details; just be sure to use thread-safe collections. In any case, the results are very interesting. When dealing with a solution like the one exposed here, the performance boost you can achieve by simply letting different threads simultaneously execute different tasks is visible at first glance by both you and your customers.

Multithreading can boost specific applications, like the one in this scenario, by leveraging spanning the work on different threads. It is not intended to be used by every application because there are cons to be considered. ASP.NET uses a thread pool and it is useful to remark that the pool size includes both ASP.NET-generated threads and the ones generated in your code. Remember that this technique, in very high traffic applications, can lead to consuming the entire pool size, and you'll need to increase it accordingly if you notice that its size is very low. Just for your information, current threads can be monitored in Control Panel under Performance Monitor. Using threads to execute multiple tasks separately is a good design decision to increase performance dramatically and to leverage ASP.NET multithread nature in a very good way.


Related Reading


More Insights






Currently we allow the following HTML tags in comments:

Single tags

These tags can be used alone and don't need an ending tag.

<br> Defines a single line break

<hr> Defines a horizontal line

Matching tags

These require an ending tag - e.g. <i>italic text</i>

<a> Defines an anchor

<b> Defines bold text

<big> Defines big text

<blockquote> Defines a long quotation

<caption> Defines a table caption

<cite> Defines a citation

<code> Defines computer code text

<em> Defines emphasized text

<fieldset> Defines a border around elements in a form

<h1> This is heading 1

<h2> This is heading 2

<h3> This is heading 3

<h4> This is heading 4

<h5> This is heading 5

<h6> This is heading 6

<i> Defines italic text

<p> Defines a paragraph

<pre> Defines preformatted text

<q> Defines a short quotation

<samp> Defines sample computer code text

<small> Defines small text

<span> Defines a section in a document

<s> Defines strikethrough text

<strike> Defines strikethrough text

<strong> Defines strong text

<sub> Defines subscripted text

<sup> Defines superscripted text

<u> Defines underlined text

Dr. Dobb's encourages readers to engage in spirited, healthy debate, including taking us to task. However, Dr. Dobb's moderates all comments posted to our site, and reserves the right to modify or remove any content that it determines to be derogatory, offensive, inflammatory, vulgar, irrelevant/off-topic, racist or obvious marketing or spam. Dr. Dobb's further reserves the right to disable the profile of any commenter participating in said activities.

 
Disqus Tips To upload an avatar photo, first complete your Disqus profile. | View the list of supported HTML tags you can use to style comments. | Please read our commenting policy.
 

Video