|
Continuous LINQ is just one expansion made possible through language extensions and LINQ on the .NET Framework 3.5.
January 14, 2008
URL:http://www.drdobbs.com/windows/continuous-linq/windows/continuous-linq/205604421
Kevin is a Research Developer at Liquidnet, an equities trading system, where he has been researching Web 2.0, social software, and .NET 3.0. He can be contacted at [email protected].
If you haven't been paying a lot of attention to the barrage of new Microsoft technology, LINQ is one of the key new features of the .NET Framework 3.5. LINQ, short for "Language Integrated Query," lets you express filtering, sorting, grouping, and aggregation tasks using your language of choice with a syntax that looks pretty familiar to anyone with experience writing SQL queries for relational databases.
No matter what kind of application you're writing or what industry the application is for, there are some basic tasks that almost every programmer needs to do. At some point, you will likely have a list of data you need sorted, filtered, grouped, or aggregated in some fashion. Before LINQ, this involved the tedium of creating a loop to iterate through all of the list items, then adding the items that match a filter to the output collection, making sure to preserve the proper sort order. Using LINQ, you can now write statements that look like this, dramatically simplifying the code:
var results =
from customer in CustomerList
where customer.Age > 21
orderby customer.LastName
select customer;
Through the magic of the language extensions feature in .NET 3.5, this query will essentially turn into something that looks like this:
IEnumerable<Customer> results =
CustomerList.Where( f => f.Age >
21).OrderBy( o=>
o.LastName ).Select( s => s);
Don't worry if this syntax looks ugly to you—that's the entire purpose behind embedding language shortcuts for LINQ queries directly in C# and VB.NET. What you see in the preceding code sample are language extensions and lambda functions. The Where language extension to IEnumerable<T> lets you filter pretty much any list, so long as the items in the list implement IEquatable<T>.
On its own, LINQ is a fantastic addition to the arsenal of any developer writing virtually any kind of application. The true power of the .NET Framework 3.5 comes with your ability to enhance the functionality of LINQ using language extensions. Language extensions let you create your own overridden implementations for Where, OrderBy, GroupBy, and the like. Continuous LINQ (CLINQ) is just one sample of such an extension to LINQ that was created using this facility.
When building database applications, the general model is you write a query to obtain some data. Data can then be modified and then sent back to the database. In general, however, you do not expect to get continuous updates from the database as data in the underlying tables change. With most relational databases, the performance hit from doing such a thing would bring the database to a halt and probably impact the application as well.
Other applications need to be quickly updated in response to continuously changing data. An example from the financial industry would be an application that listens to market data ticks. Using the current implementation of LINQ, you could write a query like this:
var tickData = from tick in MarketData.Ticks
where tick.Symbol == "AAPL"
orderby tick.Ask descending
select tick;
The problem with this implementation is that tickData is completely stale. Immediately after you run the query, tickData will never grow or shrink dynamically on its own. It would be nice if the contents of tickData could continuously update every time new market data is added to the original source of the query, MarketData.Ticks. This is what the Continuous LINQ extension does.
Continuous LINQ works by attaching listeners, or adapters, to the source collection when the Where, OrderBy, and GroupBy language extensions are invoked. At some point in the future, it might support other extensions as well, but these were the most important. The adapter's job is to listen to changes to the source collection, including all items in the source collection, and decide if that change needs to be propagated to the destination collection. For example, using the preceding market-data query, if MarketData.Ticks was derived from ContinuousCollection<T>, then adapters would be created that listen to the source data. If a new item is added to the source collection that has a symbol name of "AAPL," then the item is automatically added to the results. These adapters also maintain sort order and can be nested and chained at will. Figure 1 illustrates the flow of data through adapters from a typical CLINQ query.
You can extend any IEnumerable<T> derivative class and provide your own domain-specific implementations for the where, orderby, and groupby LINQ query operators. Listing One illustrates the where and orderby extensions to a class called ContinuousCollection<T>. This class is really nothing more than a thread-safe derivative of ObservableCollection<T> that ensures that all change publications take place on the main GUI dispatcher thread to let all WPF GUIs and CLINQ adapters see the changes.
public static class ContinuousQueryExtension
{
#region Where
public static ContinuousCollection<T> Where<T>(
this ContinuousCollection<T> source, Func<T, bool> filterFunc) where T: IEquatable<T>
{
Trace.WriteLine("Filtering Observable Collection.");
ContinuousCollection<T> output = new ContinuousCollection<T>();
FilteringViewAdapter<T> fva =
new FilteringViewAdapter<T>(source, output, filterFunc);
return output;
}
#endregion
#region OrderBy
public static ContinuousCollection<TSource> OrderBy<TSource, TKey>(
this ContinuousCollection<TSource> source, Func<TSource, TKey> keySelector)
where TSource : IEquatable<TSource>
where TKey : IComparable
{
Trace.WriteLine("Ordering Observable Collection (Ascending).");
ContinuousCollection<TSource> output = new ContinuousCollection<TSource>();
SortingViewAdapter<TSource, TKey> sva = new SortingViewAdapter<TSource, TKey>(
source, output,
new FuncComparer<TSource, TKey>(keySelector, false));
return output;
}
#endregion
#region OrderByDescending
public static ContinuousCollection<TSource> OrderByDescending<TSource, TKey>(
this ContinuousCollection<TSource> source, Func<TSource, TKey> keySelector)
where TSource : IEquatable<TSource>
where TKey : IComparable
{
Trace.WriteLine("Ordering Observable Collection (Descending).");
ContinuousCollection<TSource> output = new ContinuousCollection<TSource>();
SortingViewAdapter<TSource, TKey> sva = new SortingViewAdapter<TSource, TKey>(
source, output,
new FuncComparer<TSource, TKey>(keySelector, true));
return output;
}
#endregion
}
In the constructor of the adapters, the adapter attaches event handlers for the collection-changed events and the property-changed events for each item in the collection. Then, in response to these changes, the adapter determines if the changed item needs to be propagated to the destination collection or if it needs to be removed from the destination collection (for example, a modified item no longer satisfies the query predicate and needs to be removed). The removal will cascade down the chain so it makes no difference how many where, groupby, or orderby adapters the change needs to pass through.
The great thing about CLINQ and other language extensions enabled through LINQ is that all of the complexity is hidden from developers. You don't need to look at Listing One to use the advanced functionality—simply write a LINQ query against a ContinuousCollection instead of a standard list and it "just works."
There are lots of different samples that can be written to illustrate CLINQ in action, but one of the more applicable samples I've found is an Order Book. An order book application monitors the market data for a select subset of symbols. Without CLINQ, I would have to set up manual monitoring threads that listen for changes to the underlying market data and then manually propagate those changes to the model, which is then bound to the GUI. This creates a lot of moving parts, makes debugging difficult, and can be very error-prone. With CLINQ, I can simply set up some queries in the controller code and everything is dynamically updated as needed. For example, in the demo order book application, to open a new book and create a variable that contains just the subset of market data belonging to a particular symbol, you can use a simple CLINQ query (a LINQ query against a ContinuousCollection). Listing Two shows the controller method SubscribeToSymbol, which creates a new window and sets the AskTicks and BidTicks properties of that window to CLINQ queries.
public void SubscribeToSymbol(MarketSymbol symbol)
{
if (!ModelRoot.Current.SubscribedSymbols.Contains(symbol))
{
ModelRoot.Current.SubscribedSymbols.Add(symbol);
MarketDataBook newBook = new MarketDataBook(); // WPF Window
newBook.Symbol = symbol;
newBook.AskTicks =
from tick in ModelRoot.Current.MarketData
where tick.Side == TickSide.Ask &&
tick.Symbol == symbol
orderby tick.Price descending
select tick; // SELL
newBook.BidTicks =
from tick in ModelRoot.Current.MarketData
where tick.Side == TickSide.Bid &&
tick.Symbol == symbol
orderby tick.Price
select tick; // BUY
_monitorBooks.Add(symbol, newBook);
}
}
Once the AskTicks and BidTicks properties have been set, those collections will be automatically updated as more market data streams in, without any further effort on the part of the programmer. This is a powerful way of dealing with message processing and streaming data.
We actually had an interesting time testing the performance of CLINQ. Without being bound to any GUI, we are able to pump over 1,000,000 changes per second into a source collection (with an upper cap on collection size to avoid running out of resources) with a large CPU load, and at 100,000 changes per second the load looked to be sustainable indefinitely.
The real tricky part came when trying to bind CLINQ to a WPF GUI. Thanks to Pavan Podila from the WPF Way (pavanpodila .spaces.live.com), we were able to track down a potentially fatal problem with data binding. In our ListBox, the height wasn't explicitly set, which prevented the item container style from being a VirtualizingStackPanel. Added to that was the template selector for picking the alternating grid style that could get invoked every tick for each of the hundreds of thousands of bound rows. Lesson learned: Make sure that your bound control remains virtualized and be very careful with item styles and item container styles when working with large data sets. Taking these precautions, we were able to get nearly the same throughput into CLINQ when databound as we were when using no GUI at all.
Using code like Listing Two, I've constructed a sample Order Book demo that lets you open windows into arbitrary symbols with simulated market data coming in at fixed intervals; see Figure 2.
This window contains two listboxes. Each listbox is bound to a subset of market data. The BUY listbox is bound to the bid ticks for AAPL and the SELL listbox is bound to the ask ticks for AAPL. As new market data comes in to the central market data store (could be from network messages or, in my case, from a background thread on a timer), each set of query results is updated dynamically. If new market data for symbols other than AAPL comes in, the query results are not affected. If a new tick comes in for AAPL, it is placed in the appropriate listbox based on the side of the tick. All of that work, decision making, and data propagation is done automatically by the CLINQ language extension.
Fortunately, language extensions and CLINQ specifically can be used for pleasure as well as business. The following query is one that might be used in a strategy game to detect all nearby ships within your particular radar range:
_visibleObjects =
from radarObject in _rawRadar
where radarObject
.Distance2DFrom(_myLocation)
<= RADAR_RADIUS
select radarObject;
Such a game could receive a continuous stream of network messages from connected peer applications and store those messages in a ContinuousCollection (such as _rawRadar). The collection _visibleObjects would automatically update everytime an object in the _rawRadar list changed position or objects were added or removed.
Figure 3 shows the results of that query bound to a custom-style listbox to create a WPF-based radar view of nearby enemies.
Continuous LINQ is just one expansion made possible through language extensions and LINQ on the .NET Framework 3.5. By embracing these features of the upcoming version of .NET, you can be ready to not only provide the most advanced features in your applications, but you can be assured that if there are domain-specific query features that you want in your application, you can create them yourself quickly and easily.
You can download the samples used in this article, as well as the CLINQ code itself, from my blog at dotnetaddict .dotnetdevelopersjournal.com.
Terms of Service | Privacy Statement | Copyright © 2024 UBM Tech, All rights reserved.