What You Will Need
- Visual Studio 2022
- ComponentOne Data Services
Controls Referenced
Tutorial Concept
Shows how to implement on-demand loading for a .NET datagrid application. Data virtualization is a great solution for handling very large data sets.
The ComponentOne DataCollection (C1DataCollection) is a data management library that enables common data transformations, compositions, and virtualization operations for data-bound UI controls.
Basically, it enables features like sorting, filtering, grouping, editing, incremental loading and refreshing. C1DataCollection is the “brains” behind our most powerful UI components, including FlexGrid, FlexPivot, and DataFilter. Its separation from the UI components enables powerful data features in more controls like ListViews and TreeViews. One of the most useful features it provides is on-demand loading with data virtualization.
In this blog, we’ll cover the basics of on-demand loading with data virtualization, give an overview of the C1DataCollection libraries, and explore how to improve the performance of your .NET application by implementing on-demand loading with data virtualization in the sections below:
- How Data Virtualization Helps Load Large Data Sets
- How to Implement Virtual, On-Demand Loading
- Working with ADO.NET
- Working with Entity Framework Core
- Working with a Proxy Data Source
- Working with a Different Data Source
- Other Collection Types Included in C1DataCollection
- Additional Considerations
How Data Virtualization Helps Load Large Data Sets
Data virtualization is a broad term for data management that allows an application to retrieve and manipulate data without requiring technical details about the data. This can encompass many things, but one of the most common use cases is handling large data sets.
In most scenarios today, our data comes from remote sources. This introduces a cost in terms of network usage and the time needed to load all the data. We can’t expect to load all the data from the server-side database to the client in every situation.
With data virtualization, we can understand the entire data set but only load a smaller portion (or page) of it at once. Understanding the data means we can filter, sort, and scroll the data on the client as if the entire data set existed on the client even though a subsection is all that we retrieved. This enables what we commonly refer to as on-demand or incremental loading.
Check out a live demo using FlexGrid for Blazor.
How to Implement Virtual, On-Demand Loading
The C1.DataCollection.* libraries include several types of collections for different data scenarios. The C1VirtualDataCollection base class provides the incremental, on-demand loading we can use to efficiently manage large data sets. The library also includes several implementations of this base class that you can directly use with minimal effort.
- C1AdoNetVirtualDataCollection
- C1EntityFrameworkCoreVirtualDataCollection
- C1ProxyDataCollection
All three implement C1VirtualDataCollection for asynchronously loading data under different circumstances. They support modifying or editing the data as well as server-side sorting and data filtering. They work best with ComponentOne FlexGrid.
C1AdoNetVirtualDataCollection extracts and modifies the data using a System.Data.Common.DbConnection. C1EntityFrameworkCoreVirtualDataCollection extracts and modifies the data using a Microsoft.EntityFrameworkCore.DbContext. Both are useful for standard .NET data sets including SQL Server, OData, CSV, etc. The difference comes down to the underlying data technology you prefer to use (see ADO.NET vs Entity Framework (EF)). Generally speaking, EF Core is newer, so it’s recommended for new projects.
The C1ProxyDataCollection is useful for server/client apps where the client does not have direct access to the database. We’ll cover more on this one later.
Steps to Implement On-Demand Loading with C1DataCollection:
- Determine which data access technology works best for you and add the corresponding C1.DataCollection library.
- Implement the DbConnection or DbContext to your data source.
- Instantiate the C1.DataCollection by passing in your DbConnection/DbContext.
- Set it as the data source (or ItemsSource) of your FlexGrid.
There are no extra steps required to implement the on-demand loading if you’re using one of our completed implementations. Next, we’ll review some code snippets and samples for each data technology.
Working with ADO.NET
The C1AdoNetVirtualDataCollection lives in the C1.DataCollection.AdoNet library.
using C1.DataCollection.AdoNet;
// Implement DbConnection for OData
var db = new C1.AdoNet.Odata.C1ODataConnection($@"Url={ODataServerUrl};Max Page Size=50");
// Instantiate C1DataCollection
var collection = new C1AdoNetCursorDataCollection<Invoice>(db, "Invoices");
// Set data source on datagrid
flexGrid1.ItemsSource = collection
The System.Data.Common.DbConnection is also an abstract class that requires an implementation. The good news is that ComponentOne Data Connectors includes several implementations for you, including OData, CSV, and JSON, or you can source your own.
Check out the following complete samples using C1AdoNetVirtualDataCollection:
Note, when working with WinForms, you’ll need to use the additional C1DataCollectionBindingList component which acts as a binding bridge between FlexGrid and the C1DataCollection. Make sure to add the C1.DataCollection.BindingList package.
// WinForms requires use of the binding list too
c1FlexGrid1.DataSource = new C1DataCollectionBindingList(collection);
Working with Entity Framework Core
The C1EntityFrameworkCoreVirtualDataCollection lives in the C1.DataCollection.EntityFrameworkCore library.
using C1.DataCollection.EntityFrameworkCore;
// Implement DbContext
var db = new PersonDbContext();
// Instantiate C1DataCollection
var collection = new C1EntityFrameworkCoreVirtualDataCollection<Person>(db);
// Set data source on datagrid
flexGrid1.ItemsSource = collection;
Obviously, this code leaves out the DbContext implementation which is unique to your data source. Check out the Microsoft documentation, or you can see a complete sample here: WPF FlexGrid SQLiteDataBase Sample.
Just like ADO.NET, data filtering happens completely on the server. The passed filterExpression is converted to a Linq expression, and this Linq expression is then converted to a SQL expression by EF.
Note, when working with WinForms, you’ll need to use the additional C1DataCollectionBindingList component which acts as a binding bridge between FlexGrid and the C1DataCollection.
// WinForms requires use of the binding list too
c1FlexGrid1.DataSource = new C1DataCollectionBindingList(collection);
Working with a Proxy Data Source
The C1ProxyDataCollection is part of the C1.DataCollection.SignalR.Client library. It’s useful in a client/server application when you can’t connect directly to the data source (i.e., SQL Server) from the client.
The C1ProxyDataCollection can be used when you have control over the server, and the application has a client/server architecture, like Blazor WebAssembly, WPF, or WinForms, with an ASP.NET backend. The C1DataCollectionHub class, from the C1.DataCollection.SignalR.Server library, can be used to create a SignalR hub to expose a collection of objects. When the collection is changed on the server, every client connected to the server is notified and the UI gets updated automatically.
You can check out and download a complete C1ProxyDataSource sample for WinForms, WPF, and Blazor WASM here.
Working with a Different Data Source
The above solutions work with standard .NET data access techniques. If your data source has a different shape, like a REST API or web service, you may need to implement your own C1VirtualDataCollection. The key functionality to override is the GetPageAsync method, which populates the data page by page as the user scrolls through the collection.
using C1.DataCollection;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
namespace C1DataCollection101
{
public class MyVirtualModeDataCollection : C1VirtualDataCollection<Customer>
{
public int TotalCount { get; set; } = 1_000;
protected override async Task<Tuple<int, IReadOnlyList<Customer>>> GetPageAsync(int pageIndex, int startingIndex, int count, IReadOnlyList<SortDescription> sortDescriptions = null, FilterExpression filterExpression = null, CancellationToken cancellationToken = default(CancellationToken))
{
await Task.Delay(500, cancellationToken);//Simulates network traffic.
return new Tuple<int, IReadOnlyList<Customer>>(TotalCount, Enumerable.Range(startingIndex, count).Select(i => new Customer(i)).ToList());
}
}
}
You’ll need to implement your own data collection class that implements the C1VirtualDataCollection and overrides GetPageAsync to provide the unique connections to your data source. Then, you can set this to your datagrid or listview as usual.
var collection = new MyVirtualModeDataCollection();
flexGrid1.ItemsSource= collection;
// WinForms
//flexGrid1.DataSource = new C1DataCollectionBindingList(collection);
We consider this paging even if the UX is continuous scrolling rather than pager buttons. Note that the “Virtual” types of collection need to know the full size of the data set, which could be costly or unknown. If you can’t implement the total count (for example, you’re pulling YouTube videos), then you can use the C1CursorDataCollection instead—more on that next.
Download custom samples for WinForms and WPF as part of the C1DataCollection101 sample here.
Other Collection Types Included in C1DataCollection
In addition to C1VirtualDataCollection, there is the C1CursorDataCollection. The cursor collection also loads data on demand as the user scrolls, but it is lighter as it does not need to know the total size of the data collection. When the user reaches the bottom of the list, the next page or portion is loaded. This approach is commonly seen in web image search results.
The cursor approach works by sending a token to fetch the pages sequentially. The received page contains the token to the next one.
There’s also a data collection for traditional paging through a data set. For a tutorial on using the C1PagedDataCollection component, check out our previous blog: How to Add Data-Virtualized Paging in a WinForms DataGrid.
The other data collections within C1.DataCollection are data transforming views. The main star of the library is C1DataCollection, and it’s a good go-to for common data transformations as it combines sorting, grouping, and filtering into one. You can see the full list of collection types here.
Additional Considerations
There are some important considerations to make regarding the C1DataCollection data virtualization features. Below are some additional tips and useful information:
- The features described in this article work best (and require minimal code) with FlexGrid in any .NET platform.
- For WinForms, be sure to use the C1DataCollectionBindingList as demonstrated earlier. Add the C1.DataCollection.BindingList package to find this component.
- For the standard WinUI ListView control, it is necessary to use C1CollectionView (from C1.WinUI.DataCollection), since WinUI uses the VectorChanged event from IObservableVector (instead of standard INotifyCollectionChanged) to get notified of changes in the underlying source. Check out a sample.
- For the standard WPF DataGrid, you need to set the “Mode” property on the data collection to Manual and handle the OnScroll event to load the data asynchronously. This is because when you scroll the entire grid, all the data gets accessed, which defeats the purpose of data virtualization. Below is a code snippet where “collection” is a C1VirtualDataCollection implementation.
collection.Mode = C1.DataCollection.VirtualDataCollectionMode.Manual;
grid.ItemsSource = collection;
private void OnScrollChanged(object sender, ScrollChangedEventArgs e)
{
collection.LoadAsync((int)e.VerticalOffset, (int)(e.VerticalOffset + e.VerticalHeight));
}
The C1DataCollection library can be licensed with any ComponentOne license! So, get started by downloading a free trial!
Top comments (0)