In this article I will introduce a performant solution on how to implement a list with a large amount of data. I will start with displaying a short list and next will suppose that the data set is very large. The last step will be loading this large set from the backend. In each step I will introduce the challenges and the solutions.
Display a list of data
Let’s suppose we have created a new component and inside the component we have items, an array with 10 elements such as: items = Array.from({length: 10}).map((_, i) => `Item #${i}`);
We can easily display them by taking advantage of the Angular Material List Component. Our template will be:
CSS classes container and items are defined as below:
.container
{
height: 200px;
width: 200px;
border: 1px solid black;
overflow: scroll;
}
.item { height: 50px; }We are doing a basic styling, each item will be 50px and the container should be 200px, in case there are more than 4 items, we will let the user scroll.
At this point we can see the list rendered in the browser and the solution is fast.
Display a large list of data
Let’s suppose that our items array will have 100.000 elements and change the items inside the component like this: items = Array.from({length: 100000}).map((_, i) => `Item #${i}`);
By using the same template as before, if we check the browser the solution still works and all the elements are rendered in the browser. That is what we get:
So the solution that seemed perfect for the first requirement, now it comes with performance issues. We are loading 100.000 elements and the user can only see 4 at a time, so our first thought would be why don’t we just load the elements that the user is seeing and probably a couple more. By using this logic, users will not notice any latency when scrolling and the biggest advantage is that we don’t need to render all these elements.
That’s when Virtual Scrolling comes into hand. According to Angular Scrolling Documentati: “The <cdk-virtual-scroll-viewport> displays large lists of elements performantly by only rendering the items that fit on-screen. “
Let’s add ScrollingModule and take advantage of the virtual scrolling. (In order to use ScrollingModule you should install @angular/cdk in your project).
We will replace mat-list with the cdk-virtual-scroll-viewport and *ngFor with *cdkVirtualFor. Now the template will look like this:
The viewport can be customized with different scroll strategies, you can use this array or a custom data source, but I will show just the benefits of virtual scroll and how to do a simple implementation. For more information please refer to the official documentation. This is the result in browser:
There are only 12 elements rendered and if you try it yourself you will see as you scroll down the rendered elements will change, so it will be a much better user experience.
Display a list of data fetched in backend
Now let’s suppose we will load the data from the backend. Even though we are now rendering a small number of items in the browser, if we have to fetch 100.000 items it will take a long time. So let’s suppose we can ask to load only a partial amount of data from the backend, pagination logic. We need to send a request to load the items from a specific index.
For this example, I will create a simple function that will return me the data from a starting index. It will be simple for you to update it in a real case scenario. We will use the scrolledIndexChanged event inside the viewport to get the scroll event and will update the template as below:
Now let me first show you the update component and I will get in details what each part is responsible for:
export class VirtualScrollComponent {
items = [];
pageSize = 10;
fetchedRanges = new Set<number>(); @ViewChild(CdkVirtualScrollViewport) virtualScroll: CdkVirtualScrollViewport; constructor() { } onScroll(): void {
const renderedRange = this.virtualScroll.getRenderedRange();
const end = renderedRange.end;
const total = this.items.length;
const nextRange = end + 1;
if (end == total && !this.fetchedRanges.has(nextRange)) {
this.items = [...this.items, ...this.loadData(nextRange, this.pageSize)];
this.fetchedRanges.add(nextRange);
}
}
private loadData(start: number, size: number): string[]
{
if (start === 100000) {
return [];
}
return this.range(start, size);
} private range(start: number, size: number): string[] {
return [...Array(size).keys()].map(i => `Item #${i + start}`); }
}
When the user scrolls, we will check if the rendered range is the last loaded range. In order to avoid multiple calls to our “backend” function we will use a set fetchedRanges, where we will keep all the loaded ranges. Let’s suppose the user is scrolling in index 5, we go and check that this is the last loaded range and ask to load the next range. Meanwhile, when the next range is loading, if the user continues to scroll and if we don’t save in the set that we asked for range 2, it will go and ask the backend again to get the next range. The set will make more sense when you apply it in a real application.
The loadData function is returning a piece of data and doing a simple check if the index we are asking is 100.000 and if so will return an empty array supposing that we loaded all the data. Once we return the empty array, we will not execute the load data function, because the set will have the index 100.000.
Conclusions
If you are dealing with large data sets virtual scrolling will help you render only a small amount of items and it manages to update the view when the user scrolls. If you are getting this large amount of data from the backend Infinite Scrolling will allow you to do so, without affecting the user experience.
Originally published at http://agdev.tech on February 6, 2021.
