Angular 6 ElasticSearch example – Documents Pagination with Scroll

In the post, we had known how to get All Documents in Index. This Angular 6 Elasticsearch tutorial shows you way to do pagination with scroll.

Next: Angular 6 ElasticSearch example – simple Full Text Search

Elasticsearch Tutorials:
Elasticsearch Overview
ElasticSearch Filter vs Query
ElasticSearch Full Text Queries – Basic

Elasticsearch Documents Pagination with Scroll

Add ElasticSearch to Angular 6 Project

Please visit Angular 6 ElasticSearch – Quick Start – How to add Elasticsearch.js for details.

With the post, you will know how to:
– Add ElasticSearch to Angular 6 Project
– Use it in the project

Add Document & Get All Documents in Index

– Add Document:
>> Please visit Angular 6 ElasticSearch example – Add Document to Index.

angular-6-elasticsearch-example-add-document-to-index-add-document

– Get All Documents:
>> Please visit Angular 6 ElasticSearch example – Get All Documents in Index.

angular-6-elasticsearch-example-show-documents-in-index-result

Get All Documents in Index with Scroll
Search with Scroll Param

In the Client.search() function, we add a SearchParam: scroll, set size for number of items per page and sort by id:

import { Client } from 'elasticsearch-browser';

@Injectable()
export class ElasticsearchService {

  private client: Client;

  getAllDocumentsWithScroll(_index, _type, _size): any {
    return this.client.search({
      index: _index,
      type: _type,
      // Set to 1 minute because we are calling right back
      // (Elasticsearch keeps the search context open for another 1m)
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id'],
      body: {
        'size': _size,
        'query': {
          'match_all': {}
        },
        'sort': [
          { '_uid': { 'order': 'asc' } }
        ]
      }
    });
  }
}

Assume that _size is 3, the response of search() function should be like:

TRACE: 2018-08-16T10:07:49Z
  -> POST http://localhost:9200/gkz_index/customer/_search?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "size": 3,
    "query": {
      "match_all": {}
    },
    "sort": [
      {
        "_uid": {
          "order": "asc"
        }
      }
    ]
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAABFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAAhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAAMWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            "fullname": "David Louis",
            "age": 29,
            "address": "P.O. Box 399 4275 Amet Street\nWest Allis NC 36734",
            "published": "15/08/2018, 10:32:23"
          }
        },
        {
          "_source": {
            "fullname": "Jamie Konan",
            "age": 33,
            "address": "Ap #443-336 Ullamcorper. Street\nVisalia VA 54886",
            "published": "15/08/2018, 10:32:47"
          }
        },
        {
          "_source": {
            "fullname": "Katherin Kohen",
            "age": 22,
            "address": "1964 Facilisis Avenue\nBell Gardens Texas 87065",
            "published": "15/08/2018, 10:33:09"
          }
        }
      ]
    }
  }

We will use _scroll_id to get documents in next page.

Scroll to next page
  // ...

  getNextPage(scroll_id): any {
    return this.client.scroll({
      scrollId: scroll_id,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id']
    });
  }

The response of search() function will contain next 3 (or less than 3) documents and should be like:

TRACE: 2018-08-16T10:08:43Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAABFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAAhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAAMWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAABFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAAhZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAAAMWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            "fullname": "Keegan Blair",
            "age": 38,
            "address": "Ap #761-2515 Egestas. Rd.\nManitowoc TN 07528",
            "published": "16/08/2018, 10:04:16"
          }
        },
        {
          "_source": {
            "fullname": "Katelyn Cooper",
            "age": 32,
            "address": "6059 Sollicitudin Road\nBurlingame Colorado 26278",
            "published": "16/08/2018, 10:07:07"
          }
        },
        {
          "_source": {
            "fullname": "Jack Smith",
            "age": 25,
            "address": "P.O. Box 902 3472 Ullamcorper Street\nLynchburg DC 29738",
            "published": "16/08/2018, 10:09:54"
          }
        }
      ]
    }
  }

Practice

0. Overview

Goal:

angular-6-elasticsearch-example-pagination-result

>> Click Next button for next page.

1. Generate Service & Components

Run cmd:
ng g s elasticsearch
ng g c customer/add-customer
ng g c customer/customer-details
ng g c customer/show-customers

=> Project Structure:

angular-6-elasticsearch-example-pagination-scroll-project-structure.png

2. App Module

app.module.ts

import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FormsModule, ReactiveFormsModule } from '@angular/forms';
import { AppRoutingModule } from './app-routing.module';

import { AppComponent } from './app.component';
import { AddCustomerComponent } from './customer/add-customer/add-customer.component';
import { CustomerDetailsComponent } from './customer/customer-details/customer-details.component';
import { ShowCustomersComponent } from './customer/show-customers/show-customers.component';

@NgModule({
  declarations: [
    AppComponent,
    AddCustomerComponent,
    CustomerDetailsComponent,
    ShowCustomersComponent
  ],
  imports: [
    BrowserModule,
    FormsModule,
    ReactiveFormsModule,
    AppRoutingModule
  ],
  providers: [],
  bootstrap: [AppComponent]
})
export class AppModule { }
3. ElasticSearch Service

elasticsearch.service.ts

import { Injectable } from '@angular/core';
import { Client } from 'elasticsearch-browser';

@Injectable({
  providedIn: 'root'
})
export class ElasticsearchService {

  private client: Client;

  private queryalldocs = {
    'query': {
      'match_all': {}
    }
  };

  constructor() {
    if (!this.client) {
      this.connect();
    }
  }

  private connect() {
    this.client = new Client({
      host: 'http://localhost:9200',
      log: 'trace'
    });
  }

  isAvailable(): any {
    return this.client.ping({
      requestTimeout: Infinity,
      body: 'hello ozenero!'
    });
  }

  addToIndex(value): any {
    return this.client.create(value);
  }

  getAllDocuments(_index, _type): any {
    return this.client.search({
      index: _index,
      type: _type,
      body: this.queryalldocs,
      filterPath: ['hits.hits._source']
    });
  }

  getAllDocumentsWithScroll(_index, _type, _size): any {
    return this.client.search({
      index: _index,
      type: _type,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id'],
      body: {
        'size': _size,
        'query': {
          'match_all': {}
        },
        'sort': [
          { '_uid': { 'order': 'asc' } }
        ]
      }
    });
  }

  getNextPage(scroll_id): any {
    return this.client.scroll({
      scrollId: scroll_id,
      scroll: '1m',
      filterPath: ['hits.hits._source', 'hits.total', '_scroll_id']
    });
  }
}
4. Components

customer/customer.interface.ts

export interface Customer {
    fullname: string;
    age: number;
    address: string;
    published: string;
}

export interface CustomerSource {
    source: Customer;
}
4.1 Add Document Component

Please visit: 4_Add_Document_Component (in the previous post) for details.

4.2 Details Component

customer-details.component.ts

import { Component, OnInit, Input } from '@angular/core';
import { Customer } from '../customer.interface';

@Component({
  selector: 'customer-details',
  templateUrl: './customer-details.component.html',
  styleUrls: ['./customer-details.component.css']
})
export class CustomerDetailsComponent implements OnInit {

  @Input() customer: Customer;

  constructor() { }

  ngOnInit() {
  }

}

customer-details.component.html

{{customer.fullname}}
{{customer.age}}
{{customer.address}}
{{customer.published}}

4.3 Show Documents Component

show-customers.component.ts

import { Component, OnInit } from '@angular/core';
import { CustomerSource } from '../customer.interface';
import { ElasticsearchService } from '../../elasticsearch.service';

@Component({
  selector: 'show-customers',
  templateUrl: './show-customers.component.html',
  styleUrls: ['./show-customers.component.css']
})
export class ShowCustomersComponent implements OnInit {

  private static readonly INDEX = 'gkz_index';
  private static readonly TYPE = 'customer';
  private static readonly SIZE = 3;

  customerSources: CustomerSource[];
  haveNextPage = false;
  scrollID = '';
  notice = '';

  constructor(private es: ElasticsearchService) {
    this.scrollID = '';
    this.notice = '';
    this.haveNextPage = false;
  }

  ngOnInit() {
    this.es.getAllDocumentsWithScroll(
      ShowCustomersComponent.INDEX,
      ShowCustomersComponent.TYPE,
      ShowCustomersComponent.SIZE).then(
        response => {
          this.customerSources = response.hits.hits;

          if (response.hits.hits.length < response.hits.total) {
            this.haveNextPage = true;
            this.scrollID = response._scroll_id;
          }
          console.log(response);
        }, error => {
          console.error(error);
        }).then(() => {
          console.log('Show Customer Completed!');
        });
  }

  showNextPage() {
    this.es.getNextPage(this.scrollID).then(
      response => {
        this.customerSources = response.hits.hits;
        if (!response.hits.hits) {
          this.haveNextPage = false;
          this.notice = 'There are no more Customers!';
        }
        console.log(response);
      }, error => {
        console.error(error);
      }).then(() => {
        console.log('Show Customer Completed!');
      });
  }
}

show-customers.component.html

{{notice}}

age" class="btn btn-primary">Next
5. App Routing Module

app-routing.module.ts

import { AddCustomerComponent } from './customer/add-customer/add-customer.component';
import { ShowCustomersComponent } from './customer/show-customers/show-customers.component';

import { NgModule } from '@angular/core';
import { RouterModule, Routes } from '@angular/router';

const routes: Routes = [
    { path: '', redirectTo: 'add', pathMatch: 'full' },
    { path: 'add', component: AddCustomerComponent },
    { path: 'customers', component: ShowCustomersComponent }
];

@NgModule({
    imports: [RouterModule.forRoot(routes)],
    exports: [RouterModule]
})
export class AppRoutingModule { }
6. App Component

app.component.ts

import { Component } from '@angular/core';

@Component({
  selector: 'app-root',
  templateUrl: './app.component.html',
  styleUrls: ['./app.component.css']
})
export class AppComponent {
  title = 'ozenero';
  description = 'Angular - Elasticsearch';
}

app.component.html

{{title}}

{{description}}

7. Check Result

Run Angular 6 App, go to http://localhost:4200/, add Customer Data, then choose Customers tab:

angular-6-elasticsearch-example-pagination-result

Click Next button several times to view more documents:

angular-6-elasticsearch-example-pagination-result-new-page

Open Browser Console, you can see:

TRACE: 2018-08-16T10:16:27Z
  -> POST http://localhost:9200/gkz_index/customer/_search?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "size": 3,
    "query": {
      "match_all": {}
    },
    "sort": [
      {
        "_uid": {
          "order": "asc"
        }
      }
    ]
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            // customer-1
          }
        },
        {
          "_source": {
            // customer-2
          }
        },
        {
          "_source": {
            customer-3
          }
        }
      ]
    }
  }
------------------------
Show Customer Completed!
TRACE: 2018-08-16T10:16:52Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            // customer-4
          }
        },
        {
          "_source": {
            // customer-5
          }
        },
        {
          "_source": {
            // customer-6
          }
        }
      ]
    }
  }
------------------------
Show Customer Completed!
TRACE: 2018-08-16T10:16:54Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8,
      "hits": [
        {
          "_source": {
            // customer-7
          }
        },
        {
          "_source": {
            // customer-8
          }
        }
      ]
    }
  }

When there is no more document:

TRACE: 2018-08-16T10:21:09Z
  -> POST http://localhost:9200/_search/scroll?scroll=1m&filter_path=hits.hits._source%2Chits.total%2C_scroll_id
  {
    "scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ=="
  }
  <- 200
  {
    "_scroll_id": "DnF1ZXJ5VGhlbkZldGNoAwAAAAAAAAATFmdNTC1zZTQ4VG4yZ0pGc2hVVV92ZFEAAAAAAAAAFBZnTUwtc2U0OFRuMmdKRnNoVVVfdmRRAAAAAAAAABUWZ01MLXNlNDhUbjJnSkZzaFVVX3ZkUQ==",
    "hits": {
      "total": 8
    }
  }

Sourcecode

Angular6Elasticsearch-pagination-scroll

4 thoughts on “Angular 6 ElasticSearch example – Documents Pagination with Scroll”

  1. Magnificent goods from you, man. I have understand your stuff previous
    to and you are just extremely excellent. I actually like what you’ve acquired here, really like what you are stating and the way in which you say it.
    You make it enjoyable and you still take care of to keep it sensible.
    I can’t wait to read far more from you. This is really a terrific site.

Leave a Reply

Your email address will not be published. Required fields are marked *