In this elastic search tutorial, we discuss about Paginating the search results or search result Pagination. This is part of Query DSL (Domain Specific Language)
Sections:
Simple Pagination using from and size (0:00)
Scaling problem with from and size pagination (3:00)
Elasticsearch Pagination Scroll Request (5:05)
Scaling problem with scroll pagination request (9:00)
Search after pagination in Elasticsearch (9:55)
Things to note about Elasticsearch search after pagination API (12:32)
References: [ Ссылка ]
Playlist Link: [ Ссылка ]
Hashtags:
#coding #theory #computerscience #elasticsearch #clusters #distributedSystems #tutorial #logstash #kibana #beats #aws #dataScience #pagination #queryDSL
Places where these ideas can be used:
- To paginate the search results using ElasticSearch
- To build infinite scroll using ElasticSearch
Some Theory:
Elasticsearch is a search engine based on the Lucene library. It provides a distributed, multitenant-capable full-text search engine with an HTTP web interface and schema-free JSON documents.
To paginate through a larger set of results, you can use the search API’s size and from parameters. The size parameter is the number of matching documents to return. The from parameter is a zero-indexed offset from the beginning of the complete result set that indicates the document you want to start with.
While a search request returns a single “page” of results, the scroll API can be used to retrieve large numbers of results (or even all results) from a single search request, in much the same way as you would use a cursor on a traditional database.
Pagination of results can be done by using the from and size but the cost becomes prohibitive when the deep pagination is reached. The index.max_result_window which defaults to 10,000 is a safeguard, search requests take heap memory and time proportional to from + size. The scroll API is recommended for efficient deep scrolling but scroll contexts are costly and it is not recommended to use it for real time user requests. The search_after parameter circumvents this problem by providing a live cursor. The idea is to use the results from the previous page to help the retrieval of the next page.
(This tutorial is part of a series of tutorials on Elasticsearch, logstash and Kibana. It uses docker for purpose of installation, and may even use aws in the future.)
