Skip to main content

Cookies on UKHSA data dashboard

We use some essential cookies to make this service work.

We'd like to set additional cookies so we can remember your settings, understand how people use the service and make improvements.

View cookies
UKHSA data dashboard

BetaThis is a new service - your feedback will help us to improve it.

Access our data

Welcome to the UKHSA data dashboard API developer's guide. Here, you'll find concise instructions for interacting with our API and comprehensive Swagger documentation detailing endpoint information, parameters, and example responses.

Examples: API Pagination

Last updated on Monday, 2 September 2024 at 03:45pm

API Pagination

API pagination is a technique used in API design. It allows the retrieval of large data sets to be broken down into a more manageable size. It does this by allowing the data requested from a large dataset to be divided up into smaller chunks or pages. The UKHSA data dashboard follows this approach with its public API, this page includes examples of how to interact with a paginated API.

Pagination meta data

The UKHSA data dashboard uses a page-based approach to pagination. This technique specifies a page number and a number for how many records to display on a page. As part of its response, it includes meta data that provides the following information:

  • count: the number of records available in this request
  • next: the URL to the next page of results if one is available
  • previous: the URL. to the previous page of results if one is available

This meta data can be used to help us understand how many records are available as well as indicate if there are further pages of results. If we take the number of records and divide that number by the page size, which by default is 5 we'll be able to calculate how many pages of results there are.

For example a data set that includes 100 records using the default page size of 5 will include 20 pages of results, which requires 20 requests to the API to return the entire data set.

{
   "count": 1533,
   "next": "https://api.ukhsa-dashboard.data.gov.uk/themes/infectious_disease/sub_themes/respiratory/topics/COVID19/geography_types/Nation/geographies/England/metrics/COVID-19_cases_casesByDay?page=2",
   "previous": null,
   "results": [
       {
           "theme": "infectious_disease",
           "sub_theme": "respiratory",
           "topic": "COVID-19",
           "geography_type": "Nation",
           "geography": "England",
           "geography_code": "E92000001",
           "metric": "COVID-19_cases_casesByDay",
           "metric_group": "cases",
           "stratum": "default",
           "sex": "all",
           "age": "all",
           "year": 2020,
           "month": 1,
           "epiweek": 5,
           "date": "2020-01-30",
           "metric_value": 1.0
       },
           // results continued...
   ]
}

Query parameters

Meta data provides information that enables us to better understand the APIs response data and query parameters can be used to improve how we request that data. There are a number of query parameters covered in the filtering section of access our data documentation, but 2 parameters in particular are useful when interacting with pagination. The page_size and the page parameter.

The page_size parameter enables us to increase the number of results returned per page. By default, this number is 5 but can be increased up to 365. This is achieved by adding the query parameter to the end of the request page_size=365

The page parameter allows us to request a specific page of results by applying the page parameter to the request in the following format ?page=2

url = "https://api.ukhsa-dashboard.data.gov.uk/themes/infectious_disease/sub_themes/respiratory/topics/COVID-19/geography_types/Nation/geographies/England/metrics/COVID-19_cases_casesByDay"


# The requests library is used to make an HTTP request
# included is the page_size parameter to increase the page size
# the response is then parsed as JSON
response = requests.get(url, params={"page_size": 365}).json()
results = response["results"]


# The results of the request are then printed to standard out
# with Python's print() function.
print(results)

Example: single request with custom page number

The following example makes a single request to the public API using a query parameter to retrieve the second page of results from our first example with a page size of 365.

url = "https://api.ukhsa-dashboard.data.gov.uk/themes/infectious_disease/sub_themes/respiratory/topics/COVID-19/geography_types/Nation/geographies/England/metrics/COVID-19_cases_casesByDay"


# The requests library is used to make an HTTP request
# the response is then parsed as JSON
response = requests.get(url, params={"page_size": 365, "page": 2}).json()
results = response["results"]


# The results of the request are then printed to standard out
# with Python's print() function.
print(results)

Example: create a function that requests time series slice

The following example creates a function that makes a series of requests to retrieve all data for a particular data set. To do this we utilise the next property from the meta data we receive in each response. Our function stores the results from each request in a variable called data, because each call will receive the url for the following page of results the only query parameter we'll need is the page_size.

def get_time_series_data(url: str) -> list[dict[str, str]:


    data = []


    while url:
        # the requests package is used to make an HTTP request to our current URL
        response = requests.get(url, params={"page_size": 365})


        if not response.ok:
            raise Exception("Error, request failed.")




        # we parse our response as JSON
        response_data = response.json()


        # then take the next and results items from the response object
        results = response_data["results"]
        next_url = response_data["next"]


        data.extend(results)
        url = next_url


    return data




data = get_time_series_data(url)

Related content

Back to top