Django cursor pagination

A cursor based pagination system for Django. Instead of refering to specific pages by number, we give every item in the queryset a cursor based on its ordering values. We then ask for subsequent records by asking for records after the cursor of the last item we currently have. Similarly we can ask for records before the cursor of the first item to navigate back through the list.

This approach has two major advantages over traditional pagination. Firstly, it ensures that when new data is written into the table, records cannot be moved onto the next page. Secondly, it is much faster to query against the database as we are not using very large offset values.

There are some significant drawbacks over "traditional" pagination. The data must be ordered by some database field(s) which are unique across all records. A typical use case would be ordering by a creation timestamp and an id. It is also more difficult to get the range of possible pages for the data.

The inspiration for this project is largely taken from this post by David Cramer, and the connection spec for Relay GraphQL. Much of the implementation is inspired by Django rest framework's Cursor pagination.. The main difference between the Disqus approach and the one used here is that we require the ordering to be totally determinate instead of using offsets.

Installation

pip install django-cursor-pagination

Usage

from cursor_pagination import CursorPaginator

from myapp.models import Post


def posts_api(request, after=None):
    qs = Post.objects.all()
    page_size = 10
    paginator = CursorPaginator(qs, ordering=('-created', '-id'))
    page = paginator.page(first=page_size, after=after)
    data = {
        'objects': [serialize_page(p) for p in page],
        'has_next_page': page.has_next,
        'last_cursor': paginator.cursor(page[-1])
    }
    return data


async def posts_api_async(request, after=None):
    qs = Post.objects.all()
    page_size = 10
    paginator = CursorPaginator(qs, ordering=('-created', '-id'))
    page = await paginator.apage(first=page_size, after=after)
    data = {
        'objects': [serialize_page(p) for p in page],
        'has_next_page': page.has_next,
        'last_cursor': paginator.cursor(page[-1])
    }
    return data

Reverse pagination can be achieved by using the last and before arguments to paginator.page.

Caveats

• The ordering specified must uniquely identify the object.
• If a cursor is given and it does not refer to a valid object, the values of has_previous (for after) or has_next (for before) will always return True.
• NULL comes at the end in query results with ORDER BY both for ASC and DESC.