Skip to main content
Always paginate. Even if your test environment has 10 users, production might have 10,000. The SDK coordinates pagination for you, so use it from the start.

Basic pattern

Every List, Entitlements, and Grants method returns one page of results and a token for the next page. The SDK passes the token you returned back to your next call in opts.PageToken.Token:
The SDK calls your method repeatedly until you return an empty token, so your job is to fetch and convert one page at a time. The examples on this page use the V2 method signatures. The V1 interface differs only in how it passes and returns the token. See V1 syncer differences.
Use the API’s next token to decide when to stop, not the result count. Some APIs return empty pages before the final page, so len(results) < pageSize isn’t reliable. Return an empty token only when the API signals there are no more pages.

Page size

The SDK passes the page size it wants in opts.PageToken.Size, which is 0 when the SDK doesn’t specify one. The SDK asks for at most 250 items per page. Use it when your API takes a page size, and fall back to a default:

Pagination strategies

APIs paginate results differently. Your connector converts the token to whatever the API expects:

Cursor-based

When the API returns an opaque cursor, pass it straight through as the next page token:

Offset-based

Store the next offset as the token:
If the API has no total count, a full page (len(resp.Users) == limit) is a fallback signal. That fallback costs one extra request when the last page is exactly full. For APIs that return the next page in an RFC 5988 Link header, the uhttp package parses the header for you. uhttp.WithNextLinkPagination pushes the rel="next" URL onto the bag you pass it, so give it a new bag and return that bag as the next token:
On the next call, read the URL back with bag.PageToken() after you unmarshal the token. See Nested pagination with Bag. For APIs that can silently drop their cursor, uhttp.WithPaginationData makes the request fail when a successful response has no pagination data. Without it, the sync ends after one page and looks complete.

LDAP paging

LDAP servers return a paging cookie. Encode it as the token:

Nested pagination with Bag

When a single call needs to page through more than one thing, one string isn’t enough state. For example, “page through the admins of a group, then page through its members” needs to remember which list you’re in and where you are in it. The SDK’s pagination.Bag holds that state as a stack and serializes it into the page token.

How Bag works

A Bag is a stack of PageState values. The top of the stack is the current state:
Prefer NextToken. It does the advance and the serialization in one call, and its return value is the page token you hand back to the SDK.
Push an initial state before you call NextToken. An empty bag has no state to advance, so NextToken starts a new state that holds whatever token you pass, even an empty one. Check bag.Current() == nil after you unmarshal, and push the starting state if it’s nil.

Page through one list

The pattern for a single list is: unmarshal, push an initial state if the bag is empty, fetch with the current token, and advance with NextToken.

Page through several lists

To page through two lists in one method, push one state per list. The state you push last is current, so it’s processed first. When a list runs out, NextToken("") pops its state, and the next call continues with the list below it:
If your state needs more than a token and two labels, pagination.GenBag[T] holds a stack of your own struct type. Create one with pagination.GenBagFromToken[T](&opts.PageToken).

Expired page tokens

Some APIs reject a cursor after a period of time. When the API rejects the token the SDK gave you, return a restart error from List, Entitlements, or Grants instead of failing the sync:
The SDK restarts the call at restartToken. An empty restartToken restarts from the first page. The restart only works when the error comes straight from one of those list calls, and any other source fails the sync.

Modeling hierarchies

Real systems often have deep hierarchies. Declare the parent-child relationships, and the SDK walks the tree for you.

Two mechanisms connect parent and child

1. The parent declares its child types with the ChildResourceType annotation:
2. The child references its parent with WithParentResourceID:
When the SDK calls List for a child type, it passes the parent’s ResourceId, so you can scope your API call to that parent.

Example: GitHub hierarchy

Organization builder:
Repository builder, which receives the parent organization in List:

When to use hierarchies and when to use flat models

Common hierarchy mistakes

Pagination invariants

The SDK checks for common pagination bugs:
  • The token must change. If a page returns the same non-empty token it received, the sync fails with a “next page token unchanged” error. The SDK doesn’t detect longer loops, such as a token that alternates between two values.
  • An empty token means done. Return "" when there are no more pages.
  • Respect the requested page size. Use opts.PageToken.Size when your API takes a page size.
Test with small page sizes (10 to 20 items) during development to verify that pagination works before you test with production-sized datasets.