How to Paginate Through Results with Vibium

To paginate through results with Vibium, scrape the current page with findAll(), click the Next button, wait for the new results to render, and repeat in a loop until Next disappears or is disabled. Vibium auto-waits for each click to be actionable, so you do not scatter fixed sleeps between pages. The two things that make pagination robust are a clean stop condition — checking whether the Next control still exists and is enabled — and a wait for fresh content after each click, so you scrape the new page rather than re-reading the old one. The same loop handles numbered pagers, "Load more" buttons, and infinite scroll with small variations. Because Vibium drives a real Chrome browser, it sees JavaScript-rendered results on every page, and its auto-waiting keeps the loop from racing slow page transitions.

What is the pagination script?

from vibium import browser_sync as browser
 
vibe = browser.launch()
vibe.go("https://example.com/results")
 
all_items = []
while True:
    # Scrape the current page
    for item in vibe.findAll(".result"):
        all_items.append(item.text())
 
    # Stop if there is no Next button left
    next_buttons = vibe.findAll("a.next")
    if not next_buttons:
        break
 
    next_buttons[0].click()
    vibe.find(".result").wait_until("visible")  # wait for the new page
 
print(f"Collected {len(all_items)} items across all pages")
vibe.quit()

This reads every result on the page, checks for a Next link, clicks it, waits for the next page to render, and loops. The if not next_buttons: break is the stop condition — when findAll() returns an empty list, there is no next page and the loop ends.

How does each step work?

1. vibe.findAll(".result") — returns the full list of results on the current page so you can read each one. See find element for selector options.
2. vibe.findAll("a.next") — checks whether a Next control still exists; an empty list means you are on the last page.
3. next_buttons[0].click() — advances to the next page. Vibium scrolls it into view and waits until it is actionable.
4. find(".result").wait_until("visible") — waits for the new page's results to render before the next loop reads them.
5. vibe.quit() — shuts the browser down once every page is collected.

The key to correctness is waiting for fresh content after each click, so each loop iteration scrapes a genuinely new page.

How do I stop on a disabled Next button?

Some pagers keep the Next button in the DOM but disable it on the last page. Check is_enabled() instead of presence:

while True:
    for item in vibe.findAll(".result"):
        all_items.append(item.text())
 
    nxt = vibe.find("button.next")
    if not nxt.is_enabled():
        break
 
    nxt.click()
    vibe.find(".result").wait_until("visible")

is_enabled() returns False when the button carries the disabled attribute, giving you a clean stop signal even when the control never disappears.

How do I handle infinite scroll instead of Next buttons?

Feeds that load more content as you scroll have no Next button. Scroll to the bottom with evaluate(), wait for new items, then re-read — stopping when the count stops growing:

seen = 0
while True:
    vibe.evaluate("window.scrollTo(0, document.body.scrollHeight)")
    vibe.find(".result").wait_until("visible")
 
    items = vibe.findAll(".result")
    if len(items) == seen:
        break  # no new items loaded, we have reached the end
    seen = len(items)
 
texts = [i.text() for i in vibe.findAll(".result")]
print(f"Collected {len(texts)} items")

evaluate() runs JavaScript in the page to scroll to the bottom, which triggers the feed to fetch more. When a scroll loads nothing new, len(items) stops increasing and the loop exits.

How do I avoid re-scraping duplicate rows?

When pages share some items or you only want unique records, collect into a set or de-duplicate by a stable key such as a link:

seen_urls = set()
records = []
 
for item in vibe.findAll(".result"):
    url = item.find("a").attr("href")
    if url in seen_urls:
        continue
    seen_urls.add(url)
    records.append({"title": item.find("h3").text(), "url": url})

Keying on the link keeps your dataset clean even if pagination overlaps or a page reloads results you already captured.

Tips for reliable pagination

• Wait for fresh content after each click — otherwise you re-read the previous page and collect duplicates.
• Add a safety cap — break after a maximum page count so a broken stop condition cannot loop forever.
• Save as you go — write each page's rows to a file inside the loop so a crash mid-crawl does not lose everything.

Next steps

• How to scrape a table with Vibium
• How to extract all links from a page with Vibium
• What is Vibium?