Skip to main content

Troubleshooting Bambuser Product Scraper Issues

Troubleshooting: Product Scraper​

If you are encountering issues with Bambuser's Product Scraper when adding product URLs, follow the steps below to diagnose and resolve the problem.

Test our product data scraper

If you do not have access to Bam Hub, you can still test how our product scraper will behave with your product URLs. Visit this page and enter your product URL to test.

How to Open the Browser Console​

To effectively troubleshoot, you may need to inspect the console and network activity in your browser.

  • Chrome/Edge: Right-click on the page, select Inspect, and go to the Console or Network tab.
  • Firefox: Right-click on the page, select Inspect Element, and open the Console or Network tab.
  • Safari: Enable Develop Mode in Preferences, then use Develop > Show Web Inspector.

Common Causes and Solutions​

1. Invalid or Inaccessible Product URL​

Cause:
The provided product URL is invalid, inaccessible, or blocked by restrictions such as region-based access or firewalls.

Solution:

  • Verify that the URL is publicly accessible from the United States, as the scraper operates from US-based servers. Test the URL in our product scraper testing tool.
  • If your environment is restricted (e.g., staging or test URLs), consider whitelisting Bambuser’s scraper. See the Whitelist the Scraper section for more details.

2. Empty or Incorrect Fields After Scraping​

Cause:
Fields like product name, price, or images are scraped incorrectly due to invalid metadata or unconventional page layouts.

Solution:

As a workaround, you can manually update product details in Bambuser, just note to not click "refresh" button after manually editing information. Otherwise, it will re-scrape old information again

3. Scraper Blocked by CSP or Firewall​

Cause:
Your Content Security Policy (CSP) or firewall may block requests from Bambuser's scraper.

Solution:

  • Open the Console tab in DevTools and look for CSP or network errors.
  • Allow the following User-Agent or IP addresses in your CSP or firewall rules:
    • User-Agent: BambuserLiveShopping/1.0
    • Global IP: 35.224.84.15
    • EU IP: 35.240.106.166
  • For more details, see Whitelist the Scraper.

4. Incorrect Data Due to Dynamic Rendering​

Cause:
Structured data or metadata is dynamically loaded after the initial page response, making it inaccessible to the scraper.

Solution:

  • Verify if the structured data is in the initial HTML by right-clicking on the product page and selecting View Page Source.
    • If the structured data appears in the source code, it is backend rendered.
    • If not, it may be dynamically loaded via JavaScript, which can prevent the scraper from accessing it.
  • Ensure the page is server-side rendered (not JavaScript-rendered) for the scraper to access the data.

5. Thumbnail or Images Not Dipslaying Blank/White​

Cause:
The product images are blocked due to restrictions on your CDN or hosting platform. Read this article to get more info.


Whitelist the Scraper​

By User-Agent​

The scraper identifies itself using the following User-Agent:

``BambuserLiveShopping/1.0`

Allow this User-Agent in your firewall, CDN, or server configurations to ensure successful scraping.

By Static IP Address​

Whitelist the following static IP addresses if additional restrictions are in place:

  • Global server: 35.224.84.15
  • EU server: 35.240.106.166
Static IP Activation

You must inform Bambuser staff to enable the 'Static IP Proxy' for your organization in addition to whitelisting these IPs.