Skip to content

Online Reviews API

Unwrangle's Data Getter service offers the easiest way to scrape customer reviews from the web.

Getting Started

To scrape online reviews from the platforms we support, you need 2 query parameters:

  1. An API key, available here as api_key

  2. The encoded listing or product URL for which you need the reviews as url

  3. (Optional) The page number from which you want the reviews as page

The following command is an example of how you can get reviews for a listing on a platform we support:

curl -L -X GET 'https://data.unwrangle.com/api/getter/?url=<LISTING_URL>&api_key=<API_KEY>'

import requests

url = "https://data.unwrangle.com/api/getter/?url=<LISTING_URL>&api_key=<API_KEY>"

payload={}
headers = {}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

var axios = require('axios');

var config = {
method: 'get',
url: 'https://data.unwrangle.com/api/getter/?url=<LISTING_URL>&api_key=<API_KEY>',
headers: { }
};

axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});

Unirest.setTimeouts(0, 0);
HttpResponse<String> response = Unirest.get("https://data.unwrangle.com/api/getter/?url=%3CLISTING_URL%3E&api_key=%3CAPI_KEY%3E")
.asString();

require "uri"
require "net/http"

url = URI("https://data.unwrangle.com/api/getter/?url=<LISTING_URL>&api_key=<API_KEY>")

https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true

request = Net::HTTP::Get.new(url)

response = https.request(request)
puts response.read_body

<?php
$client = new http\Client;
$request = new http\Client\Request;
$request->setRequestUrl('https://data.unwrangle.com/api/getter/?url=<LISTING_URL>&api_key=<API_KEY>');
$request->setRequestMethod('GET');
$request->setOptions(array());

$client->enqueue($request)->send();
$response = $client->getResponse();
echo $response->getBody();

package main

import (
"fmt"
"net/http"
"io/ioutil"
)

func main() {

url := "https://data.unwrangle.com/api/getter/?url=%3CLISTING_URL%3E&api_key=%3CAPI_KEY%3E"
method := "GET"

client := &http.Client {
}
req, err := http.NewRequest(method, url, nil)

if err != nil {
    fmt.Println(err)
    return
}
res, err := client.Do(req)
if err != nil {
    fmt.Println(err)
    return
}
defer res.Body.Close()

body, err := ioutil.ReadAll(res.Body)
if err != nil {
    fmt.Println(err)
    return
}
fmt.Println(string(body))
}

The reviews returned will be for the page number specified (default is 1) sorted by recency.

URL

The query parameter url is where the product or listing URL must go, in an encoded format.

Say you want the reviews for this particular Yelp listing: https://www.yelp.com/biz/chateau-tivoli-bed-and-breakfast-san-francisco.

For this the listing URL must be encoded and assigned to the url query parameter.

After encoding it will look like this: https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco.

Encoding the URL is trivially easy in most programming languages. The following snippet showcases how to do this.

sudo apt-get install gridsite-clients
urlencode "LISTING URL"

import urllib.parse
encoded_url = urllib.parse.quote("LISTING URL")

encoded_url = encodeURIComponent("LISTING URL")

String encoded_url = URLEncoder.encode("LISTING URL", "UTF-8");

require 'uri'
encoded_url = URI::encode("LISTING URL")

<?php

$url_encoded = urlencode("YOUR URL");

?>

package main

import (
    "net/url"
)

func main() {
    encoded_url := url.QueryEscape("YOUR URL")
}

Supported Platforms

Platform Expected URL Format (after encoding)
Yelp https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco
Amazon https%3A%2F%2Fwww.amazon.com%2FColumbia-Glennaker-Jacket-Black-Large%2Fdp%2FB00LEX0Y3K%2F
Costco https%3A%2F%2Fwww.costco.com%2Fapple-airpods-pro.product.100525619.html
Target https%3A%2F%2Fwww.target.com%2Fp%2Fapple-airpods-with-charging-case%2F-%2FA-54191097
BestBuy https%3A%2F%2Fwww.bestbuy.com%2Fsite%2Fbose-quietcomfort-earbuds-true-wireless-noise-cancelling-in-ear-headphones-triple-black%2F6419203.p
Home Depot https%3A%2F%2Fwww.homedepot.com%2Fp%2FHDX-100-ft-16-3-Indoor-Outdoor-Extension-Cord-Orange-HD-277-525%2F100650619
Walmart https%3A%2F%2Fwww.walmart.com%2Fip%2FiPhone-Charger-2Pacsk-3-Ft-Charging-Cable-Wall-Power-Adapter-Plug-Block-Compatible-12-pro-max-11-X-8-8-Plus-7-7-Plus-6-6S-6-Plu-5S-SE-Mini-Air-Pro-Ca%2F404907731

We can add support for new platforms within 2 or 3 days in most cases. Please reach out to us at sales@unwrangle.com or on the chat box on our website.

Countries

On Yelp, each country extension is supported (for e.g. https://www.yelp.de/<biz_path>) and our API responds with the default language for that country.

Credits

An API requests costs 10 credits or 1 credit per review, whichever is higher.

Example

An example of a typical request and response is showcased below. 

curl -L -X GET 'https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>'

import requests

url = "https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>"

payload={}
headers = {}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

var axios = require('axios');

var config = {
method: 'get',
url: 'https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>',
headers: { }
};

axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});

Unirest.setTimeouts(0, 0);
HttpResponse<String> response = Unirest.get("https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>")
.asString();

require "uri"
require "net/http"

url = URI("https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>")

https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true

request = Net::HTTP::Get.new(url)

response = https.request(request)
puts response.read_body

<?php
$client = new http\Client;
$request = new http\Client\Request;
$request->setRequestUrl('https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>');
$request->setRequestMethod('GET');
$request->setOptions(array());

$client->enqueue($request)->send();
$response = $client->getResponse();
echo $response->getBody();

package main

import (
"fmt"
"net/http"
"io/ioutil"
)

func main() {

url := "https://data.unwrangle.com/api/getter/?url=https%3A%2F%2Fwww.yelp.com%2Fbiz%2Fchateau-tivoli-bed-and-breakfast-san-francisco&page=7&api_key=<API_KEY>"
method := "GET"

client := &http.Client {
}
req, err := http.NewRequest(method, url, nil)

if err != nil {
    fmt.Println(err)
    return
}
res, err := client.Do(req)
if err != nil {
    fmt.Println(err)
    return
}
defer res.Body.Close()

body, err := ioutil.ReadAll(res.Body)
if err != nil {
    fmt.Println(err)
    return
}
fmt.Println(string(body))
}

For the above request the response looks like this:

{
    "success": true,
    "url": "https://www.yelp.com/biz/chateau-tivoli-bed-and-breakfast-san-francisco",
    "page": 7,
    "total_results": 95,
    "no_of_pages": 10,
    "result_count": 10,
    "reviews": [
        {
            "id": "0NjJB4Yx0Rf8Mjw15TvJlw",
            "date": "2011-03-27",
            "rating": 5,
            "review_title": "",
            "review_text": "My boyfriend surprised me by reserving the Crocker room on my birthday (we were spending several days exploring SF). It was GORGEOUS! The room itself has an amazing ornate ceiling with custom-made paper and crazy intricate molding. There is a spacious closet, and a very clean bathroom. The bathroom has a beautiful stained glass door (still opaque, don't worry!) -- simple but pretty and clean. The bed -- it's my dream bed. It's a stunning four poster canopy bed with charming breezy curtains on three of four sides. This room is on the corner of the house, so there is a rounded nook that is in what looks like a castle turret from the outside. Just so gorgeous, cozy, and homey. The downstairs and outside of the house are the real centerpiece, though -- this has to be the prettiest Painted Lady in San Francisco! The first floor is full of gorgeous antiques and lots of nooks to relax, play chess, listen to the antique radio, etc. Walking distance from lots of great restaurants, mostly located on Haynes. Alamo Square (a gorgeous hilly park) is a few blocks away, great for a walk, walking the dog, a sweet view of the skyline/bay, etc. The brunch on Saturday AM was delicious spinach quiche, fruit, granola, various breads and bagels, coffee/tea, and mimosas! It was yummy and filling, and it was fun to chat with the other guests. Actually it was a full house and a very mixed crowd of all ages. We explored the building after eating and it seemed like every space, even the kitchen, was full of the same intricate detail and molding that the Crocker room had. So beautiful! LOVED IT. Would definitely go back!",
            "review_url": "https://www.yelp.com?hrid=0NjJB4Yx0Rf8Mjw15TvJlw",
            "lang": "en",
            "author_avatar": "https://s3-media0.fl.yelpcdn.com/photo/-Ns_Xi3BNj6ebOUOdMWm1Q/60s.jpg",
            "author_url": "https://www.yelp.com/user_details?userid=-dSJe7sj5VOzixJZa0AMtA",
            "author_name": "Jamila B.",
            "meta_data": {
                "author_contributions": 26,
                "feedback": {
                    "useful": 1,
                    "funny": 0,
                    "cool": 0
                }
            },
            "location": "Houston, TX",
            "response": null
        },
        {
            "id": "O0QbB3IMZdHH3gvCZtDFcQ",
            "date": "2011-03-15",
            "rating": 5,
            "review_title": "",
            "review_text": "Chateau Tivoli is more than a place to stay...it is an experience. The ambiance, charm and character of the place is matched (actually surpassed) by the folks who manage it and your stay here will be one to remember. The neighborhood has improved dramatically and we felt comfortable walking around at night and going to Alamo Square in the morning (just two blocks south) to read the paper and take in the views which are spectacular. While this is a very romantic place for couples, it also caters to many single guests as well so you won't feel like the lone ranger if you come here by yourself. You will just feel lucky. The bed was firm and comfortable while the towels and linens were industrial quality.  Hey, they still worked fine and the bath water was hot in the morning. What more can you ask for...the place seems just like home. If you are fond of older architecture and decorating, this place is a treasure chest with  art nouveau throughout the down stairs, consistent to the time the home was built. The wall papering and wordwork is a marvel just to observe and appreciate. If you want a real San Francisco experience this is the place.The location is right in the center of SF and a short walk or bus ride away from just about anyplace you care to go. We hit North Beach, SOMA, Union Square and Golden Gate Park with little effort. The breakfast and wine/cheese reception are top notch and easily worth $50, making your room more economical. It is very, very quiet inside with very little street noise noticed. Parking was better than imagined during the 3pm check-in and we only had to move the car once for the street sweeping. And the coffee is GREAT!We include a special thanks to Meredith and Nico for making our stay so wonderful and we plan to return soon.",
            "review_url": "https://www.yelp.com?hrid=O0QbB3IMZdHH3gvCZtDFcQ",
            "lang": "en",
            "author_avatar": "https://s3-media0.fl.yelpcdn.com/photo/6tQ1_XUqJa93WFa7PyVUew/60s.jpg",
            "author_url": "https://www.yelp.com/user_details?userid=anALdN7mvj8ia-m4MxE-Ew",
            "author_name": "John B.",
            "meta_data": {
                "author_contributions": 53,
                "feedback": {
                    "useful": 2,
                    "funny": 0,
                    "cool": 0
                }
            },
            "location": "Monterey, CA",
            "response": null
        },
        ...7 more reviews
        {
            "id": "PBpTBnQujWpiH4nzrf_93g",
            "date": "2010-05-24",
            "rating": 5,
            "review_title": "",
            "review_text": "In 1989 I bought a first edition of \"The Painted Ladies\" - a book about the San Francisco Victorians that were being restored. There is an entry in \"The Painted Ladies\" on the Chateau Tivoli, recently restored and just prior to becoming a B&B. For 20+ years I have carried the book around the world with me to remind me of my native San Francisco and its beautiful Victorians. I have at various times over the past 20 years walked past the CT when walking through the Western Addition. If it is possible to stalk a house, then I am guilty. This past weekend, my boy friend and I had the opportunity to stay at the Chateau Tivoli. Most obsessions end badly, this one, ended perfectly! We arrived in time for the Wine & Cheese. Sitting in the front parlor amongst all the Victorian furnishings, my boy friend said I looked more comfortable and at ease there than in my own home. The Inn Keeper, Nico, was helpful and amicable without being saccharin. You can tell he loves the house and enjoys meeting the guests. Upon arrival he gave us a tour of the house and as many of the guest rooms as he could that were open. Beautiful! Just stunningly beautiful. My BF and I were craving sushi and Nico recommended \"Tsunami\" on Broderick and Fulton (please see review for Tsunami). He even made reservations for us. We stayed in the Aimee Crocker room. Accommodations: it seems that unless you get a suite, your bathroom comes with only a shower. As we were going to be at The Chateau for not even 12 hours, we did not splurge this time. Still, our bedroom was comfortable and despite being surrounded by all the antiques, it was all very cozy. It seems every chair in the Chateau is designed for a person to comfortably curl-up in. My BF found our bed to be on the soft side - I thought it perfect. The Chateau keeps extra blankets in a wardrobe down the hall and unless one is staying there during a heat wave, I would suggest grabbing an extra blanket. It is a Victorian and thus, does not heat like a modern house. I have lived in Victorian era buildings most my adult life, so extra layers of clothing or blankets are just the norm to me. But I think others might not be used to how cool it can get. We did not get a chance to partake in the brunch offered at the house. It smelled yummy though. Will I come back? In a heart beat! I want to stay longer and in one of the suites. I am already plotting ways for a an anniversary gift to send my parents there.",
            "review_url": "https://www.yelp.com?hrid=PBpTBnQujWpiH4nzrf_93g",
            "lang": "en",
            "author_avatar": "https://s3-media0.fl.yelpcdn.com/photo/7dM72AS-Jh0PGLrP71GT4g/60s.jpg",
            "author_url": "https://www.yelp.com/user_details?userid=Mrqlo4m0Kkj544FPg8VaMw",
            "author_name": "Robert A.",
            "meta_data": {
                "author_contributions": 110,
                "feedback": {
                    "useful": 7,
                    "funny": 0,
                    "cool": 2
                }
            },
            "location": "Oakland, CA",
            "response": null
        }
    ],
    "meta_data": {
        "lang": "en",
        "lang_stats": [
            {
                "code": "en",
                "count": 95
            },
            {
                "code": "de",
                "count": 1
            },
            {
                "code": "nl",
                "count": 1
            }
        ]
    }
}

Additional Filters

For some platforms we support additional filters. They are listed below:

Yelp

On Yelp, you can also specify the language of the reviews you want with a query parameter lang it's value must be a 2 letter language code, for e.g. de.

As you observe towards the end of the example response above, the meta data section includes the number of reviews the listing has in each language. The language code must coincide with one of the languages listed to get a response.