Online Reviews API (async)

This service can be used to extract customer reviews from various platforms on the internet. The endpoints that make the interface of the service are described below.

Getting Started

Follow these steps to get started with our Online Reviews API:

Sign up for a free trial here
Get your API key from the console
Create a post request to add a new scraping job*

Here is a link to our API's Postman collection. You can import this into Postman, add your API key in the Authorization header and get started.

The video embedded below shows an example of scraping reviews with our API

*The last step can be done using the snippets on our documentation page or using the postman collection linked in this file.

Authentication

We use API keys to allow access to its API. You can get an API key by purchasing credits for or subscribing to our service.

The API key must be included in every API request to the server in a header with the following format:

Authorization: Token API_KEY

Reminder

Replace API_KEY with your actual API key.

Supported Platforms and URL Formats

The following table lists each platform supported and the URL formats expected by our server. On receiving this URL, our server will execute the scraping job and post the data to your webhook_url as it's complete.

Platform	Expected URL Format
Amazon	https://www.amazon.com/Lucky-Brand-Burnout-Notch-Shirt/dp/B081J8SGH7/
eBay	https://www.ebay.co.uk/itm/203456967746
AliExpress	https://www.aliexpress.com/item/1005001566982235.html
Costco	https://www.costco.com/.product.100540132.html
Target	https://www.target.com/p/women-s-mad-love-lennie-lace-up-canvas-sneakers/-/A-79581272
Yelp	https://www.yelp.com/biz/san-francisco-dental-sleep-medicine-douglas-chenin-dds-san-francisco
Tesco	https://www.tesco.com/groceries/en-GB/products/271400233
Asda Groceries	https://groceries.asda.com/product/rioja/campo-viejo-tempranillo-rioja/9157601
Morissons	https://groceries.morrisons.com/products/kellogg-s-crunchy-nut-112028011
Iceland.co.uk	https://www.iceland.co.uk/p/clover-spread-500g/8186.html

Amazon URLs are supported for the following countries: USA (.com), UK (.co.uk), Australia (.com.au), France (.fr), Germany (.de) and India (.in).

API Credits

Each request costs a base fee of 10 credits and each review extracted costs 1 credit.

Jobs

Jobs are used to scrape reviews for a listing of your choice.

Endpoint

https://data.unwrangle.com/api/reviews/jobs/

The jobs endpoint supports the following methods: POST, GET.

Add Job

Creates a new job for scraping online reviews.

HTTP POST

https://data.unwrangle.com/api/reviews/jobs/

REQUEST BODY

The following parameters are accepted in the request body as a JSON object.

Parameter	Type	Default	Description
`"url"`	`str`	required	The `url` from where reviews need to be scraped.
`"n"`	`int`	`0`	Number of reviews to be scraped.
`"from_date"`	`str`	`""`	Date from which reviews must be scraped. String must be in isoformat i.e. `"YYYY-MM-DD"`.
`"webhook_url"`	`str`	`""`	The `url` where job's results will be posted on completion.
`"external_id"`	`int`	`undefined`	External identifier for you to keep track of job.

Example

cURL

curl -L -X POST 'https://data.unwrangle.com/api/reviews/jobs/' \
-H 'Authorization: Token API_KEY' \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
    "external_id": 243,
    "webhook_url": "https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945"
}'

Python

import requests

url = "https://data.unwrangle.com/api/reviews/jobs/"

payload="{\n    \"url\": \"https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP\",\n    \"external_id\": 243,\n    \"webhook_url\": \"https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945\"\n}"
headers = {
'Authorization': 'Token API_KEY',
'Accept': 'application/json',
'Content-Type': 'application/json'
}

response = requests.request("POST", url, headers=headers, data=payload)

print(response.text)

Ruby

require "uri"
require "net/http"

url = URI("https://data.unwrangle.com/api/reviews/jobs/")

http = Net::HTTP.new(url.host, url.port);
request = Net::HTTP::Post.new(url)
request["Authorization"] = "Token API_KEY"
request["Accept"] = "application/json"
request["Content-Type"] = "application/json"
request.body = "{\n    \"url\": \"https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP\",\n    \"external_id\": 243,\n    \"webhook_url\": \"https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945\"\n}"

response = http.request(request)
puts response.read_body

NodeJS

var axios = require('axios');
var data = JSON.stringify({"url":"https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP","external_id":243,"webhook_url":"https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945"});

var config = {
method: 'post',
url: 'https://data.unwrangle.com/api/reviews/jobs/',
headers: { 
    'Authorization': 'Token API_KEY', 
    'Accept': 'application/json', 
    'Content-Type': 'application/json'
},
data : data
};

axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});

PHP

<?php
$client = new http\Client;
$request = new http\Client\Request;
$request->setRequestUrl('https://data.unwrangle.com/api/reviews/jobs/');
$request->setRequestMethod('POST');
$body = new http\Message\Body;
$body->append('{
    "url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
    "external_id": 243,
    "webhook_url": "https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945"
}');
$request->setBody($body);
$request->setOptions(array());
$request->setHeaders(array(
'Authorization' => 'Token API_KEY',
'Accept' => 'application/json',
'Content-Type' => 'application/json'
));
$client->enqueue($request)->send();
$response = $client->getResponse();
echo $response->getBody();

RESPONSE OBJECT

{
    "created": "2021-03-27T01:37:17.228192Z",
    "id": 44,
    "url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
    "status": "queued",
    "n": null,
    "from_date": null,
    "webhook_url": "https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945",
    "external_id": 243,
    "is_webhook_sent": false,
    "credits_used": 1010,
    "completed": null
}

Note

The following details about the request body parameters:

"url": is the only required param. It must be a valid URL for a listing on a platform that we support.
"n": must be in multiples of 10 and only filters the most recent n reviews.
"from_date": param filters reviews from the date specified. Can be used in conjunction with n to get the most recent reviews from a particular date.
"webhook_url": is the target url where the results are posted on job completion. The format of the response is showcased below.
"external_id": is returned with every job response and can be used to keep track of jobs on your system.

Reminder

Replace API_KEY with your actual API key in above examples.

List Jobs

Lists all jobs created with your account.

HTTP GET

https://data.unwrangle.com/api/revieiws/jobs/

Example

cURL

curl -L -X GET 'https://data.unwrangle.com/api/reviews/jobs/' \
-H 'Authorization: Token API_KEY' \
-H 'Content-Type: application/json' \
-H 'Accept: application/json'

Python

import requests

url = "https://data.unwrangle.com/api/reviews/jobs/"

payload={}
headers = {
'Authorization': 'Token API_KEY',
'Content-Type': 'application/json',
'Accept': 'application/json'
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Ruby

require "uri"
require "net/http"

url = URI("https://data.unwrangle.com/api/reviews/jobs/")

https = Net::HTTP.new(url.host, url.port)
https.use_ssl = true

request = Net::HTTP::Get.new(url)
request["Authorization"] = "Token API_KEY"
request["Content-Type"] = "application/json"
request["Accept"] = "application/json"

response = https.request(request)
puts response.read_body

NodeJS

var axios = require('axios');

var config = {
method: 'get',
url: 'https://data.unwrangle.com/api/reviews/jobs/',
headers: { 
    'Authorization': 'Token API_KEY', 
    'Content-Type': 'application/json', 
    'Accept': 'application/json'
}
};

axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});

PHP

<?php
$client = new http\Client;
$request = new http\Client\Request;
$request->setRequestUrl('https://data.unwrangle.com/api/reviews/jobs/');
$request->setRequestMethod('GET');
$request->setOptions(array());
$request->setHeaders(array(
'Authorization' => 'Token API_KEY',
'Content-Type' => 'application/json',
'Accept' => 'application/json'
));
$client->enqueue($request)->send();
$response = $client->getResponse();
echo $response->getBody();

RESPONSE OBJECT

[{
    "created": "2021-03-27T01:37:17.228192Z",
    "id": 44,
    "url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
    "status": "complete",
    "n": null,
    "from_date": null,
    "webhook_url": "https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945",
    "external_id": 243,
    "is_webhook_sent": true,
    "credits_used": 64,
    "completed": "2021-03-27T01:40:27.716704Z"
},
{
    "created": "2021-03-27T01:29:27.174761Z",
    "id": 42,
    "url": "https://www.amazon.com/Keyboard-Headset-Rainbow-Headphone-Computer/dp/B07FSF7C2P/",
    "status": "complete",
    "n": 50,
    "from_date": null,
    "webhook_url": "https://webhook.site/#!/43ac8c7d-d22f-4d4d-8162-f124bf7d4945",
    "external_id": 12121,
    "is_webhook_sent": false,
    "credits_used": 60,
    "completed": "2021-03-27T01:33:17.773425Z"
},
...
{
    "created": "2021-03-25T10:10:52.118980Z",
    "id": 3,
    "url": "https://www.amazon.com/MERKUR-Classic-MK-34C-2-Piece-Double/dp/B002A8JO1Q",
    "status": "complete",
    "n": 20,
    "from_date": null,
    "webhook_url": "https://webhook.site/40aa220b-d736-4cfd-94b4-bde350c2e538",
    "external_id": 232324,
    "is_webhook_sent": true,
    "credits_used": 30,
    "completed": "2021-03-25T10:14:12.872290Z"
}]

Job Status

Returns the job object for a particular job id. Contained within the job object is its status.

HTTP GET

https://data.unwrangle.com/api/reviews/jobs/:id/

Example

cURL

curl -L -X GET 'https://data.unwrangle.com/api//data/reviews/jobs/44/' \
-H 'Authorization: Token API_KEY' \
-H 'Accept: application/json' \
-H 'Content-Type: appliciation/json'

Python

import requests

url = "https://data.unwrangle.com/api//data/reviews/jobs/44/"

payload={}
headers = {
'Authorization': 'Token API_KEY',
'Accept': 'application/json',
'Content-Type': 'appliciation/json'
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Ruby

require "uri"
require "net/http"

url = URI("https://data.unwrangle.com/api//data/reviews/jobs/44/")

http = Net::HTTP.new(url.host, url.port);
request = Net::HTTP::Get.new(url)
request["Authorization"] = "Token API_KEY"
request["Accept"] = "application/json"
request["Content-Type"] = "appliciation/json"

response = http.request(request)
puts response.read_body

NodeJS

var axios = require('axios');

var config = {
method: 'get',
url: 'https://data.unwrangle.com/api//data/reviews/jobs/44/',
headers: { 
    'Authorization': 'Token API_KEY', 
    'Accept': 'application/json', 
    'Content-Type': 'appliciation/json'
}
};

axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});

PHP

<?php
$client = new http\Client;
$request = new http\Client\Request;
$request->setRequestUrl('https://data.unwrangle.com/api//data/reviews/jobs/44/');
$request->setRequestMethod('GET');
$request->setOptions(array());
$request->setHeaders(array(
'Authorization' => 'Token API_KEY',
'Accept' => 'application/json',
'Content-Type' => 'appliciation/json'
));
$client->enqueue($request)->send();
$response = $client->getResponse();
echo $response->getBody();

RESPONSE OBJECT

{
    "created": "2021-03-27T15:34:25.906586Z",
    "id": 44,
    "url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
    "status": "complete",
    "n": null,
    "from_date": null,
    "webhook_url": "https://webhook.site/40aa220b-d736-4cfd-94b4-bde350c2e538",
    "external_id": 243,
    "is_webhook_sent": true,
    "credits_used": 65,
    "completed": "2021-03-27T15:38:28.674858Z"
}

The job's "status" property can have one of the following values:

Status	Meaning
`"queued"`	Job was successfully created and has been queued.
`"running"`	Job is being executed. Speed depends on subscription plan.
`"complete"`	Job is complete and results can be retrieved.
`"maintenance"`	Something went wrong and we are working on it.

Retrieve Results

Retrieve results, i.e. scraped reviews for a particular job with its id.

HTTP GET

https://data.unwrangle.com/api/reviews/jobs/:id/results/

Example

cURL

curl -L -X GET 'https://data.unwrangle.com/api//data/reviews/jobs/44/results/' \
-H 'Authorization: Token API_KEY' \
-H 'Accept: application/json' \
-H 'Content-Type: appliciation/json'

Python

import requests

url = "https://data.unwrangle.com/api//data/reviews/jobs/44/results/"

payload={}
headers = {
'Authorization': 'Token API_KEY',
'Accept': 'application/json',
'Content-Type': 'appliciation/json'
}

response = requests.request("GET", url, headers=headers, data=payload)

print(response.text)

Ruby

require "uri"
require "net/http"

url = URI("https://data.unwrangle.com/api//data/reviews/jobs/44/results/")

http = Net::HTTP.new(url.host, url.port);
request = Net::HTTP::Get.new(url)
request["Authorization"] = "Token API_KEY"
request["Accept"] = "application/json"
request["Content-Type"] = "appliciation/json"

response = http.request(request)
puts response.read_body

NodeJS

var axios = require('axios');

var config = {
method: 'get',
url: 'https://data.unwrangle.com/api//data/reviews/jobs/44/results/',
headers: { 
    'Authorization': 'Token API_KEY', 
    'Accept': 'application/json', 
    'Content-Type': 'appliciation/json'
}
};

axios(config)
.then(function (response) {
console.log(JSON.stringify(response.data));
})
.catch(function (error) {
console.log(error);
});

PHP

<?php
$client = new http\Client;
$request = new http\Client\Request;
$request->setRequestUrl('https://data.unwrangle.com/api//data/reviews/jobs/44/results/');
$request->setRequestMethod('GET');
$request->setOptions(array());
$request->setHeaders(array(
'Authorization' => 'Token API_KEY',
'Accept' => 'application/json',
'Content-Type' => 'appliciation/json'
));
$client->enqueue($request)->send();
$response = $client->getResponse();
echo $response->getBody();

RESPONSE OBJECT

{
    "created": "2021-03-27T15:34:25.906586Z",
    "id": 44,
    "url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
    "status": "complete",
    "n": null,
    "from_date": null,
    "webhook_url": "https://webhook.site/40aa220b-d736-4cfd-94b4-bde350c2e538",
    "external_id": 243,
    "is_webhook_sent": true,
    "credits_used": 65,
    "completed": "2021-03-27T15:38:28.674858Z",
    "result_count": 55,
    "reviews": [
        {
            "id": "RXEAWP8GQNIHW",
            "date": "2021-03-25",
            "author_name": "Mommy of Four",
            "author_url": "https://www.amazon.com/gp/profile/amzn1.account.AHJ5UHPD53LVSRQ5NKKQSR6XX52Q",
            "rating": 1,
            "review_title": "Not sweat proof",
            "review_url": "https://www.amazon.com/gp/customer-reviews/RXEAWP8GQNIHW/",
            "review_text": "The problem with these are they are so expensive and they aren’t sweat proof. If you dont plan to use these to workout, then they’re great. But if you plan on sweating at all, they are not designed to handle that",
            "meta_data": {
                "verified_purchase": true
            }
        },
        {
            "id": "R1OURWW919WRFH",
            "date": "2021-03-24",
            "author_name": "Rob",
            "author_url": "https://www.amazon.com/gp/profile/amzn1.account.AG6PAGNCGFYDH2AKCECKU3VIQ6YQ",
            "rating": 4,
            "review_title": "Surprisingly meh audio quality",
            "review_url": "https://www.amazon.com/gp/customer-reviews/R1OURWW919WRFH/",
            "review_text": "Had I not got a pair of mifo’s earbuds, I would have thought these were amazing. But honestly they’re good, maybe even great. Just a shame if you’ve had better for cheaper. That said, the noise canceling on this cannot be beat. I took these on a roadtrip, and could barely hear the noise from the highway, it was amazing.",
            "meta_data": {
                "verified_purchase": true
            }
        },
        ...
        {
            "id": "R35AMUQ7H0JQ1C",
            "date": "2021-02-24",
            "author_name": "Jedi",
            "author_url": "https://www.amazon.com/gp/profile/amzn1.account.AHOBJSNV5YLFCDF7OTSNOLIMJKRQ",
            "rating": 1,
            "review_title": "Apple has no shame with their pricing",
            "review_url": "https://www.amazon.com/gp/customer-reviews/R35AMUQ7H0JQ1C/",
            "review_text": "I'm an audiophile and have spent a good amount of coin on gear in my life. At $550 these headphones have the sound quality of $200 headphones. Which I guess is okay compared to the Beats cans that have $30 sound at $200.The main argument for these headphones are \"theY aRE gReat If yOU arE iN THe apPle ECosYSTeM\" aka the siren call of apple fanbois to buy the latest gimmick to to try to show off to people, but not realizing that nobody really cares.Another thing I really dislike is these have no longevity in terms of usage. They are designed and meant to be used and then replaced until the next gimmick from apple comes. Remember when airpods were all the rage? Then Air pod Pros came. There were so many issues with the airpods like the battery life, they tried to do some kind of warranty recall, but lot of them fell through the cracks and then apple just kinda swept it under the rug. Now we got this Max. Lol. I like to buy quality things that are BIFL.",
            "meta_data": {
                "verified_purchase": false
            }
        },
        ...
        {
            "id": "R3IX5FY5UWXHWW",
            "date": "2021-02-21",
            "author_name": "Jesus alejandro treviño pacheco",
            "author_url": null,
            "rating": 5,
            "review_title": "Una maravilla",
            "review_url": "https://www.amazon.com/gp/customer-reviews/R3IX5FY5UWXHWW/",
            "review_text": "El producto vale cada peso",
            "meta_data": {
                "verified_purchase": true
            }
        }
    ]
}