Online Reviews API (async)
This service can be used to extract customer reviews from various platforms on the internet. The endpoints that make the interface of the service are described below.
Getting Started
Follow these steps to get started with our Online Reviews API:
- Sign up for a free trial here
- Get your API key from the console
- Create a post request to add a new scraping job*
Here is a link to our API's Postman collection. You can import this into Postman, add your API key in the Authorization
header and get started.
The video embedded below shows an example of scraping reviews with our API
*The last step can be done using the snippets on our documentation page or using the postman collection linked in this file.
Authentication
We use API keys to allow access to its API. You can get an API key by purchasing credits for or subscribing to our service.
The API key must be included in every API request to the server in a header with the following format:
Authorization: Token API_KEY
Reminder
Replace API_KEY
with your actual API key.
Supported Platforms and URL Formats
The following table lists each platform supported and the URL formats expected by our server. On receiving this URL, our server will execute the scraping job and post the data to your webhook_url
as it's complete.
Platform | Expected URL Format |
---|---|
Amazon | https://www.amazon.com/Lucky-Brand-Burnout-Notch-Shirt/dp/B081J8SGH7/ |
eBay | https://www.ebay.co.uk/itm/203456967746 |
AliExpress | https://www.aliexpress.com/item/1005001566982235.html |
Costco | https://www.costco.com/.product.100540132.html |
Target | https://www.target.com/p/women-s-mad-love-lennie-lace-up-canvas-sneakers/-/A-79581272 |
Yelp | https://www.yelp.com/biz/san-francisco-dental-sleep-medicine-douglas-chenin-dds-san-francisco |
Tesco | https://www.tesco.com/groceries/en-GB/products/271400233 |
Asda Groceries | https://groceries.asda.com/product/rioja/campo-viejo-tempranillo-rioja/9157601 |
Morissons | https://groceries.morrisons.com/products/kellogg-s-crunchy-nut-112028011 |
Iceland.co.uk | https://www.iceland.co.uk/p/clover-spread-500g/8186.html |
Amazon URLs are supported for the following countries: USA (.com), UK (.co.uk), Australia (.com.au), France (.fr), Germany (.de) and India (.in).
API Credits
Each request costs a base fee of 10 credits and each review extracted costs 1 credit.
Jobs
Jobs are used to scrape reviews for a listing of your choice.
Endpoint
https://data.unwrangle.com/api/reviews/jobs/
POST
, GET
.
Add Job
Creates a new job for scraping online reviews.
HTTP POST
https://data.unwrangle.com/api/reviews/jobs/
The following parameters are accepted in the request body as a JSON object.
Parameter | Type | Default | Description |
---|---|---|---|
"url" |
str |
required | The url from where reviews need to be scraped. |
"n" |
int |
0 |
Number of reviews to be scraped. |
"from_date" |
str |
"" |
Date from which reviews must be scraped. String must be in isoformat i.e. "YYYY-MM-DD" . |
"webhook_url" |
str |
"" |
The url where job's results will be posted on completion. |
"external_id" |
int |
undefined |
External identifier for you to keep track of job. |
Example
1 2 3 4 5 6 7 8 9 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
|
RESPONSE OBJECT
{
"created": "2021-03-27T01:37:17.228192Z",
"id": 44,
"url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
"status": "queued",
"n": null,
"from_date": null,
"webhook_url": "https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945",
"external_id": 243,
"is_webhook_sent": false,
"credits_used": 1010,
"completed": null
}
Note
The following details about the request body parameters:
"url":
is the only required param. It must be a valid URL for a listing on a platform that we support."n":
must be in multiples of 10 and only filters the most recentn
reviews."from_date":
param filters reviews from the date specified. Can be used in conjunction withn
to get the most recent reviews from a particular date."webhook_url":
is the target url where the results are posted on job completion. The format of the response is showcased below."external_id":
is returned with every job response and can be used to keep track of jobs on your system.
Reminder
Replace API_KEY
with your actual API key in above examples.
List Jobs
Lists all jobs created with your account.
HTTP GET
https://data.unwrangle.com/api/revieiws/jobs/
Example
1 2 3 4 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
RESPONSE OBJECT
[{
"created": "2021-03-27T01:37:17.228192Z",
"id": 44,
"url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
"status": "complete",
"n": null,
"from_date": null,
"webhook_url": "https://webhook.site/43ac8c7d-d22f-4d4d-8162-f124bf7d4945",
"external_id": 243,
"is_webhook_sent": true,
"credits_used": 64,
"completed": "2021-03-27T01:40:27.716704Z"
},
{
"created": "2021-03-27T01:29:27.174761Z",
"id": 42,
"url": "https://www.amazon.com/Keyboard-Headset-Rainbow-Headphone-Computer/dp/B07FSF7C2P/",
"status": "complete",
"n": 50,
"from_date": null,
"webhook_url": "https://webhook.site/#!/43ac8c7d-d22f-4d4d-8162-f124bf7d4945",
"external_id": 12121,
"is_webhook_sent": false,
"credits_used": 60,
"completed": "2021-03-27T01:33:17.773425Z"
},
...
{
"created": "2021-03-25T10:10:52.118980Z",
"id": 3,
"url": "https://www.amazon.com/MERKUR-Classic-MK-34C-2-Piece-Double/dp/B002A8JO1Q",
"status": "complete",
"n": 20,
"from_date": null,
"webhook_url": "https://webhook.site/40aa220b-d736-4cfd-94b4-bde350c2e538",
"external_id": 232324,
"is_webhook_sent": true,
"credits_used": 30,
"completed": "2021-03-25T10:14:12.872290Z"
}]
Job Status
Returns the job object for a particular job id
. Contained within the job object is its status
.
HTTP GET
https://data.unwrangle.com/api/reviews/jobs/:id/
Example
1 2 3 4 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
RESPONSE OBJECT
{
"created": "2021-03-27T15:34:25.906586Z",
"id": 44,
"url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
"status": "complete",
"n": null,
"from_date": null,
"webhook_url": "https://webhook.site/40aa220b-d736-4cfd-94b4-bde350c2e538",
"external_id": 243,
"is_webhook_sent": true,
"credits_used": 65,
"completed": "2021-03-27T15:38:28.674858Z"
}
The job's "status"
property can have one of the following values:
Status | Meaning |
---|---|
"queued" |
Job was successfully created and has been queued. |
"running" |
Job is being executed. Speed depends on subscription plan. |
"complete" |
Job is complete and results can be retrieved. |
"maintenance" |
Something went wrong and we are working on it. |
Retrieve Results
Retrieve results, i.e. scraped reviews for a particular job with its id
.
HTTP GET
https://data.unwrangle.com/api/reviews/jobs/:id/results/
Example
1 2 3 4 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
|
RESPONSE OBJECT
{
"created": "2021-03-27T15:34:25.906586Z",
"id": 44,
"url": "https://www.amazon.com/New-Apple-AirPods-Max-Silver/dp/B08PZD76NP",
"status": "complete",
"n": null,
"from_date": null,
"webhook_url": "https://webhook.site/40aa220b-d736-4cfd-94b4-bde350c2e538",
"external_id": 243,
"is_webhook_sent": true,
"credits_used": 65,
"completed": "2021-03-27T15:38:28.674858Z",
"result_count": 55,
"reviews": [
{
"id": "RXEAWP8GQNIHW",
"date": "2021-03-25",
"author_name": "Mommy of Four",
"author_url": "https://www.amazon.com/gp/profile/amzn1.account.AHJ5UHPD53LVSRQ5NKKQSR6XX52Q",
"rating": 1,
"review_title": "Not sweat proof",
"review_url": "https://www.amazon.com/gp/customer-reviews/RXEAWP8GQNIHW/",
"review_text": "The problem with these are they are so expensive and they aren’t sweat proof. If you dont plan to use these to workout, then they’re great. But if you plan on sweating at all, they are not designed to handle that",
"meta_data": {
"verified_purchase": true
}
},
{
"id": "R1OURWW919WRFH",
"date": "2021-03-24",
"author_name": "Rob",
"author_url": "https://www.amazon.com/gp/profile/amzn1.account.AG6PAGNCGFYDH2AKCECKU3VIQ6YQ",
"rating": 4,
"review_title": "Surprisingly meh audio quality",
"review_url": "https://www.amazon.com/gp/customer-reviews/R1OURWW919WRFH/",
"review_text": "Had I not got a pair of mifo’s earbuds, I would have thought these were amazing. But honestly they’re good, maybe even great. Just a shame if you’ve had better for cheaper. That said, the noise canceling on this cannot be beat. I took these on a roadtrip, and could barely hear the noise from the highway, it was amazing.",
"meta_data": {
"verified_purchase": true
}
},
...
{
"id": "R35AMUQ7H0JQ1C",
"date": "2021-02-24",
"author_name": "Jedi",
"author_url": "https://www.amazon.com/gp/profile/amzn1.account.AHOBJSNV5YLFCDF7OTSNOLIMJKRQ",
"rating": 1,
"review_title": "Apple has no shame with their pricing",
"review_url": "https://www.amazon.com/gp/customer-reviews/R35AMUQ7H0JQ1C/",
"review_text": "I'm an audiophile and have spent a good amount of coin on gear in my life. At $550 these headphones have the sound quality of $200 headphones. Which I guess is okay compared to the Beats cans that have $30 sound at $200.The main argument for these headphones are \"theY aRE gReat If yOU arE iN THe apPle ECosYSTeM\" aka the siren call of apple fanbois to buy the latest gimmick to to try to show off to people, but not realizing that nobody really cares.Another thing I really dislike is these have no longevity in terms of usage. They are designed and meant to be used and then replaced until the next gimmick from apple comes. Remember when airpods were all the rage? Then Air pod Pros came. There were so many issues with the airpods like the battery life, they tried to do some kind of warranty recall, but lot of them fell through the cracks and then apple just kinda swept it under the rug. Now we got this Max. Lol. I like to buy quality things that are BIFL.",
"meta_data": {
"verified_purchase": false
}
},
...
{
"id": "R3IX5FY5UWXHWW",
"date": "2021-02-21",
"author_name": "Jesus alejandro treviño pacheco",
"author_url": null,
"rating": 5,
"review_title": "Una maravilla",
"review_url": "https://www.amazon.com/gp/customer-reviews/R3IX5FY5UWXHWW/",
"review_text": "El producto vale cada peso",
"meta_data": {
"verified_purchase": true
}
}
]
}