Saturday, November 08, 2008

A Free-Fallin' Introduction to Amazon Web Services

going through my old email folder the other day i came across Amazon Web Services Newsletter #3 dated 27 August 2003.
the summaries of that newsletter are given below:

===========
1. AWS News
===========

Summary
-------
* Product Description Returned
* Availability in Lite Results
* Japanese Marketplace Access
* Japanese Search
* UPC Search
* Scratchpad for UK
* RSS Feeds
* Shopping Cart Similarities
* ASIN Link Changes
* Honor System Payment API Beta Test
* SOAP Error Response
* Proper use of International Servers

=============
2. AWS Issues
=============

Summary
-------
* Determining Buyability Using Availability
* Developer Token Usage
* Using Older Versions of AWS
* Storing ASINs
* NuSOAP Compatibility

===================
3. In the Community
===================

Summary
-------
* Discussion Board
* Board Summary
* Weekly Chats
* Newsletter
* Featured Sites and Applications
* Press

==================
4. Tips and Tricks
==================

Summary
-------
* Featured style sheet
* Featured code snippet
* Featured tip
* Your contributions welcome
* Local Associates Programs

among Featured Sites and Applications is a link to an article for a Mozilla tool for browing the Amazon Catalog:
http://www.oreillynet.com/pub/a/mozilla/2003/05/02/casestudy2.html
today i took a look at another chapter referred to at the beginning of this casestudy:
http://www.oreillynet.com/pub/a/mozilla/2002/12/17/app_dev.html.

That same newsletter also features a link to an interesting article on RSS by author: Ravikiran Muvva at Code101.com:
http://www.code101.com/Code101/DisplayArticle.aspx?cid=41.

going through my email archive looking for more information i discovered that this newsletter was the oldest one i'd kept. some time after that i changed my email account setup and seem to have stored everything related to Amazon Web Services from newsletter #10 onwards. that means i can document quite a history on the development of Amazon Web Services to the present stage.

with this article i would like to guide the reader through an example of how i recently migrated a script from Amazon Web Services 3.0 -- which was the stage we were in on August 27th, 2003 -- to the present version Amazon E-Commerce Service 4.0.

on 6 February 2007 Amazon informed its Web Service Subscribers as follows:

*****************************************************************
AMAZON ECS 3.0 SUNSET: PLEASE MIGRATE TO AMAZON ECS 4.0
*****************************************************************
After many years of useful service, the Amazon E-Commerce Service 3.0
(Amazon ECS 3.0) web service will be shut down on March 31, 2008. Since
the introduction of Amazon ECS 4.0 in late 2004 the usage of Amazon ECS 3.0
has steadily declined as sites and applications have naturally migrated to
Amazon ECS 4.0. In June of 2006 (Version: 2006-06-28), we achieved
complete feature and data parity between the two services and are now
requesting that all existing sites and applications be migrated to Amazon
ECS 4.0 within the next 12 months. Beginning March 31, 2008, we will no
longer be accepting Amazon ECS 3.0 requests. The existing maintenance
effort saved by shutting down Amazon ECS 3.0 will be re-invested into
Amazon ECS 4.0 to allow for the introduction of more features and
capabilities.
Let's take a look at Chapter 6 from Paul Bausch's Amazon Hacks Book:

Amazon Hacks
Hack 80 Program AWS with Perl

below is the original code listing:
#!/usr/bin/perl
# amazon_http.pl
# A typical Amazon Web API Perl script using the XML/HTTP interface
# Usage: amazon_http.pl <keyword>

#Your Amazon developer's token
my $dev_key='insert developer token';

#Your Amazon affiliate code
my $af_tag='insert associate tag';

#Take the keyword from the command-line
my $keyword =shift @ARGV or die "Usage:perl amazon_http.pl <keyword>\n";

#Assemble the URL
my $url = "http://xml.amazon.com/onca/xml3?t=" . $af_tag . "&dev-t=" . $dev_key . "&type=lite&f=xml&mode=books&" . "KeywordSearch=" . $keyword;

use strict;

#Use the XML::Parser and LWP::Simple Perl modules
use XML::Simple;
use LWP::Simple;

my $content = get($url);
die "Could not retrieve $url" unless $content;

my $xmlsimple = XML::Simple->new( );
my $response = $xmlsimple->XMLin($content);

foreach my $result (@{$response->{Details}}){
#Print out the main bits of each result
print
join "\n",
$result->{ProductName}||"no title",
"ASIN: " . $result->{Asin} . ", " .
$result->{OurPrice} . "\n\n";
}
as we can see this code is meant to be used from the command line. however, i wanted to execute this code from my host's server using my browser. therefore i changed the following line from
#Take the keyword from the command-line
my $keyword =shift @ARGV or die "Usage:perl amazon_http.pl \n";
to
#Take the keyword from the command-line
my $keyword = 'Bob%20Dylan';
now my code looks like this:
#!/usr/bin/perl -w
# amazon_http.pl
# A typical Amazon Web API Perl script using the XML/HTTP interface
# Usage: amazon_http.pl <keyword>

BEGIN {
$| = 1;
open (STDERR, ">&STDOUT");
print qq~Content-type: text/html\n\n~;
}

#Your Amazon developer's token
my $dev_key='D23VNPYO8ODZL6';

#Your Amazon affiliate code
my $af_tag='downinthefloo-20';

#Take the keyword from the command-line
my $keyword = 'Bob%20Dylan';

#Assemble the URL
my $url = "http://xml.amazon.com/onca/xml3?t=" . $af_tag . "&dev-t=" . $dev_key . "&type=lite&f=xml&mode=books&" . "KeywordSearch=" . $keyword;

print "<a href=\"".$url."\" target=\"new\">The Request</a><br> <br>";

use strict;

#Use the XML::Parser and LWP::Simple Perl modules
use XML::Simple;
use LWP::Simple;

my $content = get($url);
die "Could not retrieve $url" unless $content;

my $xmlsimple = XML::Simple->new( );
my $response = $xmlsimple->XMLin($content);

foreach my $result (@{$response->{Details}}){
#Print out the main bits of each result
print
join "\n",
$result->{ProductName}||"no title",
"ASIN: " . $result->{Asin} . ", " .
$result->{OurPrice} . "\n\n";
}
other differences to the original code snippet are:
a) use of the Warning switch -w;
b) use of a code snippet that will allow me to display the details of a possible error 500 message if there is any;
c) inclusion of a link to Amazon's xml document in case of:
   ca) no items being displayed;
   cb) an error message being displayed;

alright: the script has been uploaded to the cgi-bin folder on the server and we are able to access the script using our preferred browser: http://localhost/cgi-bin/amazon_http_old.pl (i added _old to the file name since i know it's not gonna work and i got a new version ready to go :-) )

running the script will display the following error message:
The Request

Could not retrieve http://xml.amazon.com/onca/xml3?t=downinthefloo-20&dev-t=D23VNPYO8ODZL6&type=lite&f=xml&mode=books&KeywordSearch=Bob%20Dylan at PATH/TO/htdocs/cgi-bin/amazon_http_old.pl line 33.
where The Request is an active link to Amazon's xml document:

once we click on that link to see what went wrong we are being re-directed to an error page:
Amazon.com
We're Sorry - Service Gone

Amazon Ecommerce Web Service 3.0 has been deprecated after many years of useful service
on March 31st 2008. Please upgrade to the Amazon Associates Web Service 4.0 as detailed
in the migration guide. Please visit Amazon Associates Web Service Developer Forum for more
information. If you came to this page from an RSS feed, visit Amazon's Product RSS Feeds
page for an upgrade.
visiting the migration guide we can quickly detect a few things that have changed; for instance: the base url:
http://xml.amazon.com/onca/xml3
would now be
http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService
the Developer Token (dev-t) has now become the AWSAccessKeyId;
the search "mode" would now be a SearchIndex value;
the request "type" has in the past been lite or heavy;
the xml feed result would display more details with the heavy type than with the lite type.
this request parameter is now being named "ResponseGroup" and allows for a number of various feed results.
the main ResponseGroups are Small, Medium and Large.
to be on the safe side when we now alter our request url we are going to use &ResponseGroup=Large.
i should mention here that all those terms are case-sensitive.
when we go back to our migration guide once again for the KeywordSearch paramter we find now that Amazon states two values instead of just one: SearchIndex, Keywords; this means for a KeywordSearch we also require the SearchIndex value; since our mode was books, our SearchIndex should be Books (in this case with a capital 'B'), also the SearchIndex values are case-sensitive. furthermore, these SearchIndices vary from country to country (locale to locale).
our parameters now look like this: &SearchIndex=Books&Keywords=Bob%20Dylan. with ECS 4.0 two further parameters have been introduced, of which the first one (Operation) is required, the second one (Version) is not obligatory.
to lookup the details of a single item we would use the parameter &Operation=ItemLookup; however, we would need the ASIN or ISBN number of that item, but since we do not know such a number yet we will use the &Operation=ItemSearch parameter.

Amazon is continually adding features for making ECS requests. recently, for example, they have added links to add items to a WishList, Baby-, or WeddingRegistry, or to Tell-A-Friend to their xml response documents. sometimes when they have added a new product range to a particular locale it may be added to the list of available SearchIndex values. to use these added features it may be necessary to add the paramter &Version=2008-08-19 to the request url.

here you may find the latest Developer Guide with many more details than i would be able to cover within this article.

it looks like we can now start to assemble our new request url:

well, i forgot about the associate id: in the past that used to be just "t"; that has now become: "AssociateTag".

http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService
&AssociateTag=downinthefloo-20
&AWSAccessKeyId=1BZ9VRA5AKNBPYJM6YR2
&Operation=ItemSearch
&SearchIndex=Books
&ResponseGroup=Large
&Keywords=Bob%20Dylan
let's see if this request will give us a new xml feed document (try this link): The Request.

it worked: this time we see no error message but the xml feed document we expected and can now analyze the elements we need within the foreach loop of our script in order to display some items in our browser window:

here are the details i have chosen for my short example:

foreach my $result (@{$response->{Items}->{Item}}){
#Print out the main bits of each result
print
join
"\n",
"<a href=\"".$result->{LargeImage}->{URL}."\" target=\"image\"><img src=\"".$result->{MediumImage}->{URL},"\" border=\"0\" alt=\"".$result->{ItemAttributes}->{Title}."\"></a>",
"<br>Title: ".$result->{ItemAttributes}->{Title},
"<br>Author: ".$result->{ItemAttributes}->{Author},
"<br>Binding: ".$result->{ItemAttributes}->{Binding},
"<br>ASIN: ".$result->{ASIN},
"<br>Price: ".$result->{ItemAttributes}->{ListPrice}->{FormattedPrice} || "price unavailable",
"<br><a href=\"".$result->{DetailPageURL}."\" target=\"_blank\">buy now</a><hr>",
"<br>\n\n";
}
my complete code example now looks as follows:

#!/usr/bin/perl
# amazon_http.pl
# A typical Amazon Web API Perl script using the XML/HTTP interface
# Usage: amazon_http.pl <keyword>

#Your Amazon developer's token
my $dev_key='1BZ9VRA5AKNBPYJM6YR2';

#Your Amazon affiliate code
my $af_tag='downinthefloo-20';

#Take the keyword from the command-line
my $keyword = 'Bob%20Dylan';

#Assemble the URL
my $url = "http://ecs.amazonaws.com/onca/xml?Service=AWSECommerceService&AssociateTag=" . $af_tag . "&AWSAccessKeyId=" . $dev_key . "&Operation=ItemSearch&SearchIndex=Books&&Keywords=" . $keyword . "&ResponseGroup=Large";

print "<a href=\"".$url."\" target=\"new\">The Request</a><br> <br>";

use strict;

#Use the XML::Parser and LWP::Simple Perl modules
use XML::Simple;
use LWP::Simple;

my $content = get($url);
die "Could not retrieve $url" unless $content;

my $xmlsimple = XML::Simple->new( );
my $response = $xmlsimple->XMLin($content);
foreach my $result (@{$response->{Items}->{Item}}){
#Print out the main bits of each result
print
join
"\n",
"<a href=\"".$result->{LargeImage}->{URL}."\" target=\"image\"><img src=\"".$result->{MediumImage}->{URL},"\" border=\"0\" alt=\"".$result->{ItemAttributes}->{Title}."\"></a>",
"<br>Title: ".$result->{ItemAttributes}->{Title},
"<br>Author: ".$result->{ItemAttributes}->{Author},
"<br>Binding: ".$result->{ItemAttributes}->{Binding},
"<br>ASIN: ".$result->{ASIN},
"<br>Price: ".$result->{ItemAttributes}->{ListPrice}->{FormattedPrice} || "price unavailable",
"<br><a href=\"".$result->{DetailPageURL}."\" target=\"_blank\">buy now</a><hr>",
"<br>\n\n";
}
at last you can see what the result of this short script will look like on the screen:
http://www.downintheflood.com/cgi-bin/amazon_http.pl

hope you enjoyed this rather short example from a long article!

tom.paine