Android & iOS screenshot fetcher
Here's a python script that I started a long time ago to
fetch screenshots of
the top ranking Android and iPhone app shots from the App stores. I had a
vision that I would make a public site that lists the screen shots and design
patterns of the best mobile apps, a listing that could provide inspiration and
learning points for mobile developers and designers.
Fast forward months and I realize that I haven't had the time to start this project. Been busy with customer projects. However, some nice people have actually created sites with lists of mobile screenshots. Take a look of these nice sites:
- AndroidNiceties: a collection of Android screenshots and
- pttrns: a collection of iPhone screenshots.
These sites are good starting points for people looking for design inspiration. Be sure also to read:
Anyway, I'm still publishing my screenshot fetcher script, and the montage of shots I just created of the top Android and iOS apps. Enjoy.
Top free apps in Finland, November 2012
Both Google Play and App Store provide pages that list the top ranking free and paid apps in the store. The script fetches these listing pages and then scrapes the individual app pages for the screenshots. App Store provides both iPhone and iPad screenshots separately, Google Play does not differentiate phone vs tablet shots.
The script uses
urllib2 from the standard python library to fetch the HTML
pages from the app stores. lxml is used for the scraping of
HTML of the page. lxml is a wrapper lib to the great C library libxml2. A
few years ago I tested some python xml libs and found lxml to be fastest.
Been using that since.
Having been exposed to XPath queries earlier, I'm using XPath to find the particular DOM elements from the HTML. lxml also provides lxml.cssselect module, which provides CSS alike selectors for finding elements. To find out what elements to find for in the pages, I used the web inspector in Chrome.
Google Play seems to be able to resize the screenshots upon fetch, see the height parameter in the script. To get the original size of the shot, I'm using a large height, 2000 pixels.
The script takes
a single argument "android" or "ios" to determine which app
store to fetch shots from. The downloaded shots will appear under a sub
folder "android", "iphone" or "ipad".
Here's the complete python script. You can download the script too.
#!/usr/bin/python # -*- coding: utf-8 -*- # screenshot-fetcher.py: fetch android/ios app screenshots from app stores # # Author: Tomi.Mickelsson@iki.fi import os, sys from lxml.html import parse from urllib2 import urlopen def fetch_ios_top_list(free_apps=True): " Fetch list of top ios apps " if free_apps: url = "http://www.apple.com/itunes/charts/free-apps/" else: url = "http://www.apple.com/itunes/charts/paid-apps/" doc = parse(url).getroot() grid = doc.get_element_by_id("grid") return [x.find("a").get("href") for x in grid.iter("li")] def fetch_ios_app_shots(rank, url): " Fetch all screenshots of this iphone/ipad app " print "fetch app page", url doc = parse(urlopen(url)).getroot() # get title title = doc.xpath("//div[@id='title']//h1").text print "TITLE", title # get iphone shots iphoneimg = doc.xpath("//div[contains(@class, 'iphone-screen-shots')]//img") for i, img in enumerate(iphoneimg): url = img.get("src") # if "landscape" in img.get("class"): # fname = "L-"+fname save_img(url, "iphone", rank, title, i) # get ipad shots ipadimg = doc.xpath("//div[contains(@class, 'ipad-screen-shots')]//img") for i, img in enumerate(ipadimg): url = img.get("src") save_img(url, "ipad", rank, title, i) def fetch_android_top_list(free_apps=True): " Fetch list of app urls of top android apps " if free_apps: url = "https://play.google.com/store/apps/collection/topselling_free" else: url = "https://play.google.com/store/apps/collection/topselling_paid" # 2nd page: # https://play.google.com/store/apps/collection/topselling_free?start=24&num=24 doc = parse(urlopen(url)).getroot() li = doc.xpath("//ul[contains(@class, 'snippet-list')]//li//a[contains(@class, 'thumbnail')]") return [x.get("href") for x in li] def fetch_android_app_shots(rank, url): " Fetch all screenshots of this iphone/ipad app " if not url.startswith("http"): url = "https://play.google.com/" + url doc = parse(urlopen(url)).getroot() # get title title = doc.xpath("//h1[contains(@class, 'doc-banner-title')]") if title: title = title.text print "TITLE", title # get shots li = doc.xpath("//img[contains(@class, 'doc-screenshot-img')]") imglist = [x.get("src") for x in li] for i, url in enumerate(imglist): url = url.replace("h230", "h2000") # height of image save_img(url, "android", rank, title, i) def save_img(url, dir, rank, title, imgindex): " Fetch an image from the url to a file " fname = "%s/%03d.%s_%d.jpg" % (dir, rank, title[:20], imgindex) fname = fname.replace(" ", "_") print " ",fname f = urlopen(url) data = f.read() f.close() dest = file(fname, "wb") dest.write(data) dest.close() def main(): ostype = sys.argv[-1] # last arg # create dirs for downloaded screenshots dirs = ["android", "iphone", "ipad"] for d in dirs: if not os.path.exists(d): os.mkdir(d) free_apps = True if ostype == "android": # fetch android shots urllist = fetch_android_top_list(free_apps) print "android app count:", len(urllist) for i, url in enumerate(urllist): fetch_android_app_shots(i+1, url) elif ostype == "ios": # fetch ios shots urllist = fetch_ios_top_list(free_apps)[:50] print "ios app count:", len(urllist) for i, url in enumerate(urllist): fetch_ios_app_shots(i+1, url) else: print "unknown os" if __name__ == '__main__': main()
Remember to install the lxml library:
sudo easy_install lxml
ImageMagick is a wonderful package of command line tools for versatile manipulation of images. For the creation of the montage of screenshots, I'm using the tool named montage:
montage -label %f -tile 5x -title 'iPhone top 50 apps - Nov 2012' \ -pointsize 11 -font Helvetica -geometry 160x240+4+8 \ -background '#ddddff' * ../iphone50.jpg
This single invocation of montage generates the whole image including sorting shots based on name, labeling shots, having a layout of 5 columns, and resizing shots to 160x240. Very powerful.
To make a smaller final image (to save bandwidth), I only include the 50 of
the top 100 iOS apps. Likewise, I resize the images to smaller size because I
want to save bandwidth of my site. You can leave the
-geometry option out to
have the full size shots in your own montage.