scrape package

Submodules

scrape.google module

contains the GoogleImageScraper class used for scraping google images

class scrape.google.GoogleImageScraper(key, cx, params)[source]

Bases: object

scrapes google images

issue_query()[source]

issue google image query

Returns:None
Return type:None
response

google image search response

Returns:response
Return type:dict

scrape.wood_database module

contains the WoodDatabaseScraper class used for scraping wood-database.com

class scrape.wood_database.WoodDatabaseScraper[source]

Bases: object

get_descriptions(urls)[source]

gets description data for each wood type

Parameters:urls (iterable) – links generated by get_wood_urls
Returns:dict of lists
Return type:dict

lists all wood texture urls for wood-database.com

Parameters:urls (iterable) – links generated by get_wood_urls
Returns:urls
Return type:list
get_wood_urls()[source]

yields links of each different wood page

Yields:str – url
scrape_images(root, prefix=None)[source]

scrapes and saves all texture images from wood-database.com

Parameters:
  • root (str) – directory to save images in
  • opt (prefix) – filename prefix default: None
write_images(root, links, prefix=None)[source]

writes all images to a given root directory

Parameters:
  • root (str) – root directory
  • links (list) – list of image links provided by get_image_links
  • opt (prefix) – filename prefix default: None
Returns:

None
Return type:

None

Module contents