Hi! We've renamed ScraperWiki.
The product is now QuickCode and the company is The Sensible Code Company.

Blog

‘Documentation is like sex: when it is good, it is very, very good; and when it is bad, it is better than nothing’

You may have noticed that the design of the ScraperWiki site has changed substantially.

As part of that, we made a few improvements to the documentation. Lots of you told us we had to make our documentation easier to find, more reliable and complete.

We’ve reorganised it all under one contents page, called Documentation throughout the site, including within the code editor. All the documentation is listed there. (The layout shamelessly inspired by Django).

Of course, everyone likes different kinds of documentation – talk to a teacher and they’ll tell you all about different learning styles. Here’s what we have on offer, all available in Ruby, Python and PHP (thanks Tom and Ross!).

  • New style tutorials – very directed recipes, that show you exactly how to make something specific in under 30 minutes. More on these in a future blog post.
  • Live tutorials – these are what we now call the ScraperWiki special sauce tutorials. Self contained chunks of code with commentary that you fork and edit and run entirely in your browser. (thanks Anna and Mark!)
  • Copy and paste guides – a new type of reference to a library, which gives you code snippets you can quickly copy into your scraper. With one click. (thanks Julian!)
  • Interactive API documentation – for how to get data out of ScraperWiki. More on that in a later blog post. (thanks Zarino!)
  • Reference documentation – we’ve gone through it to make sure it covers exactly what we support.
  • Links for further help – an FAQ and our Google Group. But also for more gnarly questions asks on the Stack Overflow scraperwiki tag.

We’ve got more stuff in the works – screencasts and copy & paste guides to specific view/scraper libraries (lxml, Nokogiri, Google Maps…). Let us know what you want.

Finally, none of the above is what really matters about this change.

The most important thing is our new Documentation Policy (thanks Ross). Our promise to keep documentation up to date, and available alike for all the languages that we support.

Normally in websites it is much more important to have a user interface that doesn’t need documentation. Of course, you need it for when people get stuck, and it has to be good quality. But you really do want to get rid of it.

But programming is fundamentally about language. Coders need some documentation, even if it is just the quickest answer they can get Googling for an error message.

We try hard to make it so as little as possible is needed, but what’s left isn’t an add on. It is a core part of ScraperWiki.

(The quote in the title of this blog post is attributed to Dick Brandon on lots of quotation sites on the Internet, but none very reliably)

Tags: , ,

4 Responses to “‘Documentation is like sex: when it is good, it is very, very good; and when it is bad, it is better than nothing’”

  1. Michi May 25, 2011 at 11:51 pm #

    You should really document how one accesses the query string that is given to a view. It took me ages to find out that it is in an environment variable (in Ruby). So, for the record:

    query_string = ENV[“QUERY_STRING”]

  2. Francis Irving June 16, 2011 at 11:29 pm #

    Thanks MIchi, and wilco!

  3. Larry Czaplyski September 6, 2011 at 5:15 pm #

    I disagree. When documentation is bad it isn’t better than nothing. It is worse than nothing (if that’s possible), because thr user is stuck and now has no place to go.

Trackbacks/Pingbacks

  1. Scraping guides: Values, separated by commas | ScraperWiki Data Blog - August 25, 2011

    […] we revamped our documentation a while ago, we promised guides to specific scraper libraries, such as lxml, Nokogiri and so […]

We're hiring!