Weekly Database dumps
SQL database dumps of the SourceWatch are updated daily. Warning: The filesize of the database is more than 95 megabytes, so downloading it can take awhile. These SQL dumps be read into a MySQL relational database for leisurely analysis, testing of the MediaWiki software, and with appropriate preprocessing, perhaps offline reading. The decompressed size as of 2005-06-22 is 617Mb. Please email bob AT sourcewatch.org to request a copy of the weekly SQL database dump file.
The database schema is explained in schema.doc. The cur tables contain the current revisions of all pages; the old tables contain the prior edit history. Approximate file sizes are given for the compressed dumps; uncompressed they'll be significantly larger.
Please do not use a web crawler
Please do not use a web crawler to download large amounts of articles. Aggressive crawling of the server can cause a dramatic slow-down of SourceWatch.
Note: This article is adapted from Wikipedia database download page. Further helpful information may be found there.