Scraper Code

The "back-end" or "scraper code" is a collection of Perl scripts that access government websites to create and update the legislative database source data.

The scripts are in the process of being made publicly available.

The source code of the back-end are licensed under the GNU AGPL. In short, you may only make modifications to the code if you make your modifications publicly available. I am very serious about these terms. Again, you can download and use, but you cannot enhance the code without sharing your enhancements.

You can check out some of the back-end files with Subversion:

svn co svn://occams.info/govtrack/gather/us

If you checked out the files into the us directory, the scripts expect that there is a data directory along side the us directory. It's the same data directory as referenced above.

You will need a whole bunch of Perl modules to run the scripts. The best way to figure out which is to look at the use directives in the scripts, or to just run them and see what's missing.

Some scripts have some dependencies on files that I haven't made available yet.

And I haven't yet written any documentation on how to use the scripts. Sorry!