Back in April we introduced “bill prognosis” (original post), a statistical analysis of how likely bills are to be enacted. Today we’re making a few improvements. Read on for more about it.
The GovTrack blog includes site news and occasional analysis of U.S. legislation.
From a new start page to improved maps, here’s what we’ve been working on this fall:
There is a new Start page to find something to track and get email updates for, and you can now get email updates on full text keyword searches. So if you want to track something that is not one of the subject terms listed on the bills page, you can now make your own search for it.
The Members of Congress overview page now has some handy information on the number of Members of Congress by party.
The missed vote %‘s for Members of Congress, such as on the page for Senator Coats, are now computed a little differently:
- A bug introduced in June caused some votes toward the end of 1976 to be skipped. That’s now corrected.
- For Members of Congress who served non-continuously in Congress (e.g. lost an election but won a later election) we were not counting their votes from their earlier terms. We now do.
- We now compute separate medians for the House and Senate.
- Because of #3, for Members of Congress that have served in both the House and the Senate, we now compute their missed vote % for their lifetime service only in the chamber they are currently serving in.
We avoid changing numbers just for change’s sake. These changes were a part of our ongoing cleaning-house. The software we originally wrote to compute missed vote %’s is now about 8 years old, and that was holding us back from making other improvements. So we finally bit the bullet and replaced it with something new.
And some other minor changes:
- Members of Congress with accents in their name, such as Rep. Nydia Velázquez, now show up in search results when you omit the accent, such as with a search for “Velazquez”.
- Pages for Members of Congress now properly include “Jr.” and other name suffixes, such as on the page for recently resigned Rep. Jesse Jackson Jr..
- On pages for votes, we now always display the options at the top in the same order: Aye/Yea, No/Nay, Present, Not Voting.
- The advanced bill search page now has a filter for the party of the sponsor. You can choose the party or select whether it is the majority or minority party, which is helpful when doing a multiple-Congress search since the majority/minority party changes over time.
We’ve also been making improvements to our district maps in preparation for the new districts for the new Members of Congress taking office in January. That lead to the creation of a new API for developers for maps and GIS queries.
And finally, we’ve also been advising Congress on how to make more legislative data available, and we are working on a collaborative project to make the legislative data we have more comprehensive and reliable.
If you’re interested in the math behind some of our statistics — the ideology/leadership charts and the bill prognosis scores — you might find interesting a talk I gave last week. I had the opportunity to kick off the application development track at the Law Via the Internet (LVI) 2012 conference at the Cornell Law School with my presentation “Observing the Unobservables in the United States Congress” [slides | video].
The political reality we know today is entirely manufactured. Can Big Data help us cut through the spin to see what is really going on? Yes it can. This talk will present several statistical techniques used on GovTrack.us to quantify what is really going on in the U.S. Congress, including applying Google’s PageRank algorithm to Members of Congress, principle components analysis on bill sponsorship, and logistic regression on the success of bills.
The slides have Python code samples for computing the statistics.
- Find Similar Bills: There are often multiple bills introduced on a topic and major changes to bills are sometimes introduced as entirely new bills. These related bills can be hard to find, especially if you want to look across legislative sessions. But now you can find a “similar bills” link on every bill page which will let you browse through other bills by how closely the text of the bills are similar. For instance, if you go to S. 657: National Blue Alert Act of 2011 and click the similar bills link you’ll learn that there is also a bill to create a Silver Alert.
- Search by Cosponsor/Committee: The advanced bill search now has fields to filter by cosponsor and committee assignment.
- We added links to OpenSecrets, VoteSmart, and Twitter on pages for Members of Congress so you can find more information about them.
- We analyzed Rep. Paul Ryan‘s legislative record in two posts: Ryan’s Record: By The Numbers and The VP Candidates Agreed on 52 Substantive Bills.
- And we fixed a number of bugs on the site. For instance, sponsors and cosponsors on bills from past years were shown with their current title (i.e. Rep. from District X) but now are shown with the title they had at the time they sponsored the bill. Our sponsorship statistical analyses now show up for historical Members of Congress, such as President Obama and Vice President Biden.
[GovTrack] addresses an increasingly important and complex challenge: finding, understanding, and tracking government legislation. GovTrak is a well-executed example of how the combination of data, tools, and analytics can deliver power to those who are seeking the truth.
Partisan politics drives us to look at differences. But during the time Rep. Paul Ryan served along side then-Sen. Joe Biden from 1999 to 2008, our VP candidates voted the same way on 52 substantive bills.
Here are the 52 bills which the two candidates both supported:
Major new laws:
- H.R. 4762 (106th): 527 Organization Disclosure bill
This bill amended campaign finance law to require “527 organizations” to disclose their contributors. This bill came four years before the “swift boat” TV ad run by a 527 organization hurt Democratic presidential candidate John Kerry in 2004.
- H.R. 3162 (107th): Uniting and Strengthening America by Providing Appropriate Tools Required to Intercept and Obstruct Terrorism (USA PATRIOT ACT) Act of 2001
This is the PATRIOT Act, the highly controversial law that gave the executive branch new criminal justice powers following 9/11.
- H.R. 3295 (107th): Help America Vote Act of 2002
Following the controversial close of the 2000 elections and its pregnant chads, this bill gave funding to the states to modernize their elections equipment.
- H.R. 1 (107th): No Child Left Behind Act of 2001
This hallmark law of the Bush presidency tied federal education funding to performance measures, like test scores.
- S. 3 (108th): Partial-Birth Abortion Ban Act of 2003
This law bans late term abortion.
- S. 1 (110th): Honest Leadership and Open Government Act of 2007
This law created new lobbying disclosure rules and gift bans in Congress.
- S. 900 (106th): Gramm-Leach-Bliley Act
This 1999 law repealed the financial market regulations that prevented commercial banks and investment banks from being one and the same. The repeal is considered by some to have been a cause of the 2008 market crash.
- H.R. 5140 (110th): Economic Stimulus Act of 2008
This bill, which created tax rebates, was an early response to the credit crisis that lead several months later to the world wide recession.
- H.R. 1424 (110th): Emergency Economic Stabilization Act of 2008
This bill created TARP, the program of massive investments in private banks to prevent the financial crises from worsening.
Finance, trade, and related laws:
- S. 2578 (107th): Debt Limit bill
- H.R. 10 (107th): Railroad Retirement and Survivors’ Improvement Act of 2001
- H.R. 3108 (108th): Pension Funding Equity Act of 2004
- H.R. 4 (109th): Pension Protection Act of 2006
- H.R. 4040 (110th): Consumer Product Safety Improvement Act of 2008
- H.R. 3009 (107th): Trade Act of 2002
- H.R. 434 (106th): Trade and Development Act of 2000
- S. 2796 (106th): Water Resources Development Act of 2000
- H.R. 1308 (108th): Working Families Tax Relief Act of 2004
- H.R. 1180 (106th): Ticket to Work and Work Incentives Improvement Act of 1999
- H.R. 4425 (106th): Cerro Grande Fire Assistance Act
- H.R. 2646 (107th): Farm Security and Rural Investment Act of 2002
- H.R. 1000 (106th): Wendell H. Ford Aviation Investment and Reform Act for the 21st Century
- H.R. 2622 (108th): Fair and Accurate Credit Transactions Act of 2003
- H.R. 3 (109th): Safe, Accountable, Flexible, Efficient Transportation Equity Act: A Legacy for Users
- S. 256 (109th): Bankruptcy Abuse Prevention and Consumer Protection Act of 2005 (also see H.R. 833 (106th): Bankruptcy Reform Act of 2000 and H.R. 333 (107th): Bankruptcy Reform Act of 2001)
- H.R. 4444 (106th): U.S.-China Relations Act of 2000
- H.R. 3167 (107th): Gerald B. H. Solomon Freedom Consolidation Act of 2001
- H.R. 4759 (108th): United States-Australia Free Trade Agreement Implementation Act
- H.J.Res. 97 (108th): Approving the renewal of import restrictions contained in the Burmese Freedom and Democracy Act of 2003.
- H.J.Res. 52 (109th): Approving the renewal of import restrictions contained in the Burmese Freedom and Democracy Act of 2003.
- H.R. 1828 (108th): Syria Accountability and Lebanese Sovereignty Restoration Act of 2003
- H.R. 2330 (108th): Burmese Freedom and Democracy Act of 2003
- H.R. 5682 (109th): Henry J. Hyde United States-India Peaceful Atomic Energy Cooperation Act of 2006
- H.R. 7081 (110th): United States-India Nuclear Cooperation Approval and Nonproliferation Enhancement Act
National security laws:
- S. 2845 (108th): Intelligence Reform and Terrorism Prevention Act of 2004
- H.R. 3210 (107th): Terrorism Risk Insurance Act of 2002
- H.R. 5005 (107th): Homeland Security Act of 2002
- H.R. 6061 (109th): Secure Fence Act of 2006
- H.R. 3275 (107th): Suppression of the Financing of Terrorism Convention Implementation Act of 2002
- S. 2271 (109th): USA PATRIOT Act Additional Reauthorizing Amendments Act of 2006
- H.R. 3199 (109th): USA PATRIOT Improvement and Reauthorization Act of 2005
- H.R. 1 (110th): Implementing Recommendations of the 9/11 Commission Act of 2007
Bills that did not become law (at least not under these bill numbers):
- H.R. 976 (110th): Children’s Health Insurance Program Reauthorization Act of 2007
- H.R. 6331 (110th): Medicare Improvements for Patients and Providers Act of 2008
- H.R. 4810 (106th): Marriage Tax Relief Reconciliation Act of 2000
(There were no substantive bills that the candidates both opposed. That’s because it’s unlikely a bill will get a vote in both chambers of Congress unless there is strong support for the bill. That’s the same reason why most of these bills did become law, and why most had bipartisan support.)
There are at least 111 bills in all in which Ryan and Biden voted the same way when you include appropriations/authorizations bills. For the full list, see this spreadsheet on Google Docs. There are, of course, many more bills on which they voted differently, and many more bills that did not come up for a vote in both chambers that they probably would have disagreed on had they had the chance.
For more on Ryan’s record, see my previous post:
Rep. Paul Ryan, the Republican party’s presumptive Vice Presidential nominee, took office 13 years ago. We can learn a lot from his legislative record as the congressman from Wisconsin’s 1st district.
Budget, taxes, and Medicare
During his tenure in Congress Ryan sponsored 75 bills, mostly related to the budget, taxes, and our government-run health care programs. Although he is known today for wanting to privatize Medicare, many of his bills attempt to reform Congress’s budgeting process in smaller pieces. His bill H.R. 5259 in 2002 would have changed budgeting to occur every two years rather than every year, in an attempt to make Congress’s time spent on budgeting more efficient.
The two bills he wrote that have become law modified excise taxes on arrows and named a post office. He’s currently the chair of the House Committee on the Budget. Budgeting hasn’t been going well. Last year the government almost defaulted on its debts because no budget had been passed! (The standoff between the two parties goes well beyond Ryan’s control, though.)
Ideology & Leadership
Our unique analysis of ideology and leadership in Congress puts Ryan right in the middle of the Republican House members:
Ideology is based on a statistical analysis that puts congressmen with similar patterns of co-sponsorship of bills closer together. Ryan co-sponsors bills that the middle of his party tends to co-sponsor. He’s neither extreme nor a centrist.
In this chart, congressional leaders are those representatives who tend to get a lot of cosponsors without necessarily cosponsoring other bills in return. Ryan is right about in the middle. But he is a little below the average leadership score of the 44 Republican representatives serving as long as Ryan.
Leadership is based on an analysis that’s similar to how Google decides which web pages to show first in search results. (More analysis details.)
Crossing party lines?
From Ryan’s position along the ideology axis of the chart above, you’d guess that he crosses party lines about an average number of times for House Republicans.
In a Washington Post story today that cites statistics from GovTrack, one former staffer said Ryan was all but compromising:
[T]hose who have watched Ryan’s recent career . . . say finding common ground has not seemed to be Ryan’s interest. “No, goodness, gracious.” said Steve Bell, a longtime Republican staffer on the Hill, who now works at the Bipartisan Policy Center.
But the statistics tell another story.
Of the 975 bills Ryan cosponsored since coming to DC, 22% were introduced by Democrats. That’s right in the middle. The freshmen members of the Republican caucus this Congress — many of them from the Tea Party — tended to cosponsor Democrats’ bills only 11% of the time. The Republicans except the freshmen did so 25% of the time. Overall, Ryan is at the 58th percentile, so a little more cross-partisan than most Republican congressmen.
Similar conclusions come from looking at the number of cosponsors of Ryan’s bills that were Democrats. Of the 75 bills he sponsored since he took office, 26% of his cosponsors were Democrats. Republican freshmen got 19%, Republicans except freshmen got 29%. Compared to the whole party, Ryan is at the 53rd percentile — he’s right in the middle.
Busy as always, we’ve got some more new things on GovTrack this month:
- Congressional District Maps now include redistricting maps for the upcoming elections. Except for Rhode Island — we’re still working on locating the new districts there. You can switch back and forth between your current district and the district you’ll be voting in this November.
- The bills page has a new Statistics Tab where you can compare the number of bills enacted in each Congressional session since 1979 and see when during a session bills tend to be introduced and enacted.
- The site now works better on mobile devices with small screen, and we’ll keep improving that.
- Bill text comparisons are now available for comparisons between versions of a single bill and between selected different but related bills.
- The font size has been increased throughout the site in response to feedback we got early on when the new site design went live in March.
- The search pages have been fixed to work on Internet Explorer, they are faster, and got some other usability tweaks.
Last post I wrote that I’d be changing the data licensing terms on GovTrack. I sincerely asked for feedback, and I got it. Gunnar, for instance, rightly pointed out that this isn’t everyone’s fight. He also noted that my dry sense of humor wasn’t really working.
The change I planned would have created substantial burdens for re-users of GovTrack’s data and yet would have had little impact, except possibly to annoy end-users. So I’m going to make a different change. This is a lot simpler:
You may not disparage services for being nonpartisan.
This will go into the terms of service to access the regularly updated raw data and API starting tomorrow. (It does not affect regular users of GovTrack or of any of the users of the tools that use GovTrack data.)
I don’t know of any current licensees that were a part of the boycott that started all of this, but I don’t know everyone who uses GovTrack’s data. If your organization can’t handle the new term, then I’m not above saying you can get your data elsewhere.
That said, the source code for the scripts that gather the data remain open source under the GNU AGPL license (see this github project for v2). And I do create special license agreements with other organizations as necessary. So there are at least two routes around this.
This all began because some random guys said some other company should be boycotted for being nonpartisan. It really had nothing to do with me. But as I explained to techPresident I was appalled that:
“This is the first time that someone’s called what our community does ‘evil,’” Tauberer said in an e-mail. “I don’t take that lightly. PCCC’s Rosenbaum had better stand behind that if he is going to be so brash. Is he going to take the links to GovTrack off of the PCCC web site? Because right now those links support the right’s ability to get the same information.”
I think the open gov community is used to me curmudgeonly complaining about various things. Sometimes I try to be polite. But, honestly, I’ve gotten tired of being mostly complacent. In June I called out Rep. Crenshaw for trying to slow down legislative transparency and got almost 1,500 letters sent to Congress about it. This month I’m calling out this ridiculous boycott. Yeah, this might be the start of a new pattern.
A colleague pointed out to me over the weekend that the open source movement has a long history of using licenses to promote ideological positions. The GNU GPL license — the ‘viral’ license that is part of the backbone of the open source world — says you can use my software if you believe in the same sort of openness that I do. The GPL also prevents licensees from exercising software patent rights — which is in many ways a political statement. One of the earliest leaders of the free software movement believes it is a moral imperative for computing technology to be free (free as in freedom). Over at Creative Commons, licenses make a distinction between commercial and non-commercial use, which is something that to a for-profit guy like me thinks is pretty arbitrary.
So I’m not the first to think that giving stuff away can come with substantive terms and still be open. Though in my case, the license is a terms of service, not a copyright license, so the comparison with open source only goes so far.
Data license terms to change in response to Netroots conference boycott of nonpartisan tool NationBuilder
I sent the following message out to the GovTrack data developers mail list. It affects anyone who uses GovTrack’s bulk raw data downloads and API.
Dear data re-users,
This is advanced notice that on July 20 I’ll be revising the terms of GovTrack’s generic license agreement by adding the following paragraph:
* During the time in which your organization is reusing GovTrack’s database, your website must block visitors referred by the websites of sponsors of the Netroots Nation conference. If you make the data available in bulk to others, your license agreement must carry over the same terms.
In techPresident today, Netroots Nation’s executive director Raven Brooks encouraged progressive advocacy groups to boycott NationBuilder, a non-partisan technology platform that helps campaigns build their websites, because NationBuilder sells services to right-leaning organizations. There’s a complicated history here that’s touched on in the techPresident article (http://techpresident.com/news/22556/nationbuilders-mammoth-deal-state-level-republican-committee-sparks-calls-boycott), but Brooks’s point seems to boil down to a belief that there can be no nonpartisan political tools.
GovTrack, and most tools that reuse its database, is a nonpartisan tool that has played an important role in political activism over the last several years on both sides of the political aisle. It is astonishing to me that anyone would think that technology infrastructure should choose sides. Especially since it appears that the sponsors of past Netroots Nations conferences have been users of nonpartisan political technology platforms like GovTrack. If they are going to boycott tools like GovTrack, they certainly won’t notice the change to GovTrack’s license terms.
Your feedback on these changes is welcome, especially if the feedback is in the style of satire. But this isn’t a joke. Ridiculous boycotts of technology startups require ridiculous responses.
A lot’s been happening here at GovTrack over the last few months. Here’s a quick run-down of the most recent changes:
- If you use GovTrack to build lists of items to track, the Your Lists page has gotten a much needed visual refresh. It also now provides RSS feed links for each of your lists (which we had in the old GovTrack but hadn’t yet brought over). Finally, note that Your Lists has now been tucked inside a new Track menu at the top of the page.
- We’ve put up a new page called Other Tools in the new Track menu featuring other ways you can track Congress besides using GovTrack.
- Bill pages have been slightly reorganized. They now highlight the major votes on bills so you can find the votes you are looking for faster. We’re also now starting to include links to news articles for bills.
- The Washington Post used our legislative data in an analysis of stocks being traded by Members of Congress published today, showing that conflicts of interest are pervasive.
- A lot of bugs have been fixed, e.g. voting absentee rates were missing for Barack Obama‘s time as a senator.
- The ordering of bills in the advanced search results has been improved with our “special sauce” that brings bills you’re probably looking for to the top.
- We’ve had a few great press mentions lately: check out the full list.
- For developers, we have a new experimental API to complement our bulk data downloads. And since launching the API we added to it information on roll call votes. The developer documentation has been improved.
We recently announced:
- We added legislative tracking for all 50 states. This feature is beta right now and will be developed over time. More in our announcement. (State information is made available through a partnership with LegiNation, Inc. and LegiScan, Inc.)
- Earlier this month the House’s bill to fund its own operations for 2013 almost rolled back transparency. Our advocacy effort was covered on the front page of The Washington Post. More in our blog post.
And coming up:
- You may notice that bill text comparisons are coming back. Our automatic red-lining feature is better than ever. I’ll write separately on this in a few weeks, since right now there are still some bugs to take care of.