Daves Redistricting 2.0.2

Last night I uploaded Version 2.0.2. This fixes a couple of bugs with 2.0.1 and introduces a new feature that I hope you like… Change Colors!

You can now change CD colors and change the opaqueness of the CDs essentially making them lighter or darker)! I also made the Old CD lines a little thicker and changed the county line color.

Launch Page

Please let me know what you think. Also, check out the Help page for lots of good information.

This will be all the changes for a little while, unless a bad bug is found. I’m working on a plan for next year and will let you all know about that when it’s ready.


Daves Redistricting 2.0.1

A quick diary here to say I uploaded Version 2.0.1. It will show all the districts all the time now, fixing one of your main issues. It’s faster zooming, even with all the districts, but still not as perky as 1.0. On my laptop it does seem that panning is a bit slow for the large states, like NY, TX or PA, but acceptable. It is nice to always see all of the districts.

I’ll be checking the diary tomorrow (Sat) and periodically next week, but won’t be working on the app until after Thanksgiving.

Please keep the feedback coming. Thanks.

Link to previous diary announcing DR 2.0.

Daves Redistricting 2.0 [Updated 2]

Daves Redistricting 2.0 is now ready.

There are many changes from 1.0. It uses a Bing map, so you can see roads and aerial view, and pan and zoom normally. You can load and save files normally, too. You can save a view directly to a JPEG. And there are more features to help do the job of assigning districts to CDs.

I am very excited to be releasing this. You are the first to know! I hope you like it.

Fire it up and check the “How To” for a brief idea of what’s different. And the help file explains it all. You can load 1.0 files, too.

Please let me know of any bugs. I’m sure there are some in there. I’ve only tested on Windows, so Mac users, please let me know if it’s ok on the Mac.

Update: Here’s some more info and explanation of the big changes:

1) The background is a bing map instead of a canvas. The advantages are you can see the roads and you get pan/zoom in a way you are use to (using the pan and zoom controls or the hold-mouse-and-drag that are common to bing and google maps). The disadvantages are that rendering is slower necessitating showing only a subset of the voting districts at a time, and that hold-mouse-and-drag no longer colors (because it pans) so you have to double-click to change the mode to coloring and back. I’ve tried to mitigate the performance disadvantage by giving you control over showing those districts (All, None, Auto with control over Auto). Another disadvantage is that the colors are paler in order to show the roads thru. I plan to give you a little more control over that.

2) File Save/Load is improved. I hope you all find this to be a big win. I always disliked the necessity of saving your work in the obfuscated files.

3) Area views. Saving the 1.0 area maps in some XML that required another tool was a real hack. And I had really broken the look of those, too, with all of the crossing lines. All of that is gone and now you can save directly to JPEG. The only issue is that Bing does not allow its road map or aerial map backgrounds to be saved — proprietary. (I’m prevented from rendering the background into the bitmap that is translated into the JPEG.)

4) Labels: This may be harder to notice, but all of the labels (CDs, Counties, etc.) can be moved and resized. Hover over them to get the tool tip. Hold mouse and drag to move; Hold mouse and hit ‘s’ to make it smaller or ‘l’ to make it larger. You can even delete some of the labels.

5) The Cur CD check box is new. It lets you focus on the currently selected new CD, hiding all the others. If you switch CDs when this is checked, it will pan and zoom to the newly selected CD.

5) Find CD Parts is a new feature that helps find little pieces of a CD that you may have left behind when changing CDs around. It uses only the bounding rectangle, so it won’t catch non-contiguous pieces that are really close to the main CD or in especially snake-like situations, but it can help. I may try to implement a real contiguity check someday, but that is much harder.

6) There still is Auto Assign according to Old CDs. I’ve also added Auto Assign by quadrants — something I came up with. It does not balance the CDs, but starts you out with rectangular districts that you can then modify to balance properly. I’ve made a map for Texas using this and it’s reasonably compact. In the future, I hope to add additional algorithms that others have developed or will develop.

7) I didn’t include city shapes this time, because the road map helps you see where the cities are. However, the road map doesn’t show the actual boundaries, which can of course, be useful. I will look into adding those back. But remember that adding more shapes slows down Bing’s rendering, so I need to think about how to limit the city shapes to the ones that are really useful.

And keep sending feedback. I will try to make improvements over the next couple of months and then work on all the new data when it comes out, so the app is ready for really helping you and others next year.

[Update 2]

From the feedback it seems you all are facing two major issues:

1) It’s too slow, even with the “only show some districts” mitigation. And that mitigation is a pain because you can’t see as much as you need.

2) The Bing pan control seems to go wild.

For (1) I am trying a change that looks promising and, I hope, will get the speed to be close to 1.0 speed. I thought about this all day (while doing other stuff), tried a couple of things. I need to work on it more and it will be at least a few days, possibly after T-day before I would have this ready. So stay tuned.

On (2), I was finally able to sort of duplicate the behavior. I can see that if I hold the mouse in the pan control and then quickly (really fast) move it out of the control, the map does speed away in that direction. My guess is that on the Mac at least, the pan control is not calibrated to the mouse as well and is too sensitive, so you all hardly have to move the mouse to get the map winging away. Since I don’t do anything for that control (the Bing object does) I don’t have a good solution, other than using the drag on the map method of panning.


Redistricting App: How you can help get partisan data

I know it’s been almost 6 months since my last post. Time flies when you’re…working a lot; ok, I was having some fun, too.

A couple of people have sent in partisan data, which I’ve uploaded. North Carolina has been there since early June (thanks Peter!), and New Mexico is there today (thanks Neil!). And I fixed the VTD data for Indiana. (Thanks to all of you who reported that bug.)

Also, I’m ready to accept other partisan data from volunteers! Here’s how.

If you are able to track down 2008 presidential data that somewhat matches the 2000 voting districts (or census block groups) for a state, you can put that data in a .CSV file (comma separated values) and email it to me. The trick is that voting districts often change and many have between 2000 and 2008. Some states don’t have voting districts published by the Census Bureau, so that’s a problem. If you want to do more work, you can try to map the 2008 data back through any changes from the 2000 voting districts.

For CA, NY and TX, JeffMd created new shapes to (roughly) match 2008 voting districts. In general, I would not recommend this.

Anyway, check out the web page describing what to do and we can get more partisan data in the app so we can have more fun until the new 2010 census data comes out.

BTW, I will attempt to make some improvements to the app and will get 2010 census data in there after it comes out.


Redistricting App: New Features, New Data

Partisan Data a la Jeffmd is now available for TX and CA. Similar to NY, you have to click the “Use Test Data” checkbox near the top right before you select the state. These have newer population data (2008 I think) and the presidential data.

Secondly, I added 2 features: “Demographics by color” and “Partisan by color.” These are checkboxes in the Show section. Each colors all districts in the state according to demographics or partisan data.

More below the fold.

About the features….

When you click “Demographics by color, each district is colored by creating an RGB (Red, Green Blue) color like this: the percentage of the population that is White gives the Red value, the percentage Hispanic gives a Green value and the percentage of Black population gives a Blue value. The combination is the result. So districts that are redder have larger white populations, greener larger Hispanic populations and bluer larger Black population. I did not factor in Asian, Native American or Other population into the color. (I just tried to keep it simple for now.) You’ll notice sometimes districts colored black. That’s usually a large majority Native American district (the color value is 0, which is black). Some districts are colored white, which is the case when there is no population.

For “Partisan by color” It’s simply blue for the Democratic pct, Red for the Republican pct, and Green for Other.

Note: the Show Selected Old and New is shortened to Show Old/New CD. The coloring for this is now messed up, which has to do with other fixes I made, which are.

Fixes: (1) you should no longer see those weird dead zone triangles. I fixed the polygons. Except this messes up the Show Old/New CD and also the maps created by Save Maps.

(2) panning when you are way zoomed in is now slower and easier.

Note: if you already tried the California partisan data, you’ll need to manually delete the file vt06_d00_data_spec.csv in your Silverlight...mangled directory names…DaveRedistrictingCalifornia directory to force a reload of the corrected file from the server. (The first file had Hispanic and Other mixed up; the second still had some negative numbers; third time’s the charm).


Redistricting App Change

Quick update on the app. Because so many of you have been using the app (and it was discussed on the DailyKos front page last weekend) and because the test data for NY is so large, we’ve blown thru my bandwidth quota for the month. I’ve upped it a couple of times, and we’re over that, too. So far it seems the site still works, thanks to the grace of my host.

But, I’ve made a change so that once you load the date for a state, the app saves it on your machine in the same place as the drf.xml files. If you then refresh the browser and reload the state, it grabs the data from your machine, saving a lot of bandwidth and…bonus…making it a lot faster to load.

I’ve tested this with NY, NV and a couple of other states (and with combinations of voting districts and block groups) and it seems to work. Please let me know if you have problems.

(Also, I know about the Indiana voting district problem and will get to that soon.)


Some Apportionment Surprises in New Population Estimates

Election Data Services posted their reapportionment projections yesterday…and there are some big surprises.

Polidata has a good summary

States gaining based upon July 1, 2009 estimates projected to April 1, 2010: AZ +1; FL +1; GA +1; NV +1; SC +1; TX +4; UT +1; WA +1.

States losing based upon July 1, 2009 estimates projected to April 1, 2010: IL (1); IA (1); LA (1); MA (1); MI (1); MN (1); NJ (1); NY (1); OH (2); PA (1).

Thus, changes in seats compared to the previous projections based upon 2008 estimates would be: AZ gains only 1 seat while MO stays even and OR stays even while WA gains 1 seat.

Check it out: Washington would gain a seat; Arizona only gains 1; Oregon doesn’t gain any.

This table is from the EDS PDF. Reapportionment Estimates 2009

From the table you can see that Washington jumps to 432nd or 433rd in all the estimates. Also, Missouri, which had been projected to lose a seat would not lose any. Texas is still projected to gain 4 seats, but that 4th seat is much more tenuous.

2010 is going to be very interesting….

New York State Of Mind [Updated]

New York State now has partisan data in Daves Redistricting! Thanks to jeffmd for the ton of work he did to make this happen. Please read below the fold before trying this!

Jeffmd created all new shapefiles for New York voting districts. The population numbers are 2010 estimates (gleaned by Jeff from available data), so there’s no need to check “Use new population estimates.” And the partisan data (2008 presidential election) was put together by Jeff to match those districts.

To access this new data, you need to check the “Use Test Data” checkbox in the upper right corner of the app before selecting New York State. (Because the data format is different than I have been using, I’ve separated it into a separate directory on the server.)

Warning: the shapefiles file is very large (17MB). (Texas has the next largest at 7MB.) So it will take minutes to load — go get coffee.

As most of you know, NY will almost definitely lose 1 CD after 2010. I started trying to build a new plan (as I tested out the new data and software changes). Quite a challenge. You already have various minority majority districts in the NYC area (Black majority as well as Hispanic majority). Keeping those balanced correctly while shifting to accomodate 1 fewer district will be a challenge. (And interesting to see neighborhoods in Brooklyn where McCain kicked butt — those of you from the city understand this much better than an outsider like me.) Now, the Hispanic demographic is not broken down further, which could hinder your job. The Census Bureau does have this data and I may be able to make that change in the future, but for now, this is what we have.

Enjoy redistricting NY. Don’t stay up too late and jeopordize your job or marriage!

Update: I uploaded a new version of the app and I think you are going to like this!

(1) Fixed percentages so that they are calculated by rounding. This was simply my coding error.

(2) Made the sizes of the CD labels smaller for the smaller districts. Especially good for NYC.

(3) Made it so you can drag the new CD labels where you want them! Woohoo!

First (little bit) of Partisan Data in Daves Redistricting

Last night I uploaded upgraded software that can handle partisan data. The state of Maryland is the first state (and only so far) to include that data. There are some caveats (below the fold). Also, keep checking here for ways you can help get more partisan data.

[Update: there was little glitch so partisan didn’t show up, but it should work now! Thanks!]

Daves Redistricting App

One other thing that’s new: all states where the Census Bureau has voting district shapes from 2000, except New York, now have voting districts (in addition to block groups for many) enabled in the application. (New York will be added soon with new voting districts [thanks Jeff!])

The big caveat:

The voting districts shapes and population data come from the 2000 census. Many states changed their voting districts between 2000 and 2008, so the 2008 presidential data does not match completely. For Maryland, for example, over 200 new voting districts were created during that time, so only 1600 of 1800 match the 2000 districts. The application lets you know when this happens and then simply ignores the new districts. This could be improved if someone is able to merge the new districts data back into the old districts.

I will be asking for help to pull together partisan data for the other states. Mostly this will involve putting the data into a particular form, where the districts are identified in a way that matching the district shapes. Keep checking here (next weekend) for information on that.

A few states, important ones for redistricting, did not give the Census Bureau voting district shapes. Those states are AZ, CA, FL, OH, OR and WI. The app still supports only block groups for those. How to get partisan data for those is a good question. it could be that someone can map voting district data into block groups, or produce voting district shapes. Ideas are welcome.


California (and more) [updated again]

[Update 10/13 3:41 PDT: the Hispanic data for California, Kentucky and Wisconsin was incorrect. I’ve now uploaded the correct data. Thanks to CalifornianInTexas and nico for noticing.]

Now in Daves Redistricting!

My time coming, any day, don’t worry about me, no

California, preaching on the burning shore

California, I’ll be knocking on the golden door

Like an angel, standing in a shaft of light

Rising up to paradise, I know, I’m gonna shine

I added the data for California to the server last night. I had to make a little modification to the code to handle the new format for block group data. Both these states use block groups because no shapefiles for voting districts were provided to the Census Bureau.

Warning: for California, the Assign Old CDs operation took 10 minutes on my desktop machine (3 year old dual core AMD chip, 2.2GHz chip speed, 2GB RAM, Windows XP — CPU bound operation). But it works!

I haven’t totally verified that everything is correct, but the population numbers look right and the assigned old CDs look reasonably close.

[Updated] All states with more than 1 CD are now supported. (OK and RI fixed!)