Tim Thoughts data/rstats/policy

Goodbye Kobe

Kobe Bryant’s greatest moments will forever live on in Youtube reels and I’m thankful that I can always revisit them. Years from now when the memories fade, we’ll use them as ammunition in arguments with our younger coworkers and family members who doubted that anyone can play like that.

Here’s what I remember, the good and the bad.

The Utah airballs as a young gun. his first match-up against MJ. The 4th quarter comeback against Portland in the 2000 WCF, ending with crossover on Pippen leading to the famous “Bryant… to Shaq!” alley-oop. Game 4 when Shaq fouled out at Indiana. The first ring of three, a dynasty is born. Boulder, Colorado. A 3pt moonball buzzerbeater at Portland to clinch the division in 04. The feud, and Shaq leaves. The Smush Parker/Kwame Brown era begins (the Dark Ages, to me). Getting 62 in 3 quarters. Averaging 43 points for the first month of January 2006, peaking at the 81 vs Toronto. “Bryant for the win…BANG!” to go up 3-1, though it wasn’t meant to be. Trade rumors to Chicago, cursing out Bynum in the parking lot. Hope returns with Phil, and Pau Gasol. The train wreck of 39 point beatdown in Game 6 of the NBA Finals, as the hated Celtics steamrolled Kobe’s Lakers to win their first title in decades. Redemption through Team USA, taking over the 4th quarter to beat Spain for the gold. Those stupid and endearing Kobe and LeBron puppet commercials, hyping a Finals matchup that would never happen. That ridiculous one-footed leaning 3 point buzzerbeater against Wade’s Heat in 2009 that no professional player should ever even attempt, unless you say “Kobe”. The 2010 Finals. 17 in 6 minutes in the 3rd quarter to silence the Garden. 6-24, 15 rebounds in Game 7, still the most dramatic game of basketball I’ve ever seen. No one knew at the time that it would be the end of an era for Kobe and the Lakers, they’ve never returned to the Finals since. It’s now LeBron’s time. The failed CP3 trade. Dwight and Nash comes, it doesn’t work out. The Toronto comeback. The Achilles tear, the free throws. The struggle to return. The end.

Youtube will capture most, if not all of these. It won’t capture my memories of hanging out with my bro, watching KCAL right before my dad leaves for work. Watch parties with my high school friends during the deep playoff runs. College runs at the rec center, with Shaan setting up like Sasha Vujacic at the line. Every time I had a hard final or project to study, I’d mentally channel my inner Mamba (because his legendary work ethic stories had started to spread across the Internet at that point). I had the utmost pleasure of growing up with Kobe, and it breaks me a little to say goodbye to him because it’s recognizing a part of my life is now closed. I don’t even watch basketball that much anymore, it’s not a part of my routine (I blame Twitch, election season and Reddit highlights). I’m not that invested in fantasy basketball as I used to be, I’m not too emotionally invested in the young guns of our roster, and I don’t see how I could be. Kobe Bryant may literally the last athlete that I have any semblance of an emotional connection to. We grew up together, from those Utah airballs to that last ovation he’ll have tonight.

One last memory. Here is the play by play from a game at Milwaukee Bucks in 2009. The Lakers were losing by 6 points with 90 seconds left. I was eating at the time, and when I glanced over at the TV, my first reaction was “Huh, it’s okay Kobe’s got this”. After scoring the next 6 points, Kobe nails the turnaround buzzer beater (from the same exact spot he missed in regulation). That’s what I’ll miss most, it was the inevitability.

Thank you for all of it. The sweat of work, the pain of failure, the glory of rings. Thanks.

Alt text

Election Analysis in R

I’m going to write my first tutorial in R. While I would never call myself an expert in R (especially when I follow the footsteps of so many great R programmers), I hope some beginners to data analysis may find use in my thoughts. A more experienced programmer would (and should!) pick out inefficiencies and errors in my code.

I’m thoroughly obssessed with the US election this year, if you’ve noticed my Tweets over the last few months. The question we’ll be looking to answer for this project, what are the geographic areas in which certain candidates are strongest? Media organizations like the New York Times, Los Angeles Times, and FiveThirtyEight have worked on fascinating interactives to help visualize the chaos of the Republican and Democratic primaries. While we’re not looking to re-invent the wheel here, it will be a fun challenge to replicate some of their ideas within the R ecosystem so that we may go a bit further.

One particular thing I noticed about the NYT map is that it is divided into counties, and the LAT map is divided by state. Let’s go a bit further, and take a more granular view and see if we can build a map focusing on Congressional districts. This is bit problematic because this data isn’t readily available on most major news outlets, as they tend to report results coming in by county (for example, CBS’s coverage of the Iowa caucus).

Fortunately there’s a wonderful website called the Green Papers, which has become a one-stop shop for political nerds wishing to learn the arcane rules of state elections. What their website lacks in graphic design, it makes up for in astounding substance if you’re trying to figure out how delegates are allocated. They’ve done a lot of the heavy lifting for us, which would be agreggrating congressional-level data directly from state election offices and party websites. We will be webscraping their data whenever possible and available, I trust no other site more on the Internet for accurate delegate counts (you’ll find that the AP, CNN, CBS, NYT etc etc all vary in their exact delegate count, as many states’s final count will not be solidified until state conventions, delegate selection…it gets a little nauseating thinking about it, ugh.)

So we’ll start with the annoying part of any data project. Getting clean data. Part one will focus on web-scraping, cleaning, data manipulation. These are the mundane tasks you delegate to your interns. Just know that it’s vital to your ultimate goal of some fancy regression or app.

PART 1 web-scraping

PART 2 data cleaning

PART 3 visualization in DT

PART 4 spatial mapping

PART 5 shiny

PART 6 deploying

I’m gonna deploy the final app so that you’ll have a sneak peek of what the result looks like.

Shall we begin?

library(rvest)
library(dplyr)
library(plyr)
library(tidyr)
library(stringr)

Election Mapping in Leaflet

Packages required so far for data wrangling, cleaning, scraping. Thank you Hadley Wickham and Kyle Walker per usual.

library(dplyr)
library(rvest)
library(tidyr)
library(tigris)
library(leaflet)

I scraped the majority of it from the Green Papers using rvest. Painstakingly going through each state to make sure there wasn’t any serious flaws or discrepancies, I had to redo this step quite a bit. Learned a lot about our country’s geography. Did you know that Alaska only has one Congressional district?

I also removed Texas from the analysis because it uses state senatorial districts, instead of the uniform Congressional districts. Get with the program, Texans.

Spoiler alert, also working on the Republicans. May I say what an absolute pleasure the Democrats at least award their delegates proportionally in some consistent fashion…sigh.

cd114 <- congressional_districts(cb = TRUE, resolution = '20m')
delegate_map<- geo_join(cd114, test2, "GEOID", "GEOID") 
#test2 is my dataframe of election results, will add later
popup <- paste0("Congressional District ", delegate_map@data$CD114FP, "<br>", "Hillary Win: ", round(delegate_map$percent,2))
pal <- colorNumeric(palette = "Blues", domain = delegate_map$percent)
map<-leaflet() %>%
  addProviderTiles("CartoDB.Positron") %>%
  addPolygons(data = delegate_map, 
              fillColor = ~pal(percent), 
              fillOpacity = 1, 
              weight = .3, 
              smoothFactor = 0.5,
              popup = popup)

No big surprises here. Hillary does well in the South and bigger states. Now that I have the results in CD format, it will be quite easy to join to Census data to examine further trends. I’m working on a Shiny app to dig into this more deeply, along with the Republican demographics as well. Thoughts and suggestions welcome.

American Census Survey in R with CartoDB

honors to Kyle Walker per usual, for his handy hack of the ACS package.

Orange County’s GINI index as of the American Census Survey’s recent 2010-2014 estimates.

library(dplyr)
library(acs14lite)
library(CartoDB)
library(rgdal)
library(acs)

Set up permissions (get API keys from ACS, CartoDB)

set_api_key('zzzzzz')
cartodb('username','apikey')

Code

acs.lookup(endyear=2014, keyword="Gini") #use ACS package to look up correct table numbers
oc<- acs14(geography = 'tract', variable = c('B19083_001E', 'B19083_001M'), state = 'CA', county='Orange')
oc_tracts <- tracts('CA', 'Orange County', cb = TRUE)
oc2<- oc %>% mutate(geoid = paste0(state, county, tract), gini = round(100 *(B19083_001E),1), moe= round(100 *(B19083_001M),1)) %>% select (geoid, gini, moe)
oc_tracts2 <- geo_join(oc_tracts, oc2, "GEOID", "geoid")
r2cartodb(oc_tracts2, 'oc')

No big surprise, Newport Beach’s pretty wealthy. Digging deeper in ACS data with CartoDB sounds promising though, it’s a lot easier to visualize disparities.

Hello darkness, my old friend

(Bear with me, this point takes a while)

Last night I was scrolling through some Reddit comments in r/politics (it’s one of my insomnia habits), and came across comments criticizing Hillary Clinton for her “hawkish” policies in comparison to Bernie Sanders. Such criticisms are not entirely unfounded, but it got me thinking, do these comments even remember real hawks? The Project for the New American Century, Rumsfeld, Cheney, and all those Bush-era neo-cons were targets of my high school idealism, and it’ll be a long time before I forget how they helped persuade the DC establishment and public into a prolonged and misguided Iraq War.

While Clinton certainly favors a strategy more willing to consider using American military capabilities regarding the Syrian/ISIS conflict, to lump her in with the hawks of the Iraq War is a bit unfair. The Libya intervention during her tenure as Department of State painted her more as a liberal internationalist, as did the other key figures of the Obama administration like Samantha Power and Susan Rice.

Further clues to Hillary’s internationalism was her appointment of Professor Anne-Marie Slaughter to the Director of Policy Planning. Professor Slaughter is a scholar best known for her work on international law and “the responsibility to protect” doctrine in human security. Another clue was her “smart power” doctrine, a twist on Joseph Nye’s IR concept of soft power, and her consistent stance on the necessity of partnership between civil society and government in the context of global development.

(Actually while writing this post I discovered this link to a Foreign Policy piece about Hillary Clinton’s ideologies which will better explain her ideas, because I’m getting distracted from my main point.)

My point is, I haven’t touched IR in a while. A night’s browsing through old blogs searching for more clues to Clinton reminded me of the joy I used to have in college. Memories of overdue library books outside my syllabus, of traveling to political science conventions to meet my favorite bloggers and professors in person, trying to get a half-decent picture with Joseph Nye. Memories of friendly yet rigorous debates with my classmates and friends, of plans and goals to work in this world somehow, ambitions in grad school to make it to DC someway.

And along the way, I gave up, I didn’t make it, I wasn’t good enough for it. It pains me to wonder, what if I had been a more disciplined student? But then I wouldn’t have been me, I suppose, the boy who still keeps notebooks full of book annotations of subjects I never even took in college. All those notebooks with plans and ideas for a future that was infinitely blank, the world would be my canvas to paint with my ambition to help make this world a better place.

And reality, and costs, and burdens, and home. The closer I got the more I realized I wasn’t cut out for it. I could barely handle my Masters program, let alone the discipline for the PHD. I enjoyed my DC internship, but that beautiful city was hollow to me at that time, it wasn’t mine to keep. Even the little things mattered, like Facebook pictures of my friends’ birthdays that I missed. And of course, there was Luna, who showed me worlds and futures that I never even imagined. In the end, my place was home.

And for the last 18 months, I’ve tried to find home again. I’ve experienced new memories with my old friends, rediscovered the beauty of Los Angeles with new friends, dated after Luna, found some semblance of community with the Hack4LA/tech stuff in the city, cared a bit more about local California politics and issues, and went to 3 U2 concerts in a row, gotten closer to my brother and cousins. I’ve unfortunately also experienced my parents’ divorce, leaving my job, and lot of indecision and anxiety about my next steps in life.

I’m writing this because I miss you. The old you, the young one whose dreams exceeded his reach, before he knew the costs of reality. I’m not saying I want to go back and be you again, I just miss being that inspired and thrilled to even glimpse a bit of you, in my old bookmarks and blogs and notebooks. Of politics and policy, of my first love, how I miss you…

I remember standing in a Starbucks parking lot late at night, just venting into a Skype conversation with a friend halfway across the world, just because of immigration, of my heritage, of the boat people, of my parents. Even during my little autumn exile from a social life, only Ian Burnside was clever enough to rouse me by texting me questions about Donald Trump, of the election, of things to come. I don’t know how, but our conversation turned to nuclear proliferation the other night, and it triggered all the research I did for Model UN prep and suddenly we’re discussing second strike capability, of Scott Sagan, of a world I thought I left behind. Apparently I am the type of person who checks Reddit threads at 2am when I can’t sleep, in forlorn hopes I’ll find nuggets of the intellectual rigor I once found in those blogs, those books, those conventions. Watching the West Wing on Netflix has been painful and joyful, for so many reasons.

There’s a lot about the new me, that I really embrace. I care more about family responsibilities, of local communities here in Los Angeles. I’m more realistic with my financial and social goals, which didn’t really exist back in college honestly. I really get a kick out of the tech/data/R/stats stuff I’ve been teaching myself for the last 2 years or so, I’m hungry to really get better at this stuff. But I’m not as hungry as I could be, I’m more motivated by the polls, by my Twitter community of political journalists and data geeks, by debate transcripts, by Internet arguments I shouldn’t get involved in…

I just wish this new me can find some way to make the old me proud of my future self. Is it impossible rekindle my first love of politics into a more meaningful and realistic career within the social parameters and responsibilities of my life? I wrote a reflection not unlike this on my new blog back in August here, and not much has changed. I’m more conscious of my failings and weaknesses, and what I have to improve in my work and life habits to finish my goals (like…not stay up so late writing and reading).

A lot of work left to do in 2016. A lot of missed connections and friendships to rekindle, a lot of people yet to meet, a lot of Shiny data visualizations to build, a lot more books to read. And who knows, I may yet find the best of me is still to come. Despite all my flaws and failures (and there are many), there’s something about the call to service that keeps me motivated and that is something I can never compromise on.

So thank you, the friends who’ve fought for me, who’ve been there for me, the unsung heroes of my life who I am truly undeserving. This is one of those moments where “I love you” is too ordinary or clichéd, and only my actions will be proof. I’ve been absent by my own standards (see Thanksgiving post), and that is not me being my best self. I want to be my best self, I’m not even close to there yet.

I hope 2016 brings all of us a bit more joy, a bit more love, a bit more purpose, and a bit more clarity to the underpining ideologies of our presidential candidates. See you on the other side.

Alt text