Tim Thoughts data/rstats/policy

Middle America in the age of Trump

My good friend posted this clip of Tom Brokaw admonishing Mr. Trump’s recent remarks, and asked “Do you think the old comment about losing Cronkite and losing the Vietnam war is akin to Trump losing Tom Brokaw and losing the White House?””

My reply was getting a bit lengthy, so I decided to post it here.

1) LBJ’s apocryphal quote about losing Cronkite (“If I’ve lost Cronkite, I’ve lost Middle America”) has been debunked in a similar article. This author argues that even if the quote is true, public sentiment had swayed against the Vietnam War months before Cronkite’s report and Cronkite was following the trend.

2) It is an interesting narrative in itself that a newsman can be the barometer of public opinion in America, whether in the 60s or today. However technological changes in 50 years have diffused the power of the media from the big 3 of ABC/CBS/NBC to so many different mediums of information (social media, talk radio, email threads from your aunt, weekly Skype vents by yours truly). Cronkite and Murrow had the privilege of monopolizing the news to Middle America every evening, today’s modern life grants no such privilege to Brokaw’s generation. While it is true that TV still reigns supreme for an older generation (who obviously are more reliable voters), the ability to choose your echo chamber keeps people like Brokaw from really making their impact.

3) Not to discourage your unshakable faith in American democracy, but recent polling has shown that much of Middle America do agree with Mr. Trump’s recent comments. One national online poll found that “almost two-thirds of likely 2016 Republican primary voters favor Donald Trump’s call to temporarily ban Muslims from entering the U.S., while more than a third say it makes them more likely to vote for him.” and a more localized poll done in Fresno and San Diego also shows this as well. If anything such polling will reinforce Mr. Trump’s belief that he’s saying the right things, and will continue to do so. Trump is a modern epitome of Nietzsche’s “what doesn’t kill you makes you stronger”, attacks by the media will bolster his credibility to a base that is already distrustful of the establishment (Trump literally goes around talking about how unfair the media treats him).

4) Will this statement cost Trump the White House? I’m still bullish on this. I’m still leaning towards a Cruz-Rubio semifinal when the smoke clears, and a near toss-up against Hilary in the White House. Trump’s aura of invincibility will shatter if he does not remain #1 in the polls, and he’s dropping in the crucial Iowa caucus (Ted Cruz just took the lead in the latest poll). I have more links and numbers, but this seems like a separate post. My short answer is that back in July, Trump was criticized by many across the political spectrum for criticizing Senator John McCain as not a war hero, because “I like people that weren’t captured”. I remember reading some punditry that argued that this was the beginning of the end of Mr. Trump’s candidacy (many saw this as an attack on military veterans, a valuable constituency for any party) and yet this did not affect his polling support at all. Trump may lose at the ballot in the springtime, but it won’t be because of one remark or next.

5) To bring it back around, Trump may have a claim to unlocking that unicorn of American politics that all campaigns strive for, and that is the claim of “Middle America”. Not unlike Huey Long or George Wallace, Trump’s supporters are flocking to a populist who speaks their fears on immigration, national security, and the economy. (For a familiar walk through APUSH, check out this article by John Judis for the long view on where Trump’s supporters come from). The instinct to dismiss Trump as a theatrical anomaly is to ignore the very real fears of his predominantly working/middle-class base that feel anxiety about their place (and America’s place) in a rapidly changing globalized and tech-oriented economy, and it also ignores the global context in which far-right/anti-immigrant political forces have seen recent electoral success in Europe as well. Whether Trump wins or loses, he’s already made his impact by shifting the political discussion on his terms. It will be up to us to assuage those fears, to debunk his faulty facts and statistics, to make our case on what type of America we want to live in.

And yes I’ve been binging the West Wing, can you tell?

The new guy

But how would you even know which of the Muggles are…Muslims?

Have you no faith in the powers of the Dark Lord? There is a simple Dark spell I will teach you all, an adaptation on “Homenum Revelio” that will alert you to the presence of a Muslim.

What do we do when we find one? We used to torment Aurors with Crucio, but if it’s just extermination you want, ain’t nothing wrong with Avada…

No no no, that’s not what I want. I want you all to build…camps.


To relocate them in! They’re a serious terrorist threat, don’t you read the news? Actually they probably have already infiltrated our media, maybe that’s why it’s called the Daily Prophet! They’re everyowhere

My lord, with all due respect…we’re wizards. Couldn’t we just kill them?

No! No we’re going to build camps, with the biggest walls we can make! We’ll get them to pay for it, it won’t cost the Ministry a dime, I’ll know how to negotiate with them. We’re going to make America great again now that I’m in charge. [Remember, I love the Muslims, I think they’re great people.] (http://www.huffingtonpost.com/entry/donald-trump-muslims_56660437e4b072e9d1c742d8)

Sir I really do protest. The old boss would have just slayed everyone, and the Ministry’s not likely to respond effectively now that the Chosen One is in his lame duck term. I would like to emphasize again that we’re wizards, with wands-

He’s too stupid and cowardly to acknowledge that radical Islam is an existential threat to our way of life! We’re under attack, and I’m the only one who can make America great again. Go, now, before I turn you into another Horcux like I did Rookwood.

I miss the old boss.

Plotting the Refugees with rCharts and Streamgraphs

In the honor of Thanksgiving, I’m plotting the countries of origin of the 300,000 refugees who’ve settled in the United States since the beginning of the Syrian conflict (March 2011). In light of recent political discussions about the presence of Syrian refugees here in the United States, I thought it’d be important to show how little the Syrians number in comparison to other countries.

Data disclaimers: I got this from the Refugee Processing Center, a database maintained by the State Department and Bureau of Population, Refugees, and Migration. Not gonna post the exact data wrangling code, but I cleaned a dataset looking at refugees coming in beginning in March 2011 (the start of the Syrian Civil War) to Novemeber 2015.

I will be using the streamgraph package by [Bob Rudis] (@hrbrmstr).

some minor data cleaning

refugees <- read.csv("~/Desktop/rstats/refugees.csv", stringsAsFactors=FALSE)
refugees<- refugees[,-1]
all_origins<- refugees %>% group_by(Origin, Year) %>% summarise(Total= sum(Total))

Here is your graph

streamgraph(all_origins, "Origin", "Total", "Year", offset = "zero")%>%
  sg_axis_x(20, "Year", "%Y") %>%
  sg_legend(show=TRUE, label= "Origin") %>%

Some feedback from my cousin when I showed him a rough demo was that you couldn’t see Syria well enough. And perhaps that’s the most relevant data point I got from my analysis, was that we don’t let in that many Syrians to begin with. Also, what the hell is going on in Burma and Bhutan? Apparently our national consciousness doesn’t extend to South/Central Asia, apparently there’s been some major conflict going on under our noses this whole time.

From this article on Bhutanese refugees, “In the late 1980s and early 1990s, Bhutan drove out more than 100,000 Lhotshampas, Dhungana’s ethnic group, as the government began to see them as a political threat. It wasn’t until 2007 that several countries agreed to take in some of the Bhutanese refugees, and some 66,000 Bhutanese have been formally resettled in the U.S. since then.”

To better illustrate my point, here’s the same data presented in a multi-bar stacked chart using rCharts. The default setting is Syria, but you can click on other countries (IRQ, AFG, BHU) to add those countries to the graph.

p1 <- nPlot(Total ~ Year, group = 'Origin', data = origins, type = 'multiBarChart', id= 'chart')


I used the countrycode package to quickly convert the longform country name to the standardized ISO 3166-1 alpha-3 country codes, check it out if you’re working with international data.

Final thoughts, would you believe that there’s one person from Canada in 2013 and one person from the UK in 2014 who resettled here as refugees? I wonder what their stories are, it’s the ones you don’t expect…

Living life in peace

It was my friend’s birthday yesterday. She’s studying for grad school half the world away, but I still feel closer to her than most people. She told me she was going out this weekend to celebrate with her classmates.

The Parisian attacks were directed against places of leisure and recreation. A football stadium, a concert hall, a restaurant. The sort of place you’d go out with your friends to celebrate things like a birthday, maybe some of those unfortunate victims were out celebrating themselves on a regular weekend night.

Kenan Malik wrote “What the terrorists despised, what they tried to eliminate, were ordinary people drinking, eating, laughing, and mixing. That is what they hated - not so much the French state as the values of diversity and pluralism.”

My initial reaction to the news was sheer horror and pain for the Parisians, then incredibility and disgust at the terrorists. My second reaction was to thoughts of my friend, and all my friends like he who are just living out their normal lives in bars and concerts and restaurants with people they love.

I wonder if these cowards with guns, if they’ve got love in their lives, if they’ll ever find that inner peace we all strive for. I hope that our leaders won’t overreact and fight fire with fire, and play straight into terrorism’s hand. I am certain that people like you and me can just go on living our lives without fear, we live it with the people we love. We fight back in human gestures like this fellow, who gave me chills down my spine with this simple act of beauty.

Happy birthday friend, I wish you and the world much love and joy. God knows we all need it, as we’ve always have.

Scatterplotting the Senate

Recently I stumbled onto this graph on [govtrack] (https://www.govtrack.us/about/analysis#ideology) which plots an “ideological” and “leadership” score based on a principal components analysis. I wanted to replicate a similar chart in R, and after playing around with a few packages, I decided on the new htmlwidget scattterD3 by [Julien Barnier] (@nozav).

(I’m just gonna do the Senate for now)


Get data from Govtrack and Sunlight

govsen<- read.table("https://www.govtrack.us/data/us/114/stats/sponsorshipanalysis_s.txt", header=T, sep=",")
govsen<- tbl_df(govsen)
options(SunlightLabsKey = "Enter_Your_Key")
sr<- cg_legislators(chamber="senate", state_rank = "senior", per_page = 50)
jr<- cg_legislators(chamber="senate", state_rank = "junior", per_page = 50)
sr<- sr[-38] #Sorry Menendez :(
sunsen<- rbind(sr, jr)
sunsen<- tbl_df(sunsen)

Do that data cleaning

names(sunsen)[12]<- "ID"
sunsen$ID<- as.integer(sunsen$ID)
senate<- inner_join(sunsen, govsen, by="ID")
senate<- senate[-41]
names(senate)[23]<- "party"
senate$birthday<- parse_date_time(as.Date(senate$birthday), "%Y%m%d")
senate$age<- as.numeric(floor(difftime("2015-10-26", senate$birthday)/365.25))

Here is your graph

scatterD3(x=senate$ideology, y= senate$leadership, lab = senate$name,colors = c("#ae0000", "#00008b", "#999999"), col_var = senate$party, xlim=c(0,1), ylim=c(0,1.01))