There's been a lot of hand-wringing in Silicon Valley about its diversity problem: the tiny percentages of blacks, Hispanics, and women hired in the heart of the tech industry. Companies like Apple, Google, and Yahoo now publish self-shaming diversity reports that show they're anything but. (Silicon Valley is also very ageist, but the diversity reports haven't gone there yet.)
The implication is that Silicon Valley is racist and misogynistic. That's not exactly right -- it readily accepts Asians, both native and immigrants. And it's long been welcoming to gays.
Why poor diversity matters to Silicon Valley's bottom line
Despite its purported desire and hand-wringing over diversity, Silicon Valley isn't a full part of the real world its technologies are supposed to serve. Silicon Valley's particular mix of favored people biases it toward rich people's convenience products and away from the needs of most of the world.
I believe that one reason nearly every startup seems to be working on mommy-replacement technologies -- from Uber to TaskRabbit, from Soylent to Mark Zuckerberg's planned AI household servant -- is due to its bias toward professional-class, coddled white and Asian young men. Or they're working on messaging apps and chat bots, so they can keep hanging out with their friends. Their worldview permeates the products they create.
Elitism is the cause of what seems like racism
That worldview is not racist or misogynistic, but it is elitist and narrow. Silicon Valley is not the land of Archie Bunker or of the nativist movement. It's the opposite, in fact: Silicon Valley is a liberal bastion that is also extremely elitist, what some call "liberals who can afford to be." And it has a big component both of mercenary economics (the love of imported workers and "never leave work" perks) and of libertarianism that leaves it up to the individual to find a way in, regardless of circumstance.
In other words, Silicon Valley is a classist society, and it shows in its racial makeup -- blacks and Hispanics are disproportionately poor in the United States, so they're much less likely to go to the handful of "best" schools that Silicon Valley treats as its talent pipeline. If you went to "regular" college, you won't be taken as seriously. And if you went to a community college, where many poor students start, you have a major disadvantage.
Women are voting with their feet -- and not for computing
As for women, there's a different issue going on. Women's participation in computer science education has been steadily decreasing for years, so the pipeline of women simply gets smaller and smaller every year.
Silicon Valley struggles to draw from that shrinking pool, partly because of the adenoidal male culture that is so prevalent in the Valley and partly because women have been increasingly drawn to work that serves people, such as health care and government, in meaningful ways.
These trends won't be reversed by getting girls to study science -- because they already are. However, they aren't studying computer science. Women are the majority of science and engineering grads now, but mainly in the life and other sciences.
Ironically, women may increase their presence in IT because Silicon Valley is not all -- or even most -- of the tech industry. We often confuse the two. Silicon Valley is as much a subset of the tech world as Wall Street is of finance and accounting and Hollywood is of the performance arts.
Silicon Valley is repelling blacks, Hispanics, and women
Its particular worldview is why Silicon Valley's racial and gender profile is much worse than that of the computer science and engineering in general -- which is not good to begin with. (If you want to see how bad it was a quarter century ago, check out an in-depth report I did for IEEE Software in July 1990 -- then realize it's not amazingly different today.)
The good news is that blacks and Hispanics are growing, although slowly, within the computer and engineering professions. The bad news is Silicon Valley isn't tapping into them and, in fact, is repelling them. Both the industry at large and Silicon Valley in particular seem to have turned off women entirely.
I've looked at about a dozen recent studies, and they all show similar trends: Black and Hispanic participation in the science and engineering fields has improved significantly since the 1980s, though it remains smaller than their respective percentages of the population. The proportion of women in overall sciences education is more than men, but they're strongly avoiding computer science and engineering. Women have in fact gotten into science, but not the kind the Valley practices.
The U.S. Census Bureau has a detailed breakdown of these employment trends, though the data is from 2012. The U.S. Department of Education has a simpler summary as well, also from 2012 data.
The data show that black and Hispanic graduates end up working in government, nonprofits, health care, and established "legacy" industries outside of Silicon Valley, as do women.
It's not that Silicon Valley is actively turning down such people. As I previously mentioned, Silicon Valley isn't looking where many of these people are schooled. But worse these people are themselves not looking at Silicon Valley or the computing profession in general, as a 2015 survey from the National Association of Colleges and Employers shows.
In other words, Silicon Valley is a repelling force for them, not a magnet.
Elitism may get top talent, but not enough talent
People rightfully complain about elitism in the leadership of governments, the financial industry, and business in general. You practically have to go to the right schools -- Harvard, Wharton, Princeton, Yale, Oxford, Cambridge, ENA, IITD, and so on -- to rise to the top. And being a man of the majority race still matters greatly.
Silicon Valley is similar -- University of California (Berkeley, Davis, Irvine, Los Angeles, San Diego, Santa Barbara, and Santa Cruz campuses), Cal Poly San Luis Obispo, Stanford University, Santa Clara University, University of Southern California, University of Washington, MIT, Carnegie-Mellon, University of Illinois at Urbana-Champaign, University of Michigan, and University of Texas at Austin are among its vastly preferred sources of new talent.
That's not to say you can't get in from elsewhere -- but it's harder. (Ironically, Silicon Valley once was much more open-minded about its talent: Steve Jobs, Larry Ellison, Tim Cook, and Paul Allen did not go to elite schools, and some never even graduated.)
Silicon Valley sees itself as a meritocracy, where the cream rises to the top. The problem is it sources its milk from only a handful of farms that fit its worldview. There's much more cream, but Silicon Valley is too elitist to see it -- or for those other "farms" to want to join the Silicon Valley co-op.
It's a waste to focus on only the cream of the "right" talent pools. That means you're throwing away the milk and whey for the cows you have and ignoring the cows you don't. Silicon Valley needs to figure out how to use it all, like any sensible economy would.
When you say you have a big, growing talent shortage, you should stop limiting yourself to the (ostensibly) top talent and figure out how to use all that other talent. But Silicon Valley hasn't figured that out.
What Silicon Valley can do
To really be diverse, Silicon Valley has to do more than issue diversity reports. It needs to actively recruit -- and develop -- students outside the elite bastions. Like sports teams, it needs to send scouts out broadly, not only to the same dozen universities, and identify and connect with possible talent everywhere. If Silicon Valley were as focused on the non-elite talent pool as it is in hiring foreign workers, its diversity statistics would be much better.
Open codeathons are another approach to getting talent from outside the usual sources, but there's a lot more to computer science and engineering than coding. And such direct competitions may not attract women as much as men, so don't stop there. Look for other ways for talented people to be able to identify their talents.
Silicon Valley likes to use benefits to attract employees: high salaries, free dry cleaning, chauffeured buses, games at the office, and free food. But that's all about attracting and keeping the talent it's already identified.
It hasn't figured out how to go outside its comfort zone or create attractive work environments for other people. Breaking out of any culture is hard, and it's equally as hard for those to join a culture they see as alien, especially when they have other options.
But an industry as creative, dynamic, and purportedly merit-oriented as Silicon Valley should have the best chance of figuring this out. More than anything, it has to try different, do different, and expect different if it wants different. Can Silicon Valley do that? I'm not sure. But until it does, we'll keep seeing hand-wringing about tech diversity and diversity reports that don't change.