Special Issue


Bright Future or Cautionary Tale? How the Bay Area Shapes the Future of the U.S.

Winter 2019 Article

Data Science: Past, Present, and Future

By Vijay Mehrotra

As a college student, my grand life plan was to complete a liberal arts degree, get an MBA and become some kind of manager or executive. My freshman Calculus professor, however, predicted a different future. “According to some recent research,” he told us one day, “twenty years from now, more than 75% of college graduates will be working in jobs that don’t even exist yet.” A decade later, I found myself leading a small Silicon Valley consulting firm named Onward, and I was working in a profession that had not been conceived of when I started college and that would only be formally named many years later.

At Onward, we were doing the type of work that has come to be known as “Data Science.” There were not many of us in the field when we started in 1994, but today there are thousands of companies specializing in this field, many of them based here in the Bay Area. Though the job title did not even exist when I finished school, the Harvard Business Review recently dubbed Data Science “the sexiest job of the 21st century.”

To see how this young profession has evolved, I recently paid a visit to a consulting firm called Silicon Valley Data Science (SVDS). Walking into a tree-filled open courtyard that was ringed by two floors of office suites, I was hit with a deep sense of déjà vu: this building in Sunnyvale was an exact replica of the building in Palo Alto that had housed Onward’s first office. The two were separated by only two CalTrain stops – and about twenty years.

What I found inside the SVDS offices, however, was very different from what I remembered. Onward started by finding an anchor client and then slowly growing our business from operating revenues, eventually reaching 30 employees after eight years. By contrast, SVDS launched their company by raising $3 million in venture capital based on a healthy initial valuation, and already had 45 professionals on staff by their second anniversary. During our era, “math” and “data” were four-letter words we purposefully did not include in either our company name or on our business cards for fear of scaring off prospective clients. Today, however, there is an almost universal buzz around “predictive analytics” and “big data.” This rapid evolution and embrace of Data Science is the combined result of a number of powerful trends in the technology world and the business environment over the past two decades. Most notably, the cost of computing has plummeted and consequently the capability of data analysis software has exploded. In addition, data storage has become drastically cheaper, and software vendors have been busy creating data structures and reporting capabilities to give businesses a way to look at all that raw data that is laying around. The Internet has emerged as an amazing platform for capturing massive mountains of data, an inexpensive laboratory for conducting statistical experiments, and a superhighway for enabling information to be moved at amazing speed.

Over the past two decades the business world has grown increasingly competitive, pushing managers and executives to search for competitive advantages wherever they can. Most of today’s business leaders have grown up with digital technologies at their fingertips while seeing their companies grow increasingly dependent on information systems. Given all of this, it is not surprising that innovators in a wide range of industries have discovered that their rapidly growing piles of data – when combined with the right amount of mathematical elbow grease – can be successfully mined for gold.

So many of these developments have deep roots in the Bay Area’s technology industry. And in recent years, the growth in the mobile computing, social networking, and cloud computing has further fueled the growth in demand for services and software products that depend on Data Science. To address the needs of the business community, the University of San Francisco (USF) has had a thriving graduate program in Data Science since 2012.

Just as my freshmen Calculus professor had foreseen long ago, this constantly evolving ecosystem continues to lead to new job categories. Most recently, the growth in cloud computing has led to widespread adoption of subscription-based software pricing, which has spawned the rapidly growing and increasingly important role of Customer Success Manager. The job of the Customer Success Manager is to do whatever it takes to enable a software company’s customers to capture the benefits associated with the software package (many of which feature embedded Data Science concepts and methods).

Not surprisingly, the Bay Area technology industry is the world’s biggest hotbed for Customer Success Management professionals. Over the past several years, many of USF’s MBA graduates have become leaders in this new profession. Today, USF’s MBA Program includes the world’s first formal university initiative focused on Customer Success Management.

It is impossible to see too clearly into the future. But we can be sure that the Bay Area will continue to be at the forefront of technology and business innovation in the years to come, and that Bay Area universities, such as USF, will need to play a key role in educating professionals for many newly emerging vocations.

Vijay MehrothraVijay Mehrotra is a Professor of Business Analytics at the University of San Francisco and a columnist for Analytics magazine. His research has appeared in the MIT Sloan Management Review, Production and Operations Management, Manufacturing, Interfaces, and Manufacturing and Services Operations Management. He is also currently writing a book about the human side of creating business value from Big Data and Business Analytics.