Canada’s great big data map

Big data is usually associated with Facebook, not farming. But as Ian Macgregor writes, harnessing data from agriculture, mining, forestry and other primary industries could be the next big economic opportunity in Canada — if we do it right. This is part of a series of excerpts from essays commissioned by the Centre for International Governance Innovation.

David Thompson travelled approximately 90,000 kilometres across North America by foot and canoe in the late 1700s and early 1800s. He used the data he collected to create what he called the “great map”— and in the process unlocked the commercial potential of North America.

Big data is as important to Canada in the 21st century as Thompson’s topographical data was in the 19th century. According to IBM, in 2013, 2.5 exabytes of data was generated daily. This data comes from everywhere: sensors used to gather shopper information or industrial machinery performance; posts to social media sites; digital pictures and videos; purchase transactions; and cellphone global positioning system signals, to name a few.

It is obvious that the combination of big data with modern machine learning will unlock new commercial opportunities and significantly reduce the environmentalimpacts of canada’ s biggest industries through continued optimization and by identifying and solving new problems and challenges.

Canada has led the world in the primary industries: mining, energy, forestry and agriculture. But for the most part, the focus has been on digging, cutting and planting followed by selling after primary processing. The opportunity — the “big idea” — is to enable transformative change by collecting and cataloguing the right data for rapid application of machine learning and artificial intelligence (AI) to these industries.

The sensors to collect data — the expensive part — are already in place. But so far, efforts in big data in the primary industry space have been by dominant industrial players collecting proprietary data to improve their competitive position. For example, John Deere collects self-driving tractor information, and GE collects gas turbine maintenance information.

What is missing is a vision for public accessibility of big data (with suitable authorization access to ensure appropriate protections for security and privacy) to accelerate and unleash broader, continuous, cross-sectoral innovation. To produce broad benefits for Canadians, this data must be intelligently organized and stored, and made available on an opensource basis, like the libraries of old.

Canada is well positioned to take a leadership role in the creation of such a library, by bringing together the know-how it has fostered in its primary industries and its emerging leadership in machine learning and data science.

Canada has an historical example that is unique in the world regarding the success of an open-source library for primary industry.

The transfer of mineral rights from the federal government to Alberta after the discovery of the Turner Valley oil field south of Calgary in 1914 led to the establishment of what has now become the Alberta Energy Regulator (previously called the Energy Resources Conservation Board). One of the board’s first actions was to require public reporting of key attributes of production, geology and reservoir performance, which formed the basis for a comprehensive historical library on Alberta’s resources. Everything related to well performance and reservoir is recorded and becomes public after a one-year period following drilling.

An unintended, but beneficial, consequence of this early idea for public reporting and archived information was to lower barriers to entry for oil industry entrepreneurs. Free public access to what had traditionally been proprietary data spawned large-scale resource development in a competitive environment in Alberta that continues to this day.

The public model developed in Alberta has now been recognized as a key enabler of the rapid and continuing entrepreneurial development in Alberta as well as a best-in-class model for petroleum resource regulation in other areas of the world.

By collecting the raw data and making it open source, new big data businesses will be built and sustained in Canada, enticed by the three essential ingredients for success: the right type of data; the subject matter experts who can help identify pressing problems; and a large domestic market.

If the organizational structure is developed to link young Canadian big data professionals with Canada’s deep industry expertise and support them to found new enterprises, primary industry in Canada and around the world can be revolutionized.

These young professionals can draw canada’ s next great map.

Concurrently, a big data entrepreneurial ecosystem system must be developed that will encourage Canada’s best and brightest to pursue these data-driven opportunities at home, rather than leaving for opportunities south of the border. This ecosystem should provide managerial support for new datadriven businesses, together with small amounts of capital for new ideas that have merit.

The train has left the station, however, so we have to get moving—or a promising public economic engine will end up as a merely private profit motor.