Information is the lifeline of a businesses’ sustainability. With a rapidly changing business climate, evolution of technology, new policies and regulation as well as the emergence of new competitors, energy and utility firms are turning to data to create new business models – a model that incorporates past, current and future predictions. Not too long ago, the process of aggregating raw data was fairly logical and straightforward, however today, it is complex, costly and time consuming. As the growth of unstructured data continues to escalate, so too are the pressures for CIOs to gain better insight, confidently predict outcomes and take actions that stand out amongst a crowded marketplace.

The amount of data generated by the Smart Grid is astounding. For example, smart metering inevitably increases the amount of meter data utilities must handle – generating on average 50 bytes of data per hourly read. Additionally a synchrophasor– a phasor measurement unit that tracks electrical waves across the power grid to monitor the health of the system- takes readings sixty times a second. This adds up to four-hundred ninety-four megabytes a day, one-hundred seventy-six gigabytes of data a year per synchrophaser. Today there are a number of devices in addition to smart meters being used in the energy and utilities industry to collect data, including line default detectors, sagometers which generate 12 readings per hour at 50 bytes per read and storage devices such as batteries that produce 100 byte reads per hour. Together, these devices create an astronomical amount of data.

With the abundance of information, utilities have to find new ways to cost-effectively and securely store, archive and retrieve a virtual explosion of new information. Additionally, aging physical infrastructures and IT assets are no longer sufficient to cope with the accelerated exponential growth of data. Smart Grids use sensors, smart meters, digital controls and analytic tools to automatically monitor and control two-way energy flow. This data can be used to shift electric load to avoid power outages and locate troubled components instantly; it also empowers consumers to make better decisions by providing information about their energy consumption. Additionally, smart grid technology allows energy and utility companies to understand power demand in real time so they can improve delivery during peak hours. The data also enables utilities to integrate distributed generation such as renewable energy assets into their power generation portfolio.

As with most technology advancements – the next generation Smart Grid brings with it more data as well as more ability and need to analyze this data. So without the appropriate data storage and management infrastructure, utilities are unable to reap the benefits that Smart Grid offers. Therefore, organizations must implement an infrastructure that provides efficient, automated management and retention of information. This will provide personnel with constant, reliable access to data where ever and whenever it is needed to improve overall business operations and customer relations.

Information availability is instrumental in keeping operations up and running and customer service levels at a premium point. In the energy sector, continuous and reliable access to information ensures that personnel can access data at any given point – whether it’s during an outage, or simply when they receive a consumer inquiry regarding billing and new service setup and relocation disconnect. Many utility companies are investing in technologies that provide automated performance management, application integration, and migration of data in order to meet the ever changing business requirements of managing a Smart Grid. Performance, capacity and reliability are also mandatory for utilities as without a constant stream of energy, countries could be gravely impacted with thousands of businesses and millions of consumers paying the consequences.

As the amount of data grows with the “Smarter” Grid, the need for analytics increases dramatically. Analytics will allow utilities to better predict energy usage, prevent failures, reduce outages, analyze customer response to pricing events, ensure grid reliability and security and efficiently manage generation and grid infrastructure assets.

Utilities are also perplexed by data retention issues, what Smart Grid data to keep, what to archive, how long to archive the data, which data should be used for analytics, and which data needs to be maintained to meet security, data privacy and legal requirements. All of this calls for a rigorous understanding of the importance of each piece of data to Smart Grid business processes. In addition, security, data privacy and legal requirements are constantly increasing. Failure to meet data security, privacy and legal retention requirements can lead to costly fines. That said, keeping massive quantities of data for long periods of time also comes at a cost. With the right data retention policies and technologies that support and enable those policies, utilities can store and manage their data more effectively. Data storage technologies that are key for this type of enablement are virtualization, data de-duplication, multi-tiered archiving, and data encryption.

The Smart Grid cannot function at the highest performance and capacity without a manageable data storage solution. Utilities are sourcing customized, highly available technologies that deliver the scalability and performance to meet today’s and tomorrow’s energy demands. The solution then is to find a host of data storage and management systems that provide automated performance management, virtualization, encryption, analytics, and deduplication that are designed to provide utilities with inexpensive, efficient ways to store and extract intelligence from massive amounts of data.

Mozhi Habibi is Strategy Manager, IBM Global Energy & Utilities Strategy & Solutions.

Mozhi Habibi is a member of the Global Energy & Utilities Strategy & Solutions team, responsible for leading IBM’s energy and utility solutions strategy worldwide. IBM is celebrating its centennial anniversary this year.