Tag Archives: technology

An introduction to the Big Data Landscape

What is Big Data?

Data has always been an important asset in every industry. Since the early days of the information age, business intelligence and descriptive statistics have been used as the standard tools for extracting information and make important decisions from all kinds of collected data. However, as the cost of collecting, storing, and processing data has been dropping exponentially, the amount and the diversity of the data has reached the point where traditional approaches are no longer feasible. In fact, the term Big Data is often used to refer to any data that requires new techniques and tools in order for it to be processed and analyzed.

A more formal definition of Big Data was introduced by Gartner in 2012, in which the well-known 3Vs – Volume, Velocity, and Variety – were used to characterize Big Data. Since then, the 3Vs model has been expanded to several other characteristics, including a fourth V for Veracity and more recently a fifth for Value. Without going into the details of each V, and resisting the temptation to look for a sixth V, we could also look at Big Data from the point of view of the new set of technologies that are helping to solve the challenges in collecting, managing, and analyzing Big Data. These technologies include cloud computing and cluster computing (with Hadoop MapReduce as the most well-known example) for data storage and manipulation, and machine learning for data analysis.

What value does Big Data bring?

The value of Big Data comes from two main use cases: as a source of analytics, and as an enabler of new products and services. In the first use case, big data analytics is used to improve an existing business model by revealing insights from data which was previously too costly to store or process. Amazon’s recommendation system, USPS’s preventive maintenance system, or Walmart’s demand forecasting system are all very good examples. These companies track, collect, and store all available data, from customer transactions to social data, from GPS trails to geographical and meteorological data, then combine them together and use big data analytics to produce high value actionable insights. This would not be possible without the new big data technologies such as cluster computing and machine learning.

In the second use case, big data technologies open up completely new business models and introduce new products and services. Many recent so-called unicorn startups, such as Airbnb, Uber, or Snapdeal, are founded on and enabled by big data analytics. Their products have unique features thanks to the new technologies that they are using. Without following the big data approach, their products would not be able to compete against all the traditional business models.

Big data at eSmart Systems

Since the early of the company, big data has always been considered as our core value. Big data analytics and machine learning are involved in many of our products.  They are used to forecast electricity demand at substation level, segment customers based on their power consumption patterns, and implement demand response strategies. We have also started to use machine learning to automate the analysis of power lines imaging surveys autonomously conducted by drones with the aim of assisting failure detection and preventive maintenance.

Written by:
Dang Ha The Hien
Dang Ha The Hien2
PhD-student, eSmart Systems

Complete overview of your energy consumption with new app

In Mid-May Ringeriks-Kraft and eSmart Systems introduced the mobile app Smartliv. Smartliv provides customers with a complete overview of their energy consumption and the ability to gain greater awareness of their energy consumption.

“Ringeriks-Kraft was the first in Norway to deploy smart AMS-meters to all customers, and since we have actively worked to develop new products and services for our customers. Launching the Smartliv app is the first step in a very exciting direction,” says CEO in Ringeriks-Kraft, Ole Sunnset.

The app is the first of its kind, and it sets a new standard for customer relationship management in the energy industry.

Saving money through increased awareness

Smartliv personalizes saving tips showing how customers can save money and the environment with simple measures. It also enables customers to compare their power consumption and compete with those who live nearby.


Ringeriks-Kraft is a pioneer in an industry undergoing major change.  It was the first in Norway to deploy AMS meters to all customers within its concession area. With Smartliv, Ringeriks-Kraft takes a leading position in the Norwegian market, once again showing its pioneering power. We are proud to be the chosen supplier of the Smartliv system and thereby participate in shaping development of the energy industry,” says Knut H. H. Johansen, CEO of eSmart Systems.

Project Manager in eSmart Systems, Kari Amelie Fiva celebrates the launch of the Smartliv app.

The app is available for both IOS and Android platforms. The app is free and can be downloaded now. You must be current customer of Ringeriks-Kraft to use it.


Written by
Henriette Næss Forsetlund

Marketing and Communications Manager, eSmart Systems 

Webinar sammen med Microsoft

Fredag 9.januar holdt eSmart Systems og Microsoft et webinar om Big Data-løsninger og skyteknologi.

Big Data blir gjerne omtalt som et buzzword, men faktum er at mange bransjer og virksomheter vil bli påvirket av de teknologiske endringene som kommer. Energibransjen er blant dem. I fremtiden vil vi hente data fra mange ulike kilder, blant annet smarte strømmålere, sensorer, smarte komponenter i nettet, og eksterne kilder som sosiale nettverk. Utviklingen innen smarte nett skyter fart både i Norge og internasjonalt, og mengden data som må håndteres økes hurtig. De nærmeste årene skal rundt 2,5 millioner strømmålere i Norge byttes ut med smarte målere hvor forbruket blir meldt inn automatisk. I Europa skal 80% av el-målerne i alle EU-land være byttet ut med smarte målere innen 2020. I tillegg er smarte boliger og bygg på full fart inn i energisystemet, der både energiforbruk, lagring og lokal produksjon trenger nye systemer for styring og informasjon.

Big Data benyttes for å kunne ta bedre beslutninger basert på informasjon man har i egne systemer. Stadig flere står overfor en uhåndterlig mengde data som øker i voldsom takt. Overgangen til smarte målere vil gi en rekke fordeler for energi- og nettselskap, men for å dra fordeler av dette er det nødvendig med nye IT-systemer som er i stand til å bearbeide betydelig større datamengder og gjennomføre tilhørende analyser i sann tid.

Det finnes nå systemer som benytter seg av det siste innenfor teknologi for datahåndtering. Med skytjenesten Microsoft Azure som grunnmur har eSmart Systems designet et system som håndterer mengden og variasjonen av data, og oversetter den til innsikt. Ved bruk av skyteknologi har de klart å kombinere ulike lagringsmetodikker for å håndtere store datamengder som både kan være strukturerte og ustrukturerte, sette opp prediksjoner og avanserte maskinlæringsalgoritmer. Big Data-løsninger kombinert med skyteknologi gir brukerne muligheten til å få oversikt over dataene som produseres i nettet. Dette gir en stor merverdi i en bransje som er vant med mange silobaserte systemer som ikke snakker sammen, og ofte har begrensede analyse- og visualiseringsmuligheter.

Big Data – slett ikke et buzzword.

eSmart Systems at Microsoft Campus

Happy developer in Redmond

In September Erik Aasberg and Sigurd Seteklev visited Microsofts Redmond Campus as part of the Rapid Deployment Program (RDP) for Azure’s new cloud based data platform. All members of RDP are forward thinking companies with the ability and desire to use Microsofts latest technology.

eSmart’s main focus in the RDP was the Azure Machine Learning platform. The platform is at the heart of eSmart’s machine learning initiative, where we in addition are collaboration with Ostfold University College, and Microsoft’s Data Science team and are sponsoring a ph.d. student.

Big Data is not only about handling vast amounts of data, but in order to be able to harvest value from the data one has to exploit advanced algorithms. Machine learning is a term used for algorithms that “learn” from data, and ranges from simple algorithms fitting a line through points in a graph, to advanced artificial intelligence. The combination of big data and machine learning is therefore very powerful, and the leading companies such as Google, Facebook, Microsoft and IBM are competing to acquire the best Machine learning talent in the world (see e.g. this article about Googles purchase of DeepMind)

At eSmart the main usage of machine learning today, is prediction. In order to operate the Smart Grid efficiently, predictions are used to identify problems before they happen. One of the key advantages of eSmart’s platform is that we can handle all data from sensors, smart meters, social media, weather services and many more, and be able to quickly analyse and discover possible problems in the future.

During our stay at Microsoft’s Redmond campus, we were able to meet Data Scientists, Senior VP’s and Engineers from Microsoft, and network with fellow RDP-companies. Not only did we learn a lot about Microsoft’s vision for Azure Machine Learning, but we got thorough insight into how to best use the newest big data technologies in Microsoft Azure environment. A key advantage with the program is that we connect directly with the key people at Microsoft HQ, and can therefore harvest their knowledge as we develop the next generation of IT-systems.

Microsoft and eSmart is currently joining forces to ensure that eSmart has the most advanced predictions algorithms in the industry. We are very excited about this opportunity, and look forward seeing the results in action at a utility near you.

The Nexus of Forces

After working with relational and transactional systems for nearly two decades, the limitations of traditional technology became more and more obvious for us. It became very clear that this is not the future. A system built on such principles cannot handle the constant increase of data, both in volume, velocity and variety. We were already struggling with performance in our exchange settlement systems and trading and risk management systems. Therefore, when the chance to start over came along, using all kinds of technology now available, it was not a doubt in our minds that this is what we must do.
Gartner has defined the change now happening as “The Nexus of Forces”. Four trends, The Nexus of Forceseach one of them innovative and disruptive on their own, are now happening at once: Social, Mobile, Cloud and Information. Together they are revolutionizing business and society, disrupting old business models and creating new leaders. The Nexus is the basis of the technology platform of the future.

The Nexus of Forces
The Nexus of Forces
Source: Gartner, 2013

For an energy company, social media might not be obvious. However, it is easy to picture customer portals that incorporate social media. In addition, what if utilities could tap into social media to get near real-time information about what is going on in their area? Information that is not possible to reach through press or other official channels. Our system has a wide spread of social media

As everyone becomes constantly online, new possibilities arises. Wherever you are, you want to be able to check the status of your energy trading portfolio, whether your grid is operating as it should, and if there is a problem you should know. Our system provides client agnostic services to support all kind of client devices and receiving systems.

The cost of having on premise data centers are high and it is expensive to meet the availability, security and scalability of today’s cloud services. Our system is native cloud based.

A huge amount of data is collected daily, but the full potential of all this data is rarely reached. With new technologies like noSQL databases and analytic platforms like Hadoop, the possibility to reach the full potential is closer than ever. With the guidance of industry leaders we have established a data platform that lets you handle huge amounts of data in near real-time and do deep analytics and predictions to unlock hidden information and take right decisions.

By utilizing the Nexus of Forces, we are building the next generation IT systems for tomorrow’s energy companies.