Page 350 - From Smart Grid to Internet of Energy

314 From smart grid to internet of energy

infrastructures. It is also related with operation and trading requirements of
companies, enterprises, and consumer interactions. The source system-based
data are generated as raw-data and it is required to handle these data stacks with
extraction. Most of the challenges met in big data processing stages start with
extraction, transformation, and loading processes. Once the data is extracted, it
is filtered and normalized in transformation step, and then it is loaded to system
for processing [8–10].
The big data analytics stages are listed as acquisition, extracting/cleaning,
integration/aggregation, and interpretation. The big data generation sources
can be listed as enterprise level, IoT, internet data, biomedical sources, and
other generation sources as communication, experimental fields, and multime-
dia fields. The reliable and efficient result generation is depended to cleaning of
big data. Another important issue is synchronizing the data sources and big data
processing platforms with internal organization [7].

8.2.2 Data acquisition and storage
The second stage of big data infrastructure is big data acquisition and storage
applications which include data capture, transmission, and pre-processing.
When the data is acquired from its source, it is required to be transmitted to data
storage centers to ensure operation of analytical methods. The acquired datasets
may sometimes include unnecessary and unrelated data, and this causes incre-
ments on storage volumes. Therefore, data compressing methods are convenient
to be used for decreasing database sizes. On the other hand, pre-processing
methods are used to improve data storage efficiency. The storage challenges
require innovative solution against volume, velocity, and variety of acquired
big data stacks. The conventional hard disks are not proper solution for these
requirements. Thus, cloud computing and cloud storage opportunities are robust
solutions to meet requirements of large amount of data systems.
The data transmission can be progressed in various stages of big data
processing as: (i) transmission of acquired data to storage, (ii) integration of
data acquired from multiple sources, (iii) management of integrated data,
and (iv) transferring the stored data from storage to analysis server. The main
data acquisition methods are defined as log files, sensors, network data. The log
files are used to store regularly generated data, and they are integrated with all
digital devices. The data acquisition is performed by using almost all digital
devices such as internet servers, log files, sensors, and network devices. Sensors
are widely preferred in big data transmission since they convert the physical
amounts to readable and storage data. The WSNs provide widespread use of
applications and are convenient devices for data transmission by using their
multi-hop infrastructures. The raw data acquisition is followed by data trans-
mission process where transferred and stored data are prepared for processing
procedures [2, 8]. Machine learning, clustering and some other artificial intel-
ligence methods are widely used for data analysis and processing procedures of
big data stacks.