Page 30 - Biosystems Engineering

Micr oarray Data Analysis Using Machine Learning Methods 11

such as “and” in the following fuzzy rule: “if gene x is up-regulated and
1
gene x is down-regulated, then the probability of disease y is high.”
2
Fuzzy operators include intersection, union, and complement. The fuzzy
intersection operation is mathematically equivalent to the “and,” the union
to “or,” and complement to “not” operation, respectively. The structure of
the fuzzy rules could cause an exponential growth in the number of rules
as the number of inputs increases, resulting in unwieldy rule bases. Thus,
it is important to select an optimal set of fuzzy rules.
In SVM, parameters to select include cost of constrain violation,
kernels (e.g., linear, polynomial, radial basis) and parameters related
to kernels. Key selections in implementing genetic algorithms include
the fitness function, the genetic representation or the chromosome
coding scheme, and the genetic operators such as mutation and cross-
over. Similarly, parameters of PSO include number of particles, number
of iterations, values for the stochastic component of the PSO algo-
rithm, and measure of fitness.
Learning from data is the predominant feature of machine learning
methods. Here, we define learning as the process of finding the free
parameters of a machine learning model. Depending on the environ-
ment in which the model operates, two types of learning paradigms are
commonly used, supervised and unsupervised. Supervised learning
uses an external teacher to produce a desired output; the model learns
from training examples. Conversely, unsupervised learning does not
involve an external teacher; the network discovers collective properties
of the inputs by self-organization. Also, two types of learning styles have
been applied in neural network training. The first approach, referred to
as pattern learning, where the weights of the network are adapted imme-
diately after each pattern is fed in. The other approach, known as batch
learning, takes the entire training dataset and updates the network
parameters after the entire batch of data has been processed.
An example of a supervised learning paradigm is the back-
propagation algorithm (Rumelhart and McClelland 1986), which is
commonly used to train MLP networks. The back-propagation algorithm
consists of two passes, forward and backward. During the forward
pass, input signals are applied to the network and their effect propa-
gates through the different layers and generates the network outputs at
the output layer. Note that the synaptic weights of the network are all
kept fixed during the forward pass. During the backward pass, synap-
tic weights are adjusted in accordance with the error correction rule.
The network outputs are subtracted from the desired outputs to pro-
duce error signals. The error signals are propagated backward through
the network against the direction of the synaptic connections to deter-
mine how the synaptic weights should be adjusted in order to decrease
a predefined error function. A typical error function is the sum of
square of the error signals between the network outputs and the desired
outputs. Thus, during the backward pass, the back-propagation algo-
rithm alters the synaptic weights in the direction of the steepest