Architecture search through epigenetic evolution

We upgrade the Cortex neuroevolution framework with a genome representation suitable for evolving deep layered convolutional models. We also propose a convolutional layer layout which allows kernels of different shapes and sizes to coexist within the same layer, and present an argument as to why this may be beneficial. The proposed layout enables the size and shape of individual kernels within a convolutional layer to be evolved with a corresponding new mutation operator. The hybrid optimisation strategy based on evolving a population of networks, with structural changes implemented through epigenetic evolution and weight update implemented through backpropagation. Classification benchmarks demonstrate that the crossover operator is sufficiently robust to produce increasingly performant offspring even when the parents are trained on only a small random subset of the training dataset in each epoch, thus providing direct confirmation that learned features and behaviour can be successfully transferred from parent networks to offspring in the next generation. Importantly, this effect is a lot more pronounced when speciation is enabled in the population.