Modeling Visual Hallucination: A Generative Adversarial Network Framework

Visual hallucination refers to the perception of recognizable things that are not present. These phenomena are commonly linked to a range of neurological/psychiatric disorders. Despite ongoing research, the mechanisms through which the visual system generates hallucinations from real-world environments are still not well understood. Abnormal interactions between different regions of the brain responsible for perception are known to contribute to the occurrence of visual hallucinations. In this study, we propose and extend a generative neural network-based framework to address challenges within the visual system, aiming to create goal-driven models inspired by neurobiological mechanisms of visual hallucinations. We focus on the adversarial interactions between the visual system and the frontal lobe regions, proposing the Hallu-GAN model to suggest how these interactions can give rise to visual hallucinations. The architecture of the Hallu-GAN model is based on generative adversarial networks. Our simulation results indicate that disturbances in the ventral stream can lead to visual hallucinations. To further analyze the impact of other brain regions on the visual system, we extend the Hallu-GAN model by adding EEG data from individuals. This extended model, referred to as Hallu-GAN+, enables the examination of both hallucinating and non-hallucinating states. By training the Hallu-GAN+ model with EEG data from an individual with Charles Bonnet syndrome, we demonstrated its utility in analyzing the behavior of those experiencing hallucinations. Our simulation results confirmed the capability of the proposed model in resembling the visual system in both healthy and hallucinating states.