Simulations and deep learning for the optimization of cortical visual prostheses

Thursday 27 June 2024, 12:30 pm
PhD candidate
J. de Ruyter van Steveninck
prof. dr. R.J.A. van Wezel, prof. dr. M.A.J. van Gerven
dr. U. Güçlü

Visual neuroproshetics aim to restore a rudimentary form of vision in the blind through electrical neurostimulation. To make optimal use of the limited visual information transfer, intelligent scene-processing software is essential. The studies in this thesis explore simulation paradigms with sighted subjects and computational models to evaluate and optimize the quality of prosthetic vision for everyday tasks like mobility and scene recognition. Some of the main findings include: i) Deep-learning-based contour processing might yield advantages for mobility, but a robust implementation is required, and the optimal implementation depends on the hardware (number of electrodes). ii) Proof-of-principle experiments suggest that it is possible to evaluate and optimize the information content of prosthetic vision through simulations and end-to-end optimization. It remains important to take into account the subjective interpretability. iii) Biologically plausible simulations may yield advantages for early-stage prototyping, while bridging the gap between research and clinical reality. iv) it is important to account for the effects of eye movements. Gaze-contingent image processing using an eye-tracker is suggested to yield functional benefits. Overall this thesis demonstrates the potential of digital simulations (VR, deep neural network simulations, simulated prosthetic vision) for the early-stage prototyping and optimization of visual neuroprosthetics. 

Jaap de Ruyter van Steveninck was born in Quezon City (Philippines) in 1992. After completing his Bachelor's degree in Medicine at Utrecht University, he took additional courses in Artificial Intelligence and obtained a Master's degree in Cognitive Neuroscience (cum laude) from Radboud University Nijmegen in 2019. As a software engineer, he now works on improving technology for recognising deep-fake images at the Duck Duck Goose tech startup in Delft.