# Research projects Fall 2022

**DEADLINE FOR SUBMISSION 6 July 2022 00:00**

## ------------------------------------

## ELLIS research project O:

Dynamic pruning: constraining the sparsity of neural activity

Brain circuits operate in sparse, asynchronous activity regimes [1-5]. While it is now well established that low-rate asynchronous irregular dynamics is a consequence of interacting recurrent excitation and inhibition [6, 7], the relevance and implications of sparse distributed representations in functional contexts is not well understood.

The extreme sparseness of cortical activity (often engaging less than 1% of active neurons at any given point in time) has critical implications for the design of neuromorphic systems, as it constitutes an operating feature that can dramatically reduce energy consumption and communication costs.

Additionally, sparse activity regimes appear to be actively maintained during cortical processing [8], suggesting the existence of adaptive mechanisms whose objective is the control of sparsity.

In this project, we will investigate sparse activations in binary recurrent networks of excitatory and inhibitory units [9], aiming to:

(a) Determine the constraints that allow the system to optimally solve the tasks: employing game-theoretical methods to prune neuronal activations, we will determine the sequences of sparse distributed representations (SDRs) that allow the system to optimally solve the task

(b) Relate energy consumption with sparse activations: considering the system's binary state variable as a proxy for energy and communication cost, we will investigate if and how the optimal sequence of SDRs can minimize computational cost and energy demands.

(c) Investigate how these constraints impact the system's controllability.

(d) Determine if the sequence of sparse activations (and/or the statistical features of population activity) can be replicated via biophysically-plausible plasticity mechanisms.

**References**:

[1] - Babadi, B., & Sompolinsky, H. (2014). Sparseness and Expansion in Sensory Representations. Neuron, 83(5), 1213–1226. https://doi.org/10.1016/j.neuron.2014.07.035

[2] - Foldiak, P., & Földiak, P. (2002). Sparse coding in the primate cortex. In M. A. Arbib (Ed.), The Handbook of Brain Theory and Neural Networks (p. 7). MIT Press.

[3] - Hahnloser, R. H. R., Kozhevnikov, A. A., & Fee, M. S. (2002). An ultra-sparse code underlies the generation of neural sequences in a songbird. Nature, 419(6902), 65–70. https://doi.org/10.1038/nature00974 1,* 1

[4] - Mazzoni, A., Brunel, N., Cavallari, S., Logothetis, N. K., & Panzeri, S. (2011). Cortical dynamics during naturalistic sensory stimulations: Experiments and models. Journal of Physiology Paris, 105(1–3), 2–15. https://doi.org/10.1016/j.jphysparis.2011.07.014

[5] - Olshausen, B. A., & Field, D. J. (2004). Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4), 481–487. https://doi.org/10.1016/j.conb.2004.07.007

[6] - Boustani, S. E., & Destexhe, A. (2009). A master equation formalism for macroscopic modeling of asynchronous irregular activity states. Neural Computation, 21(1), 46–100. https://doi.org/10.1162/neco.2009.02-08-710

[7] - Brunel, N. (2000). Dynamics of Sparsely Conntected Networks of Excitatory and nhibitory Spiking Neurons. Journal of Computational Neuroscience, 8, 183–208. https://doi.org/10.1023/A:1008925309027

[8] - Ahmad, S., & Hawkins, J. (2016). How do neurons operate on sparse distributed representations? A mathematical theory of sparsity, neurons and active dendrites. 1–23.

[9] - Lazar, A., Pipa, G., & Triesch, J. (2009). SORN: A self-organizing recurrent neural network. Frontiers in Computational Neuroscience, 3(OCT), 23. https://doi.org/10.3389/neuro.10.023.2009

[10] - Mengiste, S. A., Aertsen, A., & Kumar, A. (2015). Effect of edge pruning on structural controllability and observability of complex networks. Scientific Reports, 5(1), 18145. https://doi.org/10.1038/srep18145

**Supervisors:** Renato Duarte , Tal Kachman

1. Donders Center for Brain, Cognition & Behavior, Radboud University, Nijmegen, the Netherlands;

If you are interested in applying for this position please send your one page CV and motivation letter and a list of your obtained grades to: ellis-fellowships@ru.nl with the header containing the title of the project and your s-number.

------------------------------------

## ELLIS research project I:

Energy-efficient anomaly detection using spiking neural networks: Foundations of Natural and Stochastic Computing group

*Description*: Neuromorphic architectures such as Intel’s Loihi chip are oftentimes based on spiking neural networks, allowing for temporal coding of information and energy efficient computation. For many applications, ranging from scientific use cases such as the search for exoplanets and the detection of new particles in large colliders, as well as industrial applications, such as quality control systems, there is a lot of data to search for ‘needles in the hay stack’ – most of the data can be discarded and only a very small portion is really relevant. In this project we investigate energy-efficient ways of processing data in a spiking neural network, such that the network (like a Linux command shell) is as silent as possible in the case of expected or non-interesting inputs, and only uses energy (in the form of spikes) in case of interesting inputs that need further processing. That is, we want to develop a network structure such that the bulk of inputs will be processed with as little spikes as possible, while only a small subset of the inputs will lead to dense activation.

Requirements: an interest in information-theoretic / computational complexity aspects of information processing. Previous experience with SNNs, for example, by the Neuromorphic Computing course in the AI MSc programme, is desirable. The project can range from purely theoretic work to benchmarking in Intel’s Loihi architecture.

**Supervisor**: Dr. Johan Kwisthout.

If you are interested in applying for this position please send your one page CV and motivation letter and a list of your obtained grades to: ellis-fellowships@ru.nl with the header containing the title of the project and your s-number.

------------------------------------

## ELLIS research project II:

Decentralizing multi-agent systems with influence-abstraction

To optimize systems like a building’s HVAC, a swarm of robots, or an electrical grid, we often have to control a large number of components. Unfortunately, these problems become computationally intractable as the number of components grows. Traditionally, in artificially intelligence, such problems are treated as multi-agent systems, where each component is optimized by a separate agent. Unfortunately, these components might have some interactions or share resources, so the problem remains coupled and intractable.

A common way to handle this curse of dimensionality is to find a small abstract description of the underlying problem. We will investigate how to use machine learning techniques, such as recurrent neural networks, to create abstractions that allow us to optimize each component separately. An option is the use of influence abstraction [Oliehoek et al.,2022], where a subset of nodes is able to capture external influences to the local state space of a given agent. We pose the following concrete research questions for this project:

- Can we optimize the operations of a single component using influence nodes instead of the external features?
- Can we learn how these components behave, considering that they are also influenced by other agents that are not stationary?

This project involves a combination between machine learning and formal methods. We are interested in investigating

these questions using PRISM [Kwiatkowska et al., 2011], a state-of-the-art formal verification tool, and machine learning techniques to learn the dynamics of the influence nodes.

The student working on this project will, besides ELLIS, be embedded in daily activities of the research group. The project is part of ongoing research, and a master student would directly join the ongoing collaboration with Prof. David Parker from the University of Oxford. We conduct weekly meetings with the whole team, and facilitate an open and cooperative research culture, where also students are directly involved. There will be the possibility and funding to visit at least one workshop or conference on related topics to interact with the best researchers in the field. A research visit to Oxford is also possible.

**Supervisors**: Thiago D. Simão and Nils Jansen

*References:*

Kwiatkowska, G. Norman, and D. Parker. PRISM 4.0: Verification of Probabilistic Real-time Systems. In Proc.

23rd International Conference on Computer Aided Verification (CAV’11), volume 6806 of LNCS, pages 585–591. Springer, 2011

A. Oliehoek, E. Congeduti, A. Czechowski, J. He, A. Mey, R. A. N. Starre, and M. Suau. About ‘Influence’. Blog URL https://www.fransoliehoek.net/wp/2022/02/01/a-blog-about-influence/.

If you are interested in applying for this position please send your one page CV and motivation letter and a list of your obtained grades to: ellis-fellowships@ru.nl with the header containing the title of the project and your s-number.

----------------------------------

## ELLIS research project III:

Robust neural network controllers in uncertain environments

This project concerns the robustness of neural networks in complex environments under uncertainty. Think of a delivery drone navigating through a contested urban environment, as depicted in the two examples in Figure 1 (email to Elise for the figure) . An effective and safe controller for this drone needs to take into account, for instance, sensor imprecision, restricted views, unexpected wind gusts, or other air traffic participants.

In artificial intelligence, such decision-making problems under uncertainty [Kochenderfer, 2015] are commonly modeled by Markov decision processes (MDPs) or their various extensions. Reinforcement learning is a prominent machine learning technique to create a decision-making policy for such problems, with deep reinforcement learning employing neural networks as policy representations. Effective means to prove the robustness of such neural networks policies have been proposed, for instance, in [Carr et al., 2021]. Yet, these policies are usually trained for one specific environment and thus prone to overfitting. In the real world, problems are subject to slight changes or distributional shifts. Consequently,

the robustness of such neural network controllers remains an open challenge towards real-world applicability. We propose the following approach towards increased robustness of controllers. Instead of training the neural network based on a single model, we employ a so-called family of MDPs [Ceska et al., 2019]. In a nutshell, such a family

contains a potentially infinite number of environments that share certain similarities, such as joint feature spaces. Intuitively, the two drone scenarios depicted in Figure 1 can be modelled by two MDPs that belong to the same family. We follow the following training scheme depicted in Figure 2. (1) We train the neural network controller for a randomly chosen subset of MDPs from the family. (2) We formally verify the performance of the neural network controller

to assess its robustness regarding the whole family. We will use the Python-based interface of the model checker Storm [Dehnert et al., 2017] (www.stormchecker.org). (3) In case the performance is not satisfactory, we generate so-called diagnostic data to generate new training data. As a baseline, we will employ an actor-critic architecture [Konda and Tsitsiklis, 1999], but we will expect the student to provide a broad set of experiments that investigate the appropriate neural network architecture.

**Supervisors**: Nils Jansen and Sebastian Junges

*References:*

S. Carr, N. Jansen, and U. Topcu. Task-aware verifiable rnn-based policies for partially observable markov decision processes. J. Artif. Intell. Res., 72:819–847, 2021.

M. Ceska, N. Jansen, S. Junges, and J. Katoen. Shepherding hordes of markov chains. In TACAS (2), volume 11428 of Lecture Notes in Computer Science, pages 172–190. Springer, 2019.

C. Dehnert, S. Junges, J. Katoen, and M. Volk. A storm is coming: A modern probabilistic model checker. In CAV (2), volume 10427 of LNCS, pages 592–600. Springer, 2017.

M. J. Kochenderfer. Decision Making Under Uncertainty: Theory and Application. MIT press, 2015.

V. R. Konda and J. N. Tsitsiklis. Actor-critic algorithms. In NIPS, pages 1008–1014. The MIT Press, 1999.

If you are interested in applying for this position please send your one page CV and motivation letter and a list of your obtained grades to: ellis-fellowships@ru.nl with the header containing the title of the project and your s-number.

----------------------------------

## ELLIS research project V:

Connectome

The brain forms an intricate network of spatially segregated regions, known as the 'connectome', which is closely related to cognition, age, and general health. To understand the organizational principles behind the connectome, researchers formulate null models [1], and compare their predictions observed networks, to learn which models fit our observations best. In our lab, we are developing a general framework using Bayesian model comparison and sequential Monte Carlo to provide a principled way of quantifying evidence for each of the existing null models, and learn which models --- or combinations thereof --- provide accurate predictions of brain connectivity.

The proposed thesis project focuses on one model in particular: that of hyperbolic latent geometric embedding, in which network connections are assumed to depend on some unobserved distance (in hyperbolic space) between nodes [2]. This model has been shown to capture many relevant characteristics of real-world networks, but an adequate Bayesian implementation of this model is still lacking. In this project, you will define this model, implement it in our framework, and use it as part of our quest towards understanding the connectome.

**Supervisor**: Max Hinne

[1] Váša, F., & Mišić, B. (2022). Null models in network neuroscience. Nature Reviews Neuroscience.

[2] Aldecoa, R., Orsini, C. & Krioukov, D. Hyperbolic graph generator. Comput. Phys. Commun. 196, 492–496 (2015).

If you are interested in applying for this position please send your one page CV and motivation letter and a list of your obtained grades to: ellis-fellowships@ru.nl with the header containing the title of the project and your s-number.

----------------------------------

## ELLIS research project VI:

Fellowship project opportunity at Alliander

**Project description**

Alliander aims to play a decisive role in the energy transition to a more sustainable future. The company seeks new technologies and responds to new developments in the energy sector with the goal to use the grid more efficiently and boost productivity. It allows us to connect as many renewables, electric vehicles, heat pumps and new customers to the grid as possible.

At Alliander we have a low number of measurements in the lower part of the grid (e.g. at the street level and at secondary substations), therefore we use so called bottom-up models, where we use measurements at large customers or average profiles per customer to estimate the loads on secondary substations, see e.g. *ANDES: grid capacity planning using a bottom-up, profile-based load forecasting approach*, *van de Sande et al*[i]. In this way we create pseudo-measurements that at least give some insight in what is happening in the grid

For a certain class of customers we have more information to estimate their loads then we’re actually using. For these medium-sized customers we know per month their energy consumption (and/or production) and their maximum and minimum load during a 15-minute period during each month. The only information we are using so far is the total energy consumption (production) per year, we multiply this by an average profile per customer type. Due to the fact that we’re using average profiles this could lead to an underestimation of the real load.

It turns out that using both sources of information (total and maximum/minimum consumption/production) is not trivial. Nonlinear scaling methods are non-stable, while regression methods usually cannot deal with aggregated information over an interval such as the total or maximum load.

On idea to solve this problem is to use a combination of Gaussian Processes and Bayesian Networks (e.g. *Nuclear data evaluation with Bayesian networks*, *Schnabel et al, 2021* [ii]) estimate load profiles + uncertainty bands that fit both the sum and the maximum of consumed energy. But also other methods could be tested in order to solve the problem.

**What do we offer you?**

This is a challenging and highly varied internship in an organization that is at the center of the energy transition. Alliander is at the forefront of applying Data Science in a technical environment. Obviously, this includes a good internship allowance and we support you with all means to perform your work well. We have more than enough data available and ready to perform this assignment.

**About Alliander**

Alliander is a large Dutch distribution system operator (DSO) that ensures that millions of customers have access to electricity and gas every day for living, working, transport and recreation. We stand for an energy supply that gives everyone access to reliable, affordable and sustainable energy under the same conditions. Now and in the future. That is what we work on together every day. We offer our professionals an environment for innovative and smart ideas. An environment for your energy.

You will work at the IT Advanced Analytics department. Over 60 Data Scientists and Data Engineers work in this department to solve important issues for Alliander using the often large amounts of data. Challenges that we deal with are:

- Where is the greatest chance of a power failure?
- Which households will probably be placing solar panels on their roof in the coming years?
- Can we estimate the difficulty of a task for a contractor from a photo of the customer’s meter cabinet with image recognition?

Most of the work is performed in agile working scrum teams.

*Screening policy*

Alliander screens all applicants. Depending on the position, the screening consists of the following steps: checking references, checking the authenticity of identity papers and diplomas, an integrity check and requesting a certificate of conduct (VOG).

[i] http://dx.doi.org/10.1049/oap-cired.2017.1071

[ii] https://arxiv.org/abs/2110.10322

If you are interested in applying for this position please send your one page CV and motivation letter and a list of your obtained grades to: ellis-fellowships@ru.nl with the header containing the title of the project and your s-number.

----------------------------------

## ELLIS research project VII:

Fellowship project opportunity at Alliander

**Project description**

Alliander aims to play a decisive role in the energy transition to a more sustainable future. The company seeks new technologies and responds to new developments in the energy sector with the goal to use the grid more efficiently and boost productivity. It allows us to connect as many renewables, electric vehicles, heat pumps and new customers to the grid as possible.

Alliander has measurements at various locations in its electricity network, to get insights in the currents, voltages and active/reactive powers in the grid (https://en.wikipedia.org/wiki/AC_power). Most of these measurements only measure the absolute value of the current (I), because these measurements are cheaper and more reliable then measurements of the active and reactive powers (P & Q). The latter, however, provide more insight in the behavior of the grid and different loads, and are much more suitable to do e.g. load forecasting and State-Estimation. Is it possible to estimate the direction of the power flow or even the fraction between P and Q (called the power factor: https://en.wikipedia.org/wiki/Power_factor) from absolute current measurements?

There is data from locations/substations where we do measure all the different quantities, which can be used to train a machine learning model, but there are also physical laws that constraint the solution space. For example there is Kirchhoff's current law that states that all incoming and outgoing (complex) currents or powers should add up to zero. This problem therefore lends itself very well to a Bayesian approach where physical knowledge and knowledge from data is combined.

Because the relations between different nodes in the network don’t need to be taken into account, this problem seems to be reasonable to solve within the period of one master thesis project.