Deep learning for complex networks

Image recognition, speech processing, medical diagnostics… there’s hardly an area where machine learning hasn’t made a big impact. Yet Elvin Isufi adds an extra dimension to machine learning, enabling its efficient application to complex networks such as recommender systems, social networks, and more down-to-earth water networks.

We all know that machine learning is a very hot topic, and that it requires lots of training data to be successful. A lesser-known fact outside of computer science is that it typically requires data to be represented in a grid-like space. In speech, for example, air pressure varies over one-dimensional time. In medical diagnostics, image values vary on a two- (or three-) dimensional grid. Elvin Isufi (tenure track assistant professor in the Multimedia Computing Group at TU Delft) explains: “Standard deep learning scans these grid-like data and finds the pattern that represents a word, a cat or a tumour. By contrast, networks such as the brain, social media or a water network are very complicated. They have an abnormal rather than a regular structure, and we can seek to extend deep learning to networks by capturing their geometry.”

My research looks at how machine learning models leverage the structure of complex networks

Connectivity is key

In his research, Elvin delves into the fundamental computer science aspects of this so-called geometric deep learning. He is, however, also inspired by applied challenges in different domains. Take a water and sewage network, for example. This consists of nodes that distribute the water, pipes that transport it, and a limited number of sensors to measure the water level. Geometric deep learning will allow monitoring of the entire network. This could be used to identify anomalies such as leaks and cyber-attacks, or to make predictions regarding future maintenance and demand. Elvin: “Providing the neural network with training data and then real-time data based only on these sensors will not get

you very far. You must also give it the map of all connections. My research asks how the machine learning model can leverage the structure I have defined – this connectivity – to solve the task. And, if it is not leveraging it, how can we improve things?”

reading time: min

Pieke Hoekstra

Into the black box

Elvin says that the problem with machine learning is that it is typically a black box, and we don’t know how it operates: “I want to pry this box open, using some of the techniques I know from my former background in telecommunications engineering and network signal processing, as well as state-of-the-art machine learning techniques.” This is the same curiosity Elvin revealed as a child when opening toys to see how electronic circuits made them move, and it’s what made him pursue a career in electrical engineering. “I loved the fundamental rigour with which information can be transmitted live from the radio station to a car moving at 100km/h on the highway. I now try to understand AI with the same fundamental rigour.”

Cold start

Part of Elvin’s research in geometric deep learning focuses on so-called recommender systems, used by companies such as Netflix and bol.com to make billions of recommendations around movies or products. Elvin: “A key issue is the so-called cold start problem. A new item becomes available, so how does it fit into the network of existing items and their connections? How can we recommend that item since nobody has yet bought it or interacted with it? Geometric deep learning may be able to provide a better answer.” Fundamental developments in science and engineering are often applicable to more than one area, and this may be true for Elvin’s research. It won’t be surprising if his fundamental research subsequently helps a future power grid to deal with an additional solar park, or the sewer system with an additional spill-over.

Water management links my fundamental research with almost immediate impacts on society

Rather than one person taking sole responsibility for critical actions in networks, AI can help

Water management

Even though the field of geometric deep learning emerged only about five years ago, it has such enormous potential that people from other fields already know how to find Elvin. Together with the faculty of Civil Engineering and Geoscience, he directs the AIDRO-lab. “The lab focuses on AI research into resilient and sustainable water management. It offers a link between my fundamental research and how findings can have an almost immediate impact on society.” Alongside the urban water systems already mentioned, a second focus is flooding. If a region gets flooded, such as in Germany, Belgium or the South of the Netherlands last year, it is important to know where the water will go tomorrow, or even after an hour. It’s crucial to know if the remaining dikes will be able to handle the flooding. “Existing computational models are very accurate but may take months to run. Machine learning models are much faster, but generating all possible scenarios for training them takes much too long. My research question is: how can I create machine learning techniques that learn from maybe ten or twenty scenarios? This is essential for the learned model to be applicable in any other scenario.”

Geometric deep learning helps to unleash the full power of AI

Leading in AI

The continuing digitalisation of society means that there is an abundance of data, the feedstock of AI. “TU Delft is really taking a leading position in AI research. It is a vibrant environment where I can talk to other researchers, trying to gain inspiration from the methods they are applying and the issues they have. The power network, water networks, the brain – they are much too complex for any individual person to oversee. Rather than having a person take on sole responsibility for critical actions, we can provide help using AI. Geometric deep learning helps to unleash the full power of AI.”