Artificial Intelligence and Machine Learning

Advanced Machine Learning and AI algorithms are being used to solve problems from a very diverse domain. At the moment we mostly used supervised (Linear and Logistic Regression, Decision Tree and Random Forest, Support Vector Machine, Deep Neural Networks, Convlutional Neural Network, Recurrent Neural Network, Generative Adversarial Networks) and unsupervised (K-neares neighbor, Self Organized Maps, Autoencoders) learning algorithms. Some of the application areas have been wind power forecasting, discovering new chemical compunds with desired properties, predicting pedestrian and bikers behavior in cities and identifying favourable locations for fishing. We have also used anomaly detection algorithms to identify malfunctioning sensors in a system. Recently I conducted a course on "Practical Machine Learning" at the Geilo Winter School. It included interactive python sessions. The school was attended by 80 researchers from industry and academia. I am also designing a 15 minutes weekly online course for SINTEF employees.

Aviation

In cooperation with the Norwegian Meteorological Institute we have developed a microscale terrain-induced wind and turbulence alert system. The system is operation for 20 Norwegian airports and provide realtime forecast of wind and turbulence. We are also involved in siting of airports, runways and buildings close to the airports. During our involvement in the SESAR project, we also developed a computationally efficient aircraft wake-vortex model which has all the physics required for the initiation, transport and diffusion of the vortices.

Building Physics

My research in this topic dates back to the time when I was working on my PhD. During that time I conceived and implemented the concept of equivalent geometry whihc enables accounting for the complexity in urban geometry in a mesocale model for atmospheric flows. In particular I developed models for simulating wind, solar radiation and occupants behavior to model urban climate more accurately. Although most of the work was at an urban scale it is trivial to apply the concepts and tools to individual buildings.

Computational Fluid Dynamics

Over the years my group has developed a set of tools which enable us to simulation highly complicated flows using the national supercomputing infrastructure. In particular, we can simulate incompressible turbulent and multiphase flows dominated by large variety of spatio-temporal scales. We develop solvers from scratch and at the same time we build upon the opensource OpenFoam solvers. The latter enables us to solve industrial scale problems. So far we have applied CFD in the field of meteorology, chemical reactor modeling, wind energy.

Creative Art

I n my personal capacity I have always been using painting, photographs and other such illustration to get a better insight ino the working of "black box" machine learning algorithms. I have also been effectively using art to explain complex topics to general public.

Hybrid Analysis and Modeling

We define "Hybrid Analysis and Modeling" as a modeling technique that combines the interpretability, robust foundation and understanding of a physics-based approach with the accuracy, efficiency, and automatic pattern-identification capabilities of advanced data-driven machine learning and artificial intelligence algorithms. We have just acquired a project to test out some ideas which are quite promising.

Wind Energy

We have been involved in several wind energy prjects and we practically do almost everything that can be numericaly simulated. Using our tools and expertize we can simulate anything starting from the flow aorund a 2D blade to the windfarm-windfarm interactions. Currently, our models are in operational use for forecasting wind energy from an onshore wind farm.

Pre-project and masters project topics for 2020-21

Some things worth knowing:

1. Tht titles and the project description can be adapted to the interest and skills of the students
2. Although some project might sound very challenging, there might be some ground work already in place so just get in touch to know more
3. I prefer that the students use skype to interact with me as and when required, overeaf to share their writing work (reports and articles) and github for sharing the codes
4. For physical meetings just drop a message on skype to find my availability and drop by my office

Sample of pre-projects from 2019-20

1. Meyer, E.: Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning
2. Herman Stavelin: Marine life through You Only Look Once's perspective
3. Simen Havenstrom: Proportional integral derivative controller assisted reinforcement learning for path following by autonomous underwater vehicles
4. Duy Tan Tran: GANs enabled super-resolution reconstruction of wind field
5. Camilla Sterud: Deep Reinforcement Learning for Path Following and Collision Avoidance, 5th Norwegian Big Data Symposium
6. Haakon Robinson: Dissecting deep neural networks

---

Project 1: Ruteplanlegging for autonome farkoster
Background: The ability to route planning is central to autonomous vehicles. Based on a world model, the vessel calculates optimal routes depending on the situation and mission. Routes must be planned both over greater distances, based on map data and other advance information, and in the immediate area based on data from the vehicle sensors combined with the map data. We work with route planning for both ground vehicles and boats. Some issues are relevant for both types of craft, while other issues are more specific.

In order to do multi-level route planning, hierarchical route planning is required to reuse results at a higher level when planning more detailed routes. In addition to avoiding obstacles, it is important to map the terrain so that the vehicle can choose routes through low-risk areas. This work includes automatic analysis of data on the surroundings, such as weather and terrain, to determine how suitable different areas are for different tasks.

For some types of missions it is necessary to move in hiding, other times you will move quickly or observe certain areas. Different terrain characteristics are preferred for the different tasks, and we want to develop methods for calculating this automatically. There is also a need to further develop existing optimization methods to search for routes.

There is room for several tasks within this theme. All the thesis is intended as project assignments of 15 credits, and is suitable for extension to a master's thesis. We note that FFI requires the grade B or better on average to write a master's thesis for us.

Problem Statement: Dynamic environment: The environment in which an autonomous vehicle moves will often be dynamic, so the cost of a move will depend on when the move occurs. Examples of this may be dynamic obstacles, phenomena such as queuing, need to stay close to another unit along the way, or varying weather conditions. In many cases, it is possible to estimate future changes, so that this can be taken into account in route planning. This is a challenge for traditional methods such as Dijkstra, and it is no longer certain that existing methods provide optimal routes. The task is intended to be based on graph-based route planning algorithms, such as Dijkstra, A * / D * or other. However, the student is free to come up with their own suggestions for solutions.

Contact at NTNU: Adil Rasheed (adil.rasheed@ntnu.no)
Contacts at FFI: Martin Syre Wiig (martin-syre.wiig@ffi.no)

References

1. Meyer E., Robinson, H., Rasheed, A., and San, O., Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning, IEEE Access, 8, 41466-41481, 2020
2. Meyer E, Heibergh A, Rasheed A and San O, COREG-compliant path following and collision avoidance with moving obstacles in the Trondheim fjord using Deep Reinforcement Learning, In preparation
3. Proportional integral derivative controller assisted reinforcement learning for path following by autonomous underwater vehicles

---

Project 2: Medical Diagnosis
How can we give artificial intelligence systems used for medical diagnosis the ability to say... "I have never seen this before. Don't trust me on this one." ?

Problem Description: Medical diagnosis is difficult because not every case is a typical case. Sometimes patients have signs and symptoms that are unusual. In extreme cases, due to genetic mutation, the patient is the only person in the world with the disease and its associated symptoms.

When a doctor comes across an unusual case she may say "I've never seen this before." and proceed carefully. An AI system that diagnoses patients should also have the ability to say... "I have never seen this before. Don't trust me on this one." Assuming the AI system is based on supervised learning, how can we add capability to the AI system so that it tells users to what extent its training dataset is representative of the current case? Maybe this requires a secondary AI algorithm built with unsupervised learning?

We can attempt to tackle this problem by looking at a system which performs automatic analysis of lung sounds. Such a system could perform better in the real-world if it detected input data which will perform poorly due to either the input data not being representative of the dataset or the data is of poor quality. The excluded data can then be assessed through traditional means such as review by a doctor.

Company dedeX is a med tech startup. dedeX is looking to solve the problem: how do we get the data needed to build AI algorithms that can empower medical history taking and physical examination? dedeX understands both real-world healthcare and AI and can give the student feedback from this unique perspective.

Data: Two separate datasets of annotated lung sounds with different bias and variance should be used. One is the ‘development’ dataset and the other is a ‘real-world’ dataset. The development dataset will be used to create a ‘primary’ supervised machine learning algorithm and a ‘secondary’ unsupervised machine learning algorithm.

The primary algorithm will be used to classify lung sounds in both datasets. This algorithm can be based on existing work by other researchers.

The secondary algorithm will identify data from the real-world dataset that will perform poorly when used in the primary algorithm. In simplified terms it is an unsupervised learning algorithm which looks for the data that is the least similar to data in the development dataset. When this data is excluded from the real-world dataset then the primary algorithm should achieve higher accuracy on the real-world dataset.

We have identified four sources of annotated lung sound data. The student can pick two of these for this project or alternatively find other data sources if more convenient.

1. Int. Conf. on Biomedical Health Informatics - ICBHI 2017 http://www.auditory.org/mhonarc/2018/msg00007.html
   https://www.kaggle.com/vbookshelf/respiratory-sound-database
2. Department of Community Medicine at UiT – The Arctic University of Norway
   http://bdps.cs.uit.no
   "In the Tromsø Lung Sounds project we are building a database with more than 36.000 lung sound recordings. The recordings are done as part of Tromsøundersøkelsen 7, which is an Epidemiological study that was started in 1974. The database will be used to provide educational and analysis services for lung sounds. Our contributions are methods for automated classification and similarity search for the sounds. This project is done in collaboration with Hasse Melbye at the Department of Community Medicine, University of Tromsø. The results from this project are further developed by our Medsensio AS startup.”
3. Thinklabs
   https://www.thinklabs.com/lung-sounds
4. Interactive Systems for Healthcare
   https://is4health.com

Task

1. Investigate the possibility of applying existing state-of-the-art deep learning methods to solve the above tasks. As part of this, the student is expected to perform a state-of-the-art literature review and implement the most relevant method(s) that can solve the problem.
2. Make the chosen method time effective, feasible and available for researchers or developers of AI technologies for diagnosis.

Thesis Information
Timeframe: 6 or 12 months

Contact at NTNU: Adil Rasheed (adil.rasheed@ntnu.no)
Contacts at dedeX: Jon Bekker(jon.floystad@dedex.ai)

---

Project 3: Alternate representation of Deep Neural Networks (Affiliated to the Explainable AI project EXAIGON)
Neural networks have been applied to a vast array of problems, many of them safety critical. Despite their usefulness, there are many unanswered questions regarding their robustness, stability, and overall safety. This is problematic, as they are increasingly being applied to robotics and control problems in research.

A surprising fact about neural networks is that if they only use piecewise linear activation functions (for example, ReLU), the network as a whole will be piecewise linear There is a vast literature on this class of functions, which can be leveraged in the study of neural networks. A central challenge with this approach is that the functions are extremely complicated, with huge numbers of linear regions. Despite this, these ideas have been used to:

1. Verify stability/robustness properties of networks
2. Locally “patch” neural networks using MILP optimisation
3. Visualise the internal structure of neural networks during and after training

This is a cutting-edge field within deep learning, with many possible applications and future research directions. The following research projects are available as thesis topics:

1. Model predictive control (MPC) is a flexible control framework that provides stability guarantees while respecting system constraints. The work will assess the feasibility of applying explicit MPC methods to piecewise linear neural networks.
2. Neural network patching is a relatively unexplored method to locally adjust the output of a piecewise linear neural network using optimisation. The work will involve investigate how patching can be used in the process of machine learning controller design and/or system identification, as well as how to use piecewise linear methods to identify and diagnose issues in controller output. There is potential for collaboration with other students here.
3. Computing the piecewise linear regions of a neural network is both computationally and memory intensive due to the large number of regions. However, the computations are very parallelisable, and neural networks contain a lot of structure that can be utilised. This project will involve developing efficient parallel algortihms to solve this problem (possibly using GPU programming) and compact data structures to represent the network and speed up later queries on the model.

Desired background:

1. Strong programming skills in Python and MATLAB (C and C++ are also an advantage)
2. Basic knowledge of machine learning, in particular neural networks
3. Good grasp on optimisation

Relevant bibliography:

1. Robinson, Haakon; Rasheed, Adil; San, Omer. (2019) Dissecting Deep Neural Networks. arXiv.org

Contact at NTNU: Adil Rasheed (adil.rasheed@ntnu.no)
Co-supervisor: Haakon Robinson (haakon.robinson@ntnu.no)

---

Project 4: Machine Learning Controllers

Problem Description: There exists a multitude of traditional controllers to determine the control inputs (actuators) to achieve the desired behavior of a dynamic system, e.g. feedback linearizing, backstepping, sliding mode, adaptive control etc. However, these controllers are often based on a dynamic model of the system, numeric values for system parameters and several assumptions which are not always accurate. By applying machine learning, it is possible to develop controllers that are not based on a model or assumptions. Specifically, Machine Learning Control (MLC) aims to learn an effective control law b = K(s) that maps the system output (sensors s) to the system input (actuators b) and can solve problems involving complex control tasks where it may be difficult or impossible to model the system and develop a useful control law. The pre-project phase of the project will involve:

1. Set up a simulation environment for some system (drones, ships, AUVs, etc.)
2. Implement some of the traditional controllers

The pre-project will lead to a masters project where the major steps will be as follows:

1. Set up a simulation environment for some system (drones, ships, AUVs, etc.)
2. Implement some of the traditional controllers
3. Use some MLC-approaches (regression problem, Actor-Critic reinforcement learning) to develop new controllers
4. Test and compare performance in case of
   1. Perfect model knowledge
   2. Errors in model parameters
   3. Noisy measurements
   4. Unmodeled nonlinearities
5. Investigate stability properties/robustness/convergence of ML-controllers

Supervisor: Adil Rasheed (adil.rasheed@ntnu.no), NTNU
Co-supervisor: Haakon Robinson, NTNU (haakon.robinson@ntnu.no)

Reference

1. Meyer, E., Robinson, H., Rasheed, A., and San, O., Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning, IEEE Access, 8, 41466-41481, 2020.
2. Sterud, Camilla; Robinson, Haakon; Rasheed, Adil; Moe, Signe; San, Omer. Deep Reinforcement Learning for Path Following and Collision Avoidance. 5th Norwegian Big Data Symposium; 2019-11-13

---

Project 5: Evaluating the hydrostatic characteristics of a ferry using high fidelity simulations and dimensionality reduction algorithms


Project Description:

Airflow around a typical ferry is dominated by a large variety of spatio-temporal scales and complex flow structures. State-of-the art Computational Fluid Dynamics simulations have been used to simulate the drag, yaw and pitch characteristics of such ferries. A solid body like a ferry has intricate designs all of which are not relevent for evaluating its hydrodynamic characteristics. To this end, in this project the student will evaluate and quantify the impact of geometric simplifications of a ferry on its hydrodynamic characterists. The flow simulator based on OpenFoam will be utilized to generate the 3D wind and turbulence field around a moving ferry. The results will be postprocessed using advaced dimensionality reduction algorithms like Proper Orthogonal Decomposition and AutoEncoders. The project will include the following tasks:

1. Developing a detailed water tight geometry of the ferry in STL format
Simplification / approximation of the geometry
2. Generation of computational mesh around the solid geometry
3. Conducting a set of high fidelity simulations for different wind conditions
4. Go to step 2 and try a different simplification strategy
5. Analyze the data using POD and AE

Supervisor: Adil Rasheed, NTNU
Co-supervisor: Mandar Tabib, SINTEF Digital


---

Project 6: Explaining turbulence using Deep Learning (Affiliated to the Explainable AI project EXAIGON)
Turbulent flows are dominated by a large variety of spatio-temporal scales. So far tools like Fast Fourier Transform and Wavelet Transform have proven to be the workhorse for analyzing these scales. With the recent revolution in Deep Learning and more particularly in Convolutional Neural Network (CNN) and Auto Encoders (AE) we are witnessing huge improvements in classification and data compression tasks. It is foreseen that the automatic pattern identification properties of CNN and important feature identification properties of AE can be utilized to improve our understanding of physical phenomena governed by multiple spatio-temporal scales like turbulence. In this project the focus will be to better describe flow structures observed in turbulent flows. The pre-project will involve the following tasks:

1. Develop a good understanding of CNN and AE
2. An intercomparion of Auto Encoders and Principal Component Analysis
3. Evaluation of AE as an effective feature detector

The pre-project will lead to a Masters project which will have the following steps:

1. Data generation: Numerical simulation of turbulent flows will be conducted using softwares developed at SINTEF Digital
2. Development and training of CNN and AE on turbulent flows simulation data
3. Using the trained algorithms to analyze the flow structures in turbulent flows

Supervisor: Adil Rasheed, NTNU
Co-supervisor: Mandar Tabib and Kjetil Andre Johannessen, SINTEF Digital
Learning outcome: Computational Fluid Dynamics, Turbulence Modeling, Deep Learning, Dimensionality Reduction

---

Project 7: Hybrid Analysis and Modeling as an enabler for Big Data Cybernetics


Project Description: Most of the modeling approaches lie in either of the two categories:

Physics based modelling:strong> So far, the engineering community has been driven mostly by a physics-based modeling approach. This approach consists of observing a physical phenomenon of interest, developing a partial understanding of it, putting the understanding in the form of mathematical equations and ultimately solving them. Due to partial understanding and numerous assumptions along the line from observation of a phenomena to solution of the equations, one ends up ignoring a big chunk of the physics. High-performing simulators capable of handling a billion degrees of freedom are opening new vistas in simulation-based science and engineering and combined with multiscale modeling techniques have improved significantly the predictive capabilities. Thus, the dream of establishing numerical laboratories (e.g. numerical wind tunnels) can now be realized for many interesting real world systems. However, despite their immense success, the use of high-fidelity simulator has so far been limited to the design phase. Some of the advantages of physics based models are that they have robust foundation from first principle, are trustworthy, are generalizable to new problems and robust theory exists for uncertainty and error analysis of the model results. However, these models are generally computationally expensive and not adapting to new scenarios automatically.

Data-driven modeling: While physics based models are the workhorse at the design phase, with the abundant supply of big data, opensource cutting edge and easy-to-use libraries (tensorflow, torch, openAI), cheap computational infrastructure (CPU, GPU and TPU) and high quality, readily available training resources, data-driven modeling is becoming very popular. Compared to the physics based modeling approach, this approach is based on the assumption that since data is a manifestation of both known and unknown physics, by developing a data-driven model, one can account for the full physics. The data-driven approach in general and deep learning in particular has started achieving human level performance in several tasks which were until recently considered impossible for computers. Notable among these are image classification, creative art, anomaly detection. SOme of the advantages of these models are that they can learn in real time from the data, can be highly accurate and are generally very computationally efficient making them idle for real time control applications. However, owing to their blackbox nature, poor generalizability and no robust theory for uncertainty and error quantification, their acceptability in safety critical autonomous systems is limited. Thus for a dream model we have identified some characteristics which are as follows:

1. Computational efficiency
2. Intelligence to model the unknown
3. Evolution and self adaptation of models
4. Interpretability of models

The four desired model characteristics mentioned above can be addressed by developing a new breed of modelling approach that will combine the interpretability, robust foundation and understanding of a physics-based approach with the accuracy, efficiency, and automatic pattern-identification capabilities of advanced data-driven machine learning and artificial intelligence algorithms. We call this new approach as Hybrid Analysis and Modeling.

Supervisor: Adil Rasheed, NTNU
Co-supervisor: Damiano Varagnolo, Trond Kvamsdal, NTNU
Learning outcome: Hybrid Analysis and Modeling paradigm in modeling

Reference

1. Pawar, S., Ahmed, S. E. , San, O., and Rasheed, A. An evolve-then-correct reduced order model for hidden fluid dynamics, Mathematics, 8(4), 570, 2020.
2. Ahmed, S. E. , San, O., Rasheed, A., and Iliescu, T. A long short-term memory embedding for hybrid uplifted reduced order models, Physica D: Nonlinear Phenomena, 409, 132471, 2020.
3. Pawar, S., Ahmed, S. E. , San, O., and Rasheed, A. Data-driven recovery of hidden physics in reduced order modeling of fluid flows, Physics of Fluids, 32, 036602, 2020.
4. Rasheed, A., San, O., and Kvamsdal, T. Digital twin: values, challenges and enablers from a modeling perspective, IEEE Access, 8, 21980-22012, 2020.
5. Pawar, S. , San, O., Rasheed, A. and Vedula, P. A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence, Theoretical and Computational Fluid Dynamics, accepted, 2020.
6. Vaddireddy, H., Rasheed, A., Staples, A. E. and San, O. Feature engineering and symbolic regression methods for detecting hidden physics from sparse sensor observation data, Physics of Fluids, 32, 015113, 2020.
7. Ahmed, S. E., Rahman, S. M., San, O., Rasheed, A. and Navon, I. M. Memory embedded non-intrusive reduced order modeling of non-ergodic flows, Physics of Fluids, 31, 126602, 2019.
8. Rahman, S. M., Pawar, S., San, O., Rasheed, A. and Iliescu, T. Nonintrusive reduced order modeling framework for quasigeostrophic turbulence, Physical Review E, 100, 053306, 2019.
9. Pawar, S., Rahman, S. M., Vaddireddy, H. , San, O., Rasheed, A. and Vedula, P. A deep learning enabler for non-intrusive reduced order modeling of fluid flows, Physics of Fluids, 31, 085101, 2019.

---

Project 8: Digital Twin Laboratory


Project Description: Digital twin can be defined as a virtual representation of a physical asset enabled through data and simulators for real-time prediction, optimization, monitoring, controlling, and improved decision making. Recent advances in computational pipelines, multiphysics solvers, artificial intelligence, big data cybernetics, data processing and management tools bring the promise of digital twins and their impact on society closer to reality. Digital twinning is now an important and emerging trend in many applications. Also referred to as a computational megamodel, device shadow, mirrored system, avatar or a synchronized virtual prototype, there can be no doubt that a digital twin plays a transformative role not only in how we design and operate cyber-physical intelligent systems, but also in how we advance the modularity of multi-disciplinary systems to tackle fundamental barriers not addressed by the current, evolutionary modeling practices. In the projects related to DT lab, the students are expected to work towards developing a digital twin of a room. The digital twin should be indistinguishable from its physical counterpart. The projects will be broadly of two kinds: (1) experimental and hardware related and (2) algorithm development and software related for bringing physical realism in the DT.

Some tentative topics which can be adapted to the interest of the students.

1. Instrumentation of a room with diverse sensors to create a digital twin: This will involve installing microphones (for audio signals), RGB and hyperspectral camera, sterio-vision camera, thermal camera and drones equipped with air quality sensors.
2. Visualization (AR and VR) tools for digital twin applications: This will involve developing visualization techniques and interface for interactions with the room via the DT.
3. Enabling technologies of Digital Twin: Development of novel algorithms for real-time and accurate modeling of the relevant physics like Hybrid Analysis and Modeling, Compressed Sensing, Multivaratie Analysis Tools, Machine Learning
4. Data assimilation, predictive modeling and uncertainty quantification for digital twins

Supervisor: Adil Rasheed, NTNU
Co-supervisors: Damiano Varagnolo, Annette Stahl NTNU Learning outcome: Digital Twin technologies, Machine Learning

Reference:

1. Rasheed, Adil; San, Omer; Kvamsdal, Trond. (2020) Digital Twin: Values, Challenges and Enablers from a modelling perspective. IEEE Access. vol. 8.

---

Project 9: 3D Machine Learning


Project Description: The project will focus on combining the two distinct fields of mesha generation and machine learning. The project will be conduected in collaboration with the Mathematics and Cybernetics Department of SINTEF Digital.

The pre-project will involve the following tasks:

1. Extensive literature survey on the topic of 3D Machine Learning
2. Develop a good understanding of point cloud generation and mesh generation along with data strucutre representing thsoe meshes
3. Collection and pre-processig of CAD models, point clouds and meshes from multiple sources
4. Apply Machine Learning methods for the manipulation of the meshes and point clouds

The pre-project will lead to a Masters project which can have the following applications

1. Generative designs
2. Deafeaturing
3. Geometric shape optimization

Supervisor: Adil Rasheed, NTNU
Co-supervisor: Kjetil Andre Johannessen and George Muntingh, SINTEF Digital
Learning outcome: 3D Machine Learning, CAD modeling, Mesh Generation

Reference


1. https://towardsdatascience.com/3d-deep-learning-made-easier-a-brief-introduction-to-facebooks-pytorch3d-framework-9fe3075f388a
2. https://github.com/timzhang642/3D-Machine-Learning#pose_estimation

---

Project 10: Dynamic Mode Decomposition informed deep neural network for object detection and classification (Affiliated to the Explainable AI project EXAIGON)

Project description: Dynamic Mode Decomposition method originated in the fluid dynamics community tas a method to decompose complex flow fiels into a simple representation based on spatiotemporal coherent structures. More recently Grosek and Kutz, provides a novel application of the DMD atechnique and its dynamicsl decomsposition for state-of-teh art video processing. DMD modes with Fourier frequencies near the origin (zero frequencies) are interpreted as bacl-ground portions of the given video frames, and the modes with Foureir frequencies bounded away from the origin constitute their sparse counterparts. An approximate low-rank / sparse separation is achieved at the computational cost of just one SVD and one linear equatin solve.The DMD method can operate in real time with personal lamtop-class computing power and without any parameter tuning. This project will involve combining the strenghts of DMD to identify zones in a video stream where objects are moving with the power of convolutional neural network to only classify the image.

The project will involve the following tasks:

1. Extensive literature survey of multivariate methods with stress on DMD
2. Reading of the work on Deep Neural Network based object detectiona and classification methods
3. Combining DMD and DNN
4. Writing down a report and / or article based on this work

Supervisor: Adil Rasheed, NTNU
Co-supervisor: Anastasios Lekkes, NTNU


Reference


1. Kutz N, Brunton SL, Brunton BW, Proctor JL, Dynamic Mode Decomposition, Data Driven Modeling of Complex Systems, SIAM

---

Project 11:Data-driven physics discovery
Proposals from Harald Martens, Idletechs AS, sorted by priority. Idletechs may ask for a standard report confidentiality period.

1. From thermal video to thermal models. Equations of thermal dynamics from thermal video (Torbjørn Pedersen/Harald Martens, Idletechs AS). Main supervision: Adil Rasheed ITK). Thermal image segmentation, local OTFP modelling of individual image segments and ODE/PDE development from local OTFP scores.
2. From 2D surface temperature video to model of 3D internal temperature dynamics. 3D metamodelling of thermal spatiotemporal dynamics: Simulations of Alcoa Anode temperature propagation, metamodelling, forecast of 3D temp. distribution from 2D thermal video. (Håkon Jarle Hassel/Harald Martens, Idletechs AS). Main supervision: Adil Rasheed ITK)
3. Making better use of industrial vibration sensors. Multivariate vibration spectrogram modelling: 1D and/or 2D IDLE modelling of spectrograms from industrial vibration time series, based on physics-based and data-based hybrid modelling (log frequency cepstrum analysis, 1D and/or 2D motion estimation in continuous spectrograms. (Torbjørn Pedersen/Håkon Jarle Hassel/Harald Harald Martens, Idletechs AS/ Frank Westad, CAMO/NTNU/ Torbjørn Svensen NTNU). Main supervision: Adil Rasheed ITK)
4. Faster and better estimation of non-rigid motions in industrial video monitoring. Multi-frame optical flow and intensity chang estimation in thermal video, RGB video and HSI video based on IDLE modelling (Harald Martens Idletechs AS/ Annette Stahl, ITK. Main supervision: Adil Rasheed ITK)

---

Project 12:COLREG-compliant path following and collision avoidance with moving obstacles in the Trondheim fjord (Affiliated to the Explainable AI project AUTOSIT)
Project description: This project concerns the problem of anticipating the intentions of another ship on time intervals in the order of minutes. Over such time intervals, one can typically expect the ship to perform certain manoeuvres (e.g., go straight, approach its destination port, make a complete evasive manoeuvre), but all such predictions will be highly uncertain. The goal of this project is to develop methods that are able to combine relevant sources of information to make predictions over such time intervals. The project will build upon the work by Meyer et al and will be realized through the following potential tasks.

1. Data collection and preprocessing: This will involve collecting data from different sources as well as generating data (CAD models, weather data) if required and then pre-processing them for further analysis.
2. Extending our reinforcement learning framework incorporating multiagent simulation capabilities
3. More extensive incorporation of AIS data in the reinforcement learning framework
4. Extension of the collision avoidance regulations in the current framework
5. Devising better RL training strategies
6. Run an ensemble of simulations using the trained agents to quantify uncertainty associated with long-term vessel predictions

Supervisor: Adil Rasheed (adil.rasheed@ntnu.no)
Co-supervisor: Edmund Brekke (Edmund.brekke@ntnu.no), Morten Breivik (morten.breivik@ntnu.no)


References

1. Meyer E., Robinson, H., Rasheed, A., and San, O., Taming an autonomous surface vehicle for path following and collision avoidance using deep reinforcement learning, IEEE Access, 8, 41466-41481, 2020
2. Meyer E, Heibergh A, Rasheed A and San O, COREG-compliant path following and collision avoidance with moving obstacles in the Trondheim fjord using Deep Reinforcement Learning, In preparation
3. Proportional integral derivative controller assisted reinforcement learning for path following by autonomous underwater vehicles

---

Project 13:Creating a bussiness model in the cloud using Digital Twin
Contact me to discuss the idea and how to proceed in this intersting direction

Supervisor: Adil Rasheed (adil.rasheed@ntnu.no)
Co-supervisor: Karl Johan Haarberg (karl.johan.haarberg@bipartner.no) and Gunnar Haugen (gunnar.haugen@bipartner.no)



---

Project 14: Digital Twins in precision production of healthcare services: Using case clinical pathways for colon and rectal cancer

Background: This project is a continuation of a project and consequent master thesis performed in 2019 / 2020 comprising a collaboration between Prediktor and the Østfold Hospital Trust (ØHT). The project will consider the following setup: as a patient moves through her/his medical pathway (i.e., departments, visits, hospitals, etc.) he/she will produce a series of data relative to her/his medical/clinical, logistics and economical aspects. Importantly, this data can collected from different sensors/instruments into centralized databases. This information can then be used to both model important medical aspects of that single patient, but also have a better overview of the medical healthcare system as a whole.
Aim and objectives: If enough data is available, it is meaningful to use Machine Learning concepts for creating understanding that helps both curing that patient, improving the treatments for others, and optimizing the healthcare services in a holistic way. The aim of the project is thus to focus on porting modelling and data analysis technologies used typically in the process industry to the case of healthcare. Importantly, the thesis starts from the successful proof of concepts projects that have been developed by Østfold Hospital Trust together with Prediktor and Idletechs. The objectives are thus to develop healthcare digital twin models startin from datasets from real biomedical applications.
Supervisor: Øivind Riis (oivind.riis@ntnu.no)
Co-supervisor: Adil Rasheed (adil.rasheed@ntnu.no) and Damiano Varagonolo (damiano.varagnolo@ntnu.no)