Transparent Machine Learning – Shining the light in a black box world (completed)

Starting Date: June 2018
Prerequisites: Computer Science Undergraduate Student, Enthusiasm and desire to be a problem solver
Will results be assigned to University:

Project Description

Autonomy, based on Artificial Intelligence (AI), is at the very centre of many existing and future innovative and enabling technologies including autonomous vehicles, urban air mobility, smart cities and Industry 4.0. All decisions made by an AI system are based on the underlying algorithm design and its training/profiling set. An AI algorithm whether based on Machine Learning (ML) or its sub-domain Deep Learning (DL) is in most cases acting as a black-box. This is the case as the internal process of an AI algorithm and how it reaches a particular conclusion is not recorded in detail, especially in the context of DL. However, how exactly the algorithm is trained and what impact the training will have is not deterministic. This is one of the primary causes of adversarial sampling that aims at AI algorithms to accept malicious actions/behaviour.

As reliance of human societies on autonomous systems is only going to increase, the ability to track and record the internal mechanics of an AI algorithm, from a social, reliability and investigative-capability point of view is paramount. Consider the case of an autonomous vehicle involved in an incident. To ascertain the liability and the root cause that led to the event – it is crucial to understand how AI algorithm acted leading up to the incident. In this project, we will design and deploy novel techniques to make AI algorithms transparent, trackable and accountable.

The respective UROP recipient will be pivotal in shaping the scope of this project and lead the research activities in collaboration with the ISG-SCC team.

Required Technical Skills

ISG-SCC value the enthusiasm and commitment significantly and considers technical skills as an acquirable skill during the internship. However, basic understand and knowledge of web design, user interfaces and database design will be desirable.

What will you gain during UROP?

During this internship, the objective of the ISG-SCC is to make sure that you develop your independent research and development skills and enhance your skills on a) presenting the project’s progress, b) keeping in line with time management, b) research methods, c) technical writing skills for project outcomes and d) problem identification and solving.

ISG-SCC Track Record

ISG-SCC has successfully run the UROP for the last two years. The success of the previous two years has produced a patent application (under review by patent office) and commercial demo (MVP) under development, and five research papers. Corresponding undergraduate students are named as first authors on the papers and co-inventor on the patent application. Research papers by undergraduate students have won one ‘best student paper award’, featured in a news article on Medium and being pivotal for a World Economic Forum’s project for anti-corruption project. The ethos of ISG-SCC is that undergraduate students have the talent and imagination to sort out unique and innovative solutions. They just need guidance from established researchers, and this is what ISG-SCC will provide during the UROP.