New AI Model Prevents Data Breaches
Experts from Imperial’s Computational Privacy Group studied attacks on query-based systems (QBS) controlled interfaces through which analysts can query data to extract useful aggregate information. They then developed a new AI-enabled method called QuerySnout to detect attacks on QBS.
QBS gives analysts access to statistics gathered from data like location and demographics currently used in Google Maps to show live information. In the study, published as part of the 29th ACM Conference on Computer and Communications Security, the team found that powerful and accurate attacks against QBS are easily automatically detected by just pressing a button.
These systems have the potential to enable privacy-preserving anonymous data analysis at scale. In QBS, curators maintain control over data and can check and examine queries sent by analysts to ensure that the answers returned do not reveal private information. Attackers bypass such systems by designing queries to infer personal information about specific people by exploiting vulnerabilities or implementation bugs in the system.
The risks of unknown strong “zero-day” attacks where attackers capitalize on vulnerabilities in systems have stalled the development and deployment of QBS. They can simulate data breach attacks to test the robustness of the systems, detect information leakages, and identify potential vulnerabilities. Manually designing and implementing these attacks against complex QBS is a complicated and lengthy process.
The team developed an AI-enabled method called QuerySnout that works by learning which questions to ask the system to gain answers. It then learns to combine the answers automatically to detect potential privacy vulnerabilities. The model can use machine learning to create an attack consisting of a collection of queries that combines the answers to reveal a particular piece of private information. Using a technique called ‘evolutionary search’ the QuerySnout model discovers the right sets of questions to ask, all in the safety of a ‘black-box setting.’
Presently, QuerySnout only tests a small number of functionalities. The challenge moving forward will be to scale the search to a much larger number of functionalities to make sure it discovers even the most advanced attacks.
