By Bryant Wine

Every few weeks, a new consumer application powered by artificial intelligence (AI) hits the market. During this frenzy that is a gold rush of AI, many have come to question the technology because they feel it is untested and vulnerable to ailments ranging from buggy performance to adversarial surveillance.

That is one of many reasons the Association for Computational Linguistics (ACL) hosts an annual meeting, this year being the 61st to be held July 9-14 in Toronto. Here, the venue brings together experts from around the world to discuss advances, issues, and opportunities in natural language processing (NLP) and AI.

Two of these experts are Assistant Professor Srijan Kumar and Ph.D. student Gaurav Verma, both researchers at Georgia Tech’s School of Computational Science and Engineering (CSE). Kumar, Verma, and their group are slated to present two papers at ACL 2023, sharing their research about robustness of AI models that can lead to improved AI performance and security.

“Security of AI models is paramount. Reliable and responsible development of AI models are important discussion topics at the national and international levels,” Kumar said. “As Large Language Models become part of the backbone of many products and tools with which users will interact, it is important to understand when, how, and why these AI models will fail.”

Robustness refers to the degree to which an AI model’s performance changes when using new data versus training data. To ensure that a model is performing reliably to its intended purpose, it is critical to understand their robustness.

Trust is the essential value within robustness, both for researchers that work in AI and consumers that use it.

AI models that perform unpredictably or in unintended way diminish trust in the technology, an issue relevant in today’s discourse and debate over AI security. Investigating robustness can prevent, or at least highlight, performance issues arising due to unmodeled behavior and malicious attacks.

One aspect of AI robustness Kumar’s group will present at ACL 2023 delves into multimodal deep learning, a method in which AI models receive and apply data through multiple modalities like text, images, video, and audio.

The group proposes in their paper an approach to evaluate the robustness of multimodal learning methods called Cross-Modal Attribute Insertions (XMAI). XMAI obtains meaningful text augmentations using cross-modal associations in multimodal data, which they found methods using text-only information cannot do.

The revealed vulnerability, the group explains, is that augmenting input text leads to poorer performance of the multimodal model. For example, adding more descriptive wording in search text for an image, like from “girl on a chair” to “little girl on a wooden chair,” caused the correct image to be retrieved at a lower rank.

While this finding show that models fail, it is important to keep in mind the group’s research is part of the engineering process to evaluate and improve robustness in vision-and-language models. By discussing these flaws at conferences like ACL 2023, researchers can use shared information to build the next generation of more robust multimodal systems.

Similar to multimodal learning, another class of AI models that uses text as input is prompt-based few-shot learning (FSL). While FSL is a useful framework for AI to improve task performance where the availability of labeled data is limited, Kumar’s group points out in their ACL findings paper that there is limited understanding of the methods’ adversarial robustness.

“Our findings shine a light on a significant vulnerability in FSL models – a marked lack of adversarial robustness,” Verma explained. “Our work highlights a non-trivial balancing act between accuracy and adversarial robustness of few-shot learning for NLP.”

Kumar’s team ran tests on six GLUE benchmark tasks, comparing FSL models with fully fine-tuned models. Here, they found a notable, greater drop in task performance of FSL models treated with adversarial perturbations than that of fully fine-tuned models.

In the same study, Kumar’s group found and proposed few ways to improve FSL robustness.

These include using unlabeled data for prompt-based FSLs and expanding to an ensemble of models trained with different prompts. The group also demonstrated that increasing the number of few-shot examples and model size led to increased adversarial robustness of FSL methods.

Co-authors joining Kumar and Verma in their two ACL 2023 papers include School of CSE alumni Shivaen Ramshetty and Venkata Prabhakara Sarath Nookala, as well as Subhabrata Mukherjee, a principal researcher at Microsoft Research.

Kumar’s lab, the Computational Data Science Lab for the Web and Social Media (CLAWS), lies at the intersection of AI and security, where they develop solutions to improve the security of AI models and AI solutions to improve internet security.

Kumar is like many today in that he wants to realize the potential good of AI. Here, measuring and enhancing the trustworthiness and robustness of AI models are cornerstones of the CLAWS group, ensuring the security of AI and the safety of those that use it.

“My vision for future AI models is for them to not only be accurate, but also robust, reliable, trustworthy, unbiased, and fair,” said Kumar. “These two ACL research papers, along with our other recent papers, push the scientific boundaries in the development of the next-generation of AI models.”