User-Interface Unit

Projects

LizardMorph

User Interface Project

The goal of this project is to develop a user-friendly tool that automates the placement of anatomical landmarks on lizards. Landmarks are annotated points on a specimen, such as joints, that enable researchers to measure and compare individuals. These comparisons support the identification of broader morphological patterns within and across ecosystems. The tool integrates a pose-estimation model to predict landmark locations and is connected to an interactive user interface that allows users to edit, visualize, and download the model’s predictions. By streamlining the data collection process, this tool helps researchers work more efficiently, allowing them to focus on analysis and discovery rather than manual annotation. To get the most recent updates on this project, visit our Wikipedia page: https://humanaugmentedanalyticsgroup.miraheze.org/wiki/Stroud_Lab_Projects/Lizard_Morph

BioCosmos

User Interface Project

This project aims to develop a multimodal, AI-powered interface for biologists to search a species database by image or text. At its core is a language-vision model (LVM) trained to capture fine-grained relationships across modalities: jointly learned image and text encoders bridge the gap between visual data and natural language. Users will be able to enter a text query or upload an image query into the web application interface, the LVM embeds the query, a vector search is performed against the database, and ultimately a gallery of database items most similar to the query is populated for the user. The MVP focuses on butterflies, with plans to expand the system to cover additional taxa in future releases.

To learn more, check out: https://humanaugmentedanalyticsgroup.miraheze.org/wiki/BioCosmos

Past Seminars

6/24/2025 Meeting Summary

The meeting, facilitated by Mercedes Quintana and Oliver Dobon, was attended by Le Yang Loh, Dylan, Arun Tipingiri, and Ebram Tawfik, opened with greetings and a brief pause to allow everyone to join. The group noted that the other members needed to sign up to give presentations. The group confirmed that upcoming presentations should be short (5–10 minutes). Then members signed up for slots to present over the rest of the term. Mercedes and Oliver presented overviews of the LizardMorph and BioCosmos projects. The group then reviewed the HAAG website, identifying the sections that needed to be filled in. They discussed the purpose of other sections as well, noting that Stack Overflow-style posts are meant to document questions and issues, while the “Resources” section could include established tools and tutorials. The meeting concluded with a reminder that the group will reconvene in two weeks.

7/8/2025 Meeting Summary

UI Update (Arun Tipingiri): Arun walked the group through the front-end work on the image-to-image retrieval project. He showed slides from Dr. Porto outlining the goal of aggregating and digitizing specimen data across museums and building an AI-powered search interface.
Build & Deployment (Le Yang Loh): Le demonstrated how the team uses npm run build to generate a minified static bundle in the dist/ directory, which Flask (and Passenger in production) can serve. He noted that the current app is only accessible within the PACE sandbox environment, requiring VPN authentication.
Access & Next Steps: The group confirmed that everyone who needs to test the UI must be added to the PACE sandbox and use VPN. Le mentioned that once COC approval comes through, they’ll provide Docker-based VM templates so anyone can spin up the service without needing a GPU (inference runs on CPU). Documentation on deployment will follow.

7/22/2025 Meeting Summary

This was the final meeting of the UI group this semester. Charles Clark, Oliver Dobon, and Mercedes Quintana briefly greet one another before Manikandan Jeyarajan delivers a detailed demonstration of an evaluation and reporting framework. He explains how the system logs configurations, generates charts and comparison tables, and how to evaluate new models, while noting that the data shown is for demonstration only. Earlier in the session, Ebram Tawfik gave a presentation on MCPs. Oliver raises a point about data splitting, which Manikandan acknowledges. Mercedes shares that a missing teammate, Dylan, will upload his recorded presentation to Slack, and asks the group to provide feedback. The meeting wraps up with expressions of gratitude, goodbyes, and a shared appreciation for the work done throughout the semester.

Resources

Zotero Link

FAQ