PersonOS
We study how to preserve multimodal experience for unforeseen uses, revise identity assignments with new evidence, and track facts over time.
Multimodal memory · identity alignment
Mneme: A Reliable Online Multimodal Memory
Mneme records events independently of identity assignments. It revises identity bindings as visual, audio, and contextual evidence accumulates, while preserving the original observations.
61.52% overall accuracy on M3-Bench-robot, as reported in the Mneme manuscript.
Scope: identity revision within a video session.
Our research connects identity alignment, memory across time, parametric learning, and feedback-driven strategy improvement.
Existing work
Align people with events within a video session, revising identity assignments as audiovisual and contextual evidence accumulates.
Identity assignment accuracy and preservation of original event records after correction.
Separate event records from revisable identity bindings.
Online multimodal memory with revisable identity bindings.
Next direction
Connect memories across sessions, distinguish factual changes from prior errors, and retrieve information valid for the time referenced in the question.
Identity association, temporal validity, and source attribution after successive memory updates.
Time-aware records and cross-session identity association.
Evaluates memory across extended interaction histories.
Planned research
Encode audiovisual experience in model parameters to support unseen questions, cross-event composition, and continual knowledge retention.
Unseen-question accuracy, cross-event composition, retention of prior knowledge, and the cost of storing and using memory.
Evidence-linked experience records, learnable memory representations, and shared evaluations with long-term memory research.
Generates video-specific LoRA adapters from a frozen model’s representations.
Planned research
Use task feedback and historical replay to study self-improvement (RSI) of strategies for organizing, retrieving, and using memory.
Strategy gains on held-out tasks under matched budgets, including generalization beyond replay data.
Reliable memory records, task feedback, and independent evaluation sets.
Evaluates exploration policies through replay of historical discovery trees while keeping the base agent fixed.
Foundations for our methods and evaluation.