Multimodal Agent MemorySubmit a paper

FinSight (Code Agent with Variable Memory)

FinSight: Towards Real-World Financial Deep Research

Jiajie Jin, Yuyao Zhang, Yimeng Xu, Hongjin Qian, Yutao Zhu, Zhicheng Dou · 2026-07-02

Read the paperPDF

Management
T2 Dynamic Memory
Memory changes during deployment through a stable management policy.
Primary modality
Image
Also: text
Representation
Hybrid
Structured variable data and source-linked CoA records are deliberately connected to retained chart artifacts through identifiers, VLM captions and embedding retrieval, and the report generator jointly consumes the retrieved analyses and charts in one interleaved report.

How experience becomes memory

Each path is one memory component, traced from what is stored to how it is found again.

  1. Code Agent with Variable Memory unified variable space containing structured and unstructured financial data as executable Python objects

    1. Represented asSpatial, temporal & cross-modal structured memoryCross-represented explicit
    2. Stored inFlat sequential records
    3. Retrieved byPlanner / tool-assisted retrieval
    4. Source evidenceSource directly available
    CAVM unifies data, tools and callable agents in a programmable variable space. Its data partition stores structured financial tables such as pandas DataFrames and unstructured data as executable Python objects. Agents inspect variable metadata and generate code that retrieves or modifies selected variables, allowing precise data states to persist across the long-horizon research run instead of being repeatedly flattened into the prompt.Sec. 2.2 Framework of FinSight; Sec. 2.3 Code Agent with Variable Memory · PDF pp. 3-4 (proceedings pp. 5870-5871) · Figure 2; Table 1; Algorithm 1

  2. multimodal Chain-of-Analysis segments and source-linked evidence records distilled from collected financial data

    1. Represented asPerception-to-text records & summariesCross-represented explicit
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search, Planner / tool-assisted retrieval
    4. Source evidenceSource linked & recoverable
    The Data Analysis Agent converts collected evidence into persistent Chain-of-Analysis segments. During structured writing, the Report Generation Agent retrieves relevant data and CoA segments from variable memory for each report section. CoAs carry source descriptions and element identifiers; Qwen3 embeddings and cosine similarity select relevant data and CoA records, preserving routes from report claims back to stored evidence.Sec. 2.4 Two-Stage Writing with Generative Retrieval; Appendix E Retrieval · PDF pp. 4-5 and 17 (proceedings pp. 5871-5872, 5884) · Figure 3

  3. generated financial chart artifacts retained with VLM captions and identifiers for later retrieval into the report

    1. Represented asRetained perceptual source evidenceSource- and modality-specific
    2. Stored inVector-indexed collection
    3. Retrieved bySimilarity search
    4. Source evidenceSource directly available
    Chart artifacts are generated from stored data, captioned by a VLM and assigned identifiers. Embedding retrieval matches a report-time figure description to the relevant retained chart, which is imported into the interleaved report. Before retention, the chart is repeatedly rendered and visually inspected by a VLM critic so that revised chart code produces an improved visual artifact.Sec. 2.4 Two-Stage Writing with Generative Retrieval; Sec. 2.5 Iterative Vision-Enhanced Mechanism for Visualization · PDF pp. 4-6 (proceedings pp. 5871-5873) · Figures 3-4

How memory changes (T2 operations)

  • Update · Prompted LLM/VLM record update

    At each inference step the agent reasons over the current variable-space metadata, generates executable code and applies Execute(C_t, V_{t-1}) to produce an updated variable state V_t. The LLM chooses which persistent variables to create, retrieve or modify while the CAVM execution loop itself remains fixed.

  • Consolidation · Episodic / hierarchical summarization

    During the research run, collected heterogeneous data is distilled into multiple viewpoint-specific Chain-of-Analysis segments. These durable intermediate summaries are subsequently selected and synthesized section by section, so the memory is consolidated hierarchically under a fixed two-stage procedure.

Related systems