LLM & SLM 研究日报 - 2026-07-14

LLM & SLM 研究日报

算法·训练·推理 —— 大语言模型与小语言模型的前沿研究

数据源: cs.CL + cs.LG

生成时间: 2026/7/14 09:00:11

📊 今日概况

方向 论文数
🧮 算法与架构 7
🏋️ 训练方法 4
⚡ 推理优化 2
总计扫描 49

📝 论文列表

🧮 算法与架构 (7 篇)

1. Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026

  • arXiv: 2607.09623
  • 摘要: qanta,2026,multimodal,reasoning,tossup,answering,bonus,gpt,answer,specific
  • 关键词: qanta,2026,multimodal,reasoning,tossup,answering,bonus,gpt,answer,specific

2. DKCD: Domain Knowledge-Enhanced Causal Discovery from Unstructured Data

  • arXiv: 2607.09348
  • 摘要: causal,dkcd,knowledge,discovery,unstructured,domain,ch1,ch2,reasoning,factors
  • 关键词: causal,dkcd,knowledge,discovery,unstructured,domain,ch1,ch2,reasoning,factors

3. WILDTRACE: Benchmarking Natural Evidence Trails in Long-Context Reasoning

  • arXiv: 2607.09328
  • 摘要: trails,evidence,wildtrace,reasoning,source,long,document,naturally,hop,documents
  • 关键词: trails,evidence,wildtrace,reasoning,source,long,document,naturally,hop,documents

4. HALO: Hybrid Adaptive Latent Reasoning for Language Models

  • arXiv: 2607.08775
  • 摘要: refinement,fixed,halo,token,latent,adaptive,frozen,facing,gpqa,hybrid
  • 关键词: refinement,fixed,halo,token,latent,adaptive,frozen,facing,gpqa,hybrid

5. Agora: Enhancing LLM Agent Reasoning Via Auction-Based Task Allocation

  • arXiv: 2607.09600
  • 摘要: agora,auction,expert,reasoning,llm,tools,enhancing,tradeable,agents,allocation
  • 关键词: agora,auction,expert,reasoning,llm,tools,enhancing,tradeable,agents,allocation

6. TSAI-MetaFraud: A Benchmark Dataset for Financial Fraud Transaction and Behavioral Risk Detection in Metaverse Ecosystems

  • arXiv: 2607.09528
  • 摘要: fraud,tsai,metafraud,metaverse,financial,behavioral,analytics,ecosystems,benchmark,multimodal
  • 关键词: fraud,tsai,metafraud,metaverse,financial,behavioral,analytics,ecosystems,benchmark,multimodal

7. Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

  • arXiv: 2607.09287
  • 摘要: sparse,wanda,pruning,tuning,peft,style,supra,super,adapter,llama
  • 关键词: sparse,wanda,pruning,tuning,peft,style,supra,super,adapter,llama

🏋️ 训练方法 (4 篇)

1. Complexity-Guided Component-wise Initialization for Language Model Pretraining

  • arXiv: 2607.09204
  • 摘要: initialization,pretrained,wise,spectral,component,language,checkpoints,pretraining,weight,reuse
  • 关键词: initialization,pretrained,wise,spectral,component,language,checkpoints,pretraining,weight,reuse

2. An Emergent Mirage: Is Emergent Misalignment and Realignment Indeed a Robust Phenomenon?

  • arXiv: 2607.09053
  • 摘要: realignment,misalignment,emergent,misaligned,lora,mirage,behavioral,phenomenon,robust,reported
  • 关键词: realignment,misalignment,emergent,misaligned,lora,mirage,behavioral,phenomenon,robust,reported

3. CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

  • arXiv: 2607.09543
  • 摘要: cocot,eeg,pretrained,decoding,contrastive,pretraining,multiscale,transformer,tokenizing,models
  • 关键词: cocot,eeg,pretrained,decoding,contrastive,pretraining,multiscale,transformer,tokenizing,models

4. Super-Tuning: From Activation-Aware Pruning to Sparse Fine-Tuning

  • arXiv: 2607.09287
  • 摘要: sparse,wanda,pruning,tuning,peft,style,supra,super,adapter,llama
  • 关键词: sparse,wanda,pruning,tuning,peft,style,supra,super,adapter,llama

⚡ 推理优化 (2 篇)

1. GatedLinear: Adaptive Routing of Complementary Linear Bases for Time Series Forecasting

  • arXiv: 2607.09537
  • 摘要: gatedlinear,routing,forecasting,nonstationary,bases,series,recurrence,complementary,trend,drift
  • 关键词: gatedlinear,routing,forecasting,nonstationary,bases,series,recurrence,complementary,trend,drift

2. On-Device Adaptive Battery Power Prediction for Electric Vehicles

  • arXiv: 2607.09400
  • 摘要: battery,device,vehicles,adaptation,prediction,power,electric,forecasting,pretrained,offline
  • 关键词: battery,device,vehicles,adaptation,prediction,power,electric,forecasting,pretrained,offline

今日技术热点

今日扫描到 算法与架构 7 篇、训练方法 4 篇、推理优化 2 篇。

算法与架构趋势

当前 LLM 架构正从纯 Transformer 向混合架构演进：SSM (Mamba) 和线性注意力在长序列场景展现优势，MoE 在推理成本可控的前提下持续扩展参数规模。小模型架构注重蒸馏和紧凑设计。

训练方法趋势

DPO 和直接偏好优化正在成为 RLHF 的高效替代方案。合成数据质量成为新的研究焦点。LoRA/QLoRA 已成为高效微调的事实标准。

推理优化趋势

INT4 量化 (GPTQ/AWQ) 已成熟，GGUF 格式让端侧部署成为可能。Speculative decoding 在线推理中逐步普及。KV cache 压缩是降低长上下文推理成本的关键。

关键洞察

  1. 架构多元化: Transformer 不再是唯一选择，SSM 和混合架构值得持续关注
  2. 对齐轻量化: DPO 系列方法降低了高质量对齐的门槛
  3. 推理即服务: 推理优化的研究热度反映了部署需求的爆发
  4. 小模型逆袭: 端侧 SLM 的设计思路与大模型差异显著，需要专门的技术栈
  5. 数据 > 算法: 训练数据质量对模型能力的影响被重新审视

学习建议

  1. 重点关注 Mamba/Mamba-2 和混合架构的最新论文
  2. 实践 DPO 训练流程，对比 RLHF 的效果差异
  3. 尝试 vLLM + 量化模型的端到端推理优化

注：GLM-5 API 未调用，此为备用分析

📚 附录

筛选关键词

算法: attention mechanism, mixture of experts, MoE, sparse attention, flash attention, rotary position, RoPE, grouped query, GQA, KV cache …

训练: pre-training, pretraining, post-training, fine-tuning, finetuning, supervised fine-tuning, SFT, alignment, RLHF, DPO …

推理: inference, serving, latency, throughput, speculative decoding, batching, continuous batching, PagedAttention, vLLM, quantization …

本报告由 OpenClaw 自动生成 | LLM & SLM Research Daily

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

程序员，关于技术我所知甚少，唯有探索，方得真知。