LLM & SLM 研究日报 - 2026-07-16

LLM & SLM 研究日报

算法·训练·推理 —— 大语言模型与小语言模型的前沿研究

数据源: cs.CL + cs.LG

生成时间: 2026/7/16 09:00:12

📊 今日概况

方向 论文数
🧮 算法与架构 7
🏋️ 训练方法 4
⚡ 推理优化 8
总计扫描 50

📝 论文列表

🧮 算法与架构 (7 篇)

1. Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts

  • arXiv: 2607.12696
  • 摘要: draft,expert,moe,experts,decoding,ecospec,speculative,textsc,cost,activation
  • 关键词: draft,expert,moe,experts,decoding,ecospec,speculative,textsc,cost,activation

2. Segregate, Refine, Integrate: Decomposing Multimodal Fusion for Sentiment Analysis

  • arXiv: 2607.12686
  • 摘要: textbf,multimodal,serin,modality,fusion,refine,modal,efine,gregate,segregate
  • 关键词: textbf,multimodal,serin,modality,fusion,refine,modal,efine,gregate,segregate

3. Ring-Zero: Scaling Zero RL to a Trillion Parameters for Emergent Reasoning

  • arXiv: 2607.12395
  • 摘要: reasoning,trillion,scaling,emergent,zero,behaviors,structured,ring,training,bitter
  • 关键词: reasoning,trillion,scaling,emergent,zero,behaviors,structured,ring,training,bitter

4. Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control

  • arXiv: 2607.12856
  • 摘要: rft,reasoning,storage,tes,reinforcement,rewards,verifiable,co2,planning,weight
  • 关键词: rft,reasoning,storage,tes,reinforcement,rewards,verifiable,co2,planning,weight

5. Learning Mechanistic Reasoning for Chemical Reactions with Large Language Models

  • arXiv: 2607.12771
  • 摘要: reasoning,chemical,mechanism,reactions,reaction,fukuyamabench,llms,language,a3b,mechanistic
  • 关键词: reasoning,chemical,mechanism,reactions,reaction,fukuyamabench,llms,language,a3b,mechanistic

6. Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs

  • arXiv: 2607.12650
  • 摘要: attested,tool,verified,var,agentic,formalization,reasoning,thm,llm,120
  • 关键词: attested,tool,verified,var,agentic,formalization,reasoning,thm,llm,120

7. A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs

  • arXiv: 2607.12550
  • 摘要: cache,jolt,tucker,bit,axes,lossless,layer,head,compression,residual
  • 关键词: cache,jolt,tucker,bit,axes,lossless,layer,head,compression,residual

🏋️ 训练方法 (4 篇)

1. Knowledgeless Language Models: Suppressing Parametric Recall for Evidence-Grounded Language Modeling

  • arXiv: 2607.12831
  • 摘要: kllms,grounded,language,factual,knowledgeless,evidence,recall,pretraining,knowledge,suppressing
  • 关键词: kllms,grounded,language,factual,knowledgeless,evidence,recall,pretraining,knowledge,suppressing

2. From Critic to Confidence: PPO for Language-Based Quantitative Prediction with Confidence Estimation

  • arXiv: 2607.12687
  • 摘要: ppo,critic,confidence,prediction,care,quantitative,aligned,language,estimation,actor
  • 关键词: ppo,critic,confidence,prediction,care,quantitative,aligned,language,estimation,actor

3. TerraZero: Procedural Driving Simulation for Zero-Demonstration Self-Play at Scale

  • arXiv: 2607.13028
  • 摘要: terrazero,driving,play,procedural,agents,policy,zero,self,logged,map
  • 关键词: terrazero,driving,play,procedural,agents,policy,zero,self,logged,map

4. Verifier-Based Reinforcement Fine-Tuning of Reasoning Models for Thermal Energy Storage Control

  • arXiv: 2607.12856
  • 摘要: rft,reasoning,storage,tes,reinforcement,rewards,verifiable,co2,planning,weight
  • 关键词: rft,reasoning,storage,tes,reinforcement,rewards,verifiable,co2,planning,weight

⚡ 推理优化 (8 篇)

1. PalmClaw: A Native On-Device Agent Framework for Mobile Phones

  • arXiv: 2607.13027
  • 摘要: palmclaw,device,agent,mobile,phones,execution,agents,tools,capabilities,boundaries
  • 关键词: palmclaw,device,agent,mobile,phones,execution,agents,tools,capabilities,boundaries

2. Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts

  • arXiv: 2607.12696
  • 摘要: draft,expert,moe,experts,decoding,ecospec,speculative,textsc,cost,activation
  • 关键词: draft,expert,moe,experts,decoding,ecospec,speculative,textsc,cost,activation

3. Policy-Conditioned Constrained Decoding for Column-Level Access Control in Text-to-SQL

  • arXiv: 2607.12341
  • 摘要: sql,policy,column,pcc,deterministically,decoding,query,columns,violations,text
  • 关键词: sql,policy,column,pcc,deterministically,decoding,query,columns,violations,text

4. Evaluating Health Misinformation in Low-Resource Languages: Integrating Small Language Models with a Culturally-Sensitive Responsible NLP Framework (Bangla as a Case Study)

  • arXiv: 2607.12336
  • 摘要: misinformation,cald,health,slms,culturally,bangla,language,resource,nlp,combatant
  • 关键词: misinformation,cald,health,slms,culturally,bangla,language,resource,nlp,combatant

5. Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniques

  • arXiv: 2607.12829
  • 摘要: inference,dllms,parallel,diffusion,techniques,architectural,generation,realizing,latency,algorithmic
  • 关键词: inference,dllms,parallel,diffusion,techniques,architectural,generation,realizing,latency,algorithmic

6. Evidence-Grounded Verified Agentic Reasoning: A Path Toward Eliminating LLM Hallucination in Empirical Inference via Tool-Attested Kernel Proofs

  • arXiv: 2607.12650
  • 摘要: attested,tool,verified,var,agentic,formalization,reasoning,thm,llm,120
  • 关键词: attested,tool,verified,var,agentic,formalization,reasoning,thm,llm,120

7. Lightweight Multi-Scale Anomaly Detection for Resource-Constrained Edge Devices

  • arXiv: 2607.12599
  • 摘要: lmsae,anomaly,lightweight,detection,latency,multi,scale,edge,consumption,subtle
  • 关键词: lmsae,anomaly,lightweight,detection,latency,multi,scale,edge,consumption,subtle

8. A JoLT for the KV Cache: Near-Lossless KV Cache Compression via Joint Tucker and JL-Residual Allocation for LLMs

  • arXiv: 2607.12550
  • 摘要: cache,jolt,tucker,bit,axes,lossless,layer,head,compression,residual
  • 关键词: cache,jolt,tucker,bit,axes,lossless,layer,head,compression,residual

今日技术热点

今日扫描到 算法与架构 7 篇、训练方法 4 篇、推理优化 8 篇。

算法与架构趋势

当前 LLM 架构正从纯 Transformer 向混合架构演进：SSM (Mamba) 和线性注意力在长序列场景展现优势，MoE 在推理成本可控的前提下持续扩展参数规模。小模型架构注重蒸馏和紧凑设计。

训练方法趋势

DPO 和直接偏好优化正在成为 RLHF 的高效替代方案。合成数据质量成为新的研究焦点。LoRA/QLoRA 已成为高效微调的事实标准。

推理优化趋势

INT4 量化 (GPTQ/AWQ) 已成熟，GGUF 格式让端侧部署成为可能。Speculative decoding 在线推理中逐步普及。KV cache 压缩是降低长上下文推理成本的关键。

关键洞察

  1. 架构多元化: Transformer 不再是唯一选择，SSM 和混合架构值得持续关注
  2. 对齐轻量化: DPO 系列方法降低了高质量对齐的门槛
  3. 推理即服务: 推理优化的研究热度反映了部署需求的爆发
  4. 小模型逆袭: 端侧 SLM 的设计思路与大模型差异显著，需要专门的技术栈
  5. 数据 > 算法: 训练数据质量对模型能力的影响被重新审视

学习建议

  1. 重点关注 Mamba/Mamba-2 和混合架构的最新论文
  2. 实践 DPO 训练流程，对比 RLHF 的效果差异
  3. 尝试 vLLM + 量化模型的端到端推理优化

注：GLM-5 API 未调用，此为备用分析

📚 附录

筛选关键词

算法: attention mechanism, mixture of experts, MoE, sparse attention, flash attention, rotary position, RoPE, grouped query, GQA, KV cache …

训练: pre-training, pretraining, post-training, fine-tuning, finetuning, supervised fine-tuning, SFT, alignment, RLHF, DPO …

推理: inference, serving, latency, throughput, speculative decoding, batching, continuous batching, PagedAttention, vLLM, quantization …

本报告由 OpenClaw 自动生成 | LLM & SLM Research Daily

tag:

    缺失模块。
    1、请确保node版本大于6.2
    2、在博客根目录（注意不是yilia根目录）执行以下命令：
    npm i hexo-generator-json-content --save

    3、在根目录_config.yml里添加配置： 

      jsonContent:
    meta: false
    pages: false
    posts:
      title: true
      date: true
      path: true
      text: false
      raw: false
      content: false
      slug: false
      updated: false
      comments: false
      link: false
      permalink: false
      excerpt: false
      categories: false
      tags: true

程序员，关于技术我所知甚少，唯有探索，方得真知。