LLM & SLM 研究日报
算法·训练·推理 —— 大语言模型与小语言模型的前沿研究
生成时间: 2026/7/11 11:01:40
📊 今日概况
| 方向 | 论文数 |
|---|---|
| 🧮 算法与架构 | 5 |
| 🏋️ 训练方法 | 9 |
| ⚡ 推理优化 | 10 |
| 总计扫描 | 50 |
📝 论文列表
🧮 算法与架构 (5 篇)
1. UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks
- arXiv: 2607.08768
- 摘要: uniclawbench,agent,proactive,agents,capabilities,world,benchmark,real,environments,multimodal
- 关键词: uniclawbench,agent,proactive,agents,capabilities,world,benchmark,real,environments,multimodal
2. It Takes a MAESTRO To Prune Bad Experts
- arXiv: 2607.08601
- 摘要: maestro,moe,expert,pruning,routing,experts,prune,bad,structured,importance
- 关键词: maestro,moe,expert,pruning,routing,experts,prune,bad,structured,importance
3. SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation
- arXiv: 2607.08161
- 摘要: sql,squad,language,llms,slms,text,llm,distillation,synthetic,knowledge
- 关键词: sql,squad,language,llms,slms,text,llm,distillation,synthetic,knowledge
4. Latent Memory Palace: Reasoning for Control as Autoregressive Variational Inference
- arXiv: 2607.08724
- 摘要: palace,lmp,latent,reasoning,autoregressive,variational,control,policies,memory,inference
- 关键词: palace,lmp,latent,reasoning,autoregressive,variational,control,policies,memory,inference
5. MatBind: A Shared Embedding Space for Multimodal Materials Characterization
- arXiv: 2607.08470
- 摘要: matbind,modalities,materials,embedding,shared,diffraction,physical,pxrd,space,heterogeneous
- 关键词: matbind,modalities,materials,embedding,shared,diffraction,physical,pxrd,space,heterogeneous
🏋️ 训练方法 (9 篇)
1. UltraX: Refining Pre-Training Data at Scale with Adaptive Programmatic Editing
- arXiv: 2607.08646
- 摘要: ultrax,editing,reliability,quality,refinement,supervision,training,scale,data,programmatic
- 关键词: ultrax,editing,reliability,quality,refinement,supervision,training,scale,data,programmatic
2. When Synthetic Speech Is All You Have: Better Call GRPO
- arXiv: 2607.08409
- 摘要: grpo,speech,sft,synthetic,wer,better,bottlenecked,legally,stayed,tuning
- 关键词: grpo,speech,sft,synthetic,wer,better,bottlenecked,legally,stayed,tuning
3. Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition
- arXiv: 2607.08374
- 摘要: personality,jam,psychological,theory,agnostic,predefined,judge,llm,prototypical,latent
- 关键词: personality,jam,psychological,theory,agnostic,predefined,judge,llm,prototypical,latent
4. Best-of-$N$ TTS Evaluation is Confounded by ASR Family Alignment
- arXiv: 2607.08256
- 摘要: verifier,tts,evaluator,family,asr,citep,evaluators,wer,rank,best
- 关键词: verifier,tts,evaluator,family,asr,citep,evaluators,wer,rank,best
5. Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech
- arXiv: 2607.08208
- 摘要: speaker,asr,diarization,qwen,speech,tcpmer,adaptation,conversational,official,lora
- 关键词: speaker,asr,diarization,qwen,speech,tcpmer,adaptation,conversational,official,lora
6. SLORR: Simple and Efficient In-Training Low-Rank Regularization
- arXiv: 2607.08754
- 摘要: slorr,hoyer,rank,training,regularizers,compressibility,resnet,vit,pretraining,560m
- 关键词: slorr,hoyer,rank,training,regularizers,compressibility,resnet,vit,pretraining,560m
7. Super Weights in LLMs and the Failure of Selective Training
- arXiv: 2607.08733
- 摘要: super,weights,olmo,weight,proj,lora,succeeds,failure,parameters,isolation
- 关键词: super,weights,olmo,weight,proj,lora,succeeds,failure,parameters,isolation
8. Contravariance Theory: Strong Alignment for Minimal Solutions to Hard Tasks
- arXiv: 2607.08561
- 摘要: contravariance,neuroai,alignment,privileged,convergent,axes,hard,network,dnn,brain
- 关键词: contravariance,neuroai,alignment,privileged,convergent,axes,hard,network,dnn,brain
9. CAAD: Causality-Aware Multivariate Time Series Anomaly Detection via Multi-Scale Alignment and Structural Causal Consistency
- arXiv: 2607.08555
- 摘要: caad,anomaly,causal,causality,detection,exogenous,alignment,consistency,industrial,anomalies
- 关键词: caad,anomaly,causal,causality,detection,exogenous,alignment,consistency,industrial,anomalies
⚡ 推理优化 (10 篇)
1. DominoTree: Conditional Tree-Structured Drafting with Domino for Speculative Decoding
- arXiv: 2607.08642
- 摘要: dominotree,ddtree,domino,qwen3,tree,draft,drafting,decoding,speculative,dflash
- 关键词: dominotree,ddtree,domino,qwen3,tree,draft,drafting,decoding,speculative,dflash
2. It Takes a MAESTRO To Prune Bad Experts
- arXiv: 2607.08601
- 摘要: maestro,moe,expert,pruning,routing,experts,prune,bad,structured,importance
- 关键词: maestro,moe,expert,pruning,routing,experts,prune,bad,structured,importance
3. Cross-seed explainability using Procrustes-conditioned Joint End-to-end Top-K Sparse Autoencoders
- arXiv: 2607.08499
- 摘要: procrustes,seed,sae,end,joint,top,seeds,conditioned,cross,tweeteval
- 关键词: procrustes,seed,sae,end,joint,top,seeds,conditioned,cross,tweeteval
4. SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation
- arXiv: 2607.08161
- 摘要: sql,squad,language,llms,slms,text,llm,distillation,synthetic,knowledge
- 关键词: sql,squad,language,llms,slms,text,llm,distillation,synthetic,knowledge
5. COALA: Robust Contextualized Speech-augmented Language Modeling for ASR via Contrastive Regularizer and Biasing Score Estimation
- arXiv: 2607.08117
- 摘要: coala,biasing,asr,contextualized,slms,entities,speech,augmented,regularizer,contextual
- 关键词: coala,biasing,asr,contextualized,slms,entities,speech,augmented,regularizer,contextual
6. Latent Memory Palace: Reasoning for Control as Autoregressive Variational Inference
- arXiv: 2607.08724
- 摘要: palace,lmp,latent,reasoning,autoregressive,variational,control,policies,memory,inference
- 关键词: palace,lmp,latent,reasoning,autoregressive,variational,control,policies,memory,inference
7. A Practical Investigation of Training-free Relaxed Speculative Decoding
- arXiv: 2607.08690
- 摘要: speculative,decoding,relaxed,takeaways,llm,lossless,capability,drafters,drafter,unsuited
- 关键词: speculative,decoding,relaxed,takeaways,llm,lossless,capability,drafters,drafter,unsuited
8. Resample or Reroute? Budget-Aware Test-Time Model Selection for Large Language Models
- arXiv: 2607.08665
- 摘要: budget,verifier,reroute,resample,per,resampling,correctness,ror,rerouting,selection
- 关键词: budget,verifier,reroute,resample,per,resampling,correctness,ror,rerouting,selection
9. EdgeRefine: Privacy-Utility Balance for Graphs via Jaccard Sampling under Edge Differential Privacy
- arXiv: 2607.08659
- 摘要: privacy,edgerefine,graph,edge,jaccard,noise,cora,utility,edges,differential
- 关键词: privacy,edgerefine,graph,edge,jaccard,noise,cora,utility,edges,differential
10. Federated Deep Learning for Privacy-Preserving Cardiovascular Disease Risk Prediction
- arXiv: 2607.08595
- 摘要: cardiovascular,rotterdam,federated,disease,lifelines,privacy,risk,728,739,prediction
- 关键词: cardiovascular,rotterdam,federated,disease,lifelines,privacy,risk,728,739,prediction
今日技术热点
今日扫描到 算法与架构 5 篇、训练方法 9 篇、推理优化 10 篇。
算法与架构趋势
当前 LLM 架构正从纯 Transformer 向混合架构演进：SSM (Mamba) 和线性注意力在长序列场景展现优势，MoE 在推理成本可控的前提下持续扩展参数规模。小模型架构注重蒸馏和紧凑设计。
训练方法趋势
DPO 和直接偏好优化正在成为 RLHF 的高效替代方案。合成数据质量成为新的研究焦点。LoRA/QLoRA 已成为高效微调的事实标准。
推理优化趋势
INT4 量化 (GPTQ/AWQ) 已成熟，GGUF 格式让端侧部署成为可能。Speculative decoding 在线推理中逐步普及。KV cache 压缩是降低长上下文推理成本的关键。
关键洞察
- 架构多元化: Transformer 不再是唯一选择，SSM 和混合架构值得持续关注
- 对齐轻量化: DPO 系列方法降低了高质量对齐的门槛
- 推理即服务: 推理优化的研究热度反映了部署需求的爆发
- 小模型逆袭: 端侧 SLM 的设计思路与大模型差异显著，需要专门的技术栈
- 数据 > 算法: 训练数据质量对模型能力的影响被重新审视
学习建议
- 重点关注 Mamba/Mamba-2 和混合架构的最新论文
- 实践 DPO 训练流程，对比 RLHF 的效果差异
- 尝试 vLLM + 量化模型的端到端推理优化
注：GLM-5 API 未调用，此为备用分析
📚 附录
筛选关键词
算法: attention mechanism, mixture of experts, MoE, sparse attention, flash attention, rotary position, RoPE, grouped query, GQA, KV cache …
训练: pre-training, pretraining, post-training, fine-tuning, finetuning, supervised fine-tuning, SFT, alignment, RLHF, DPO …
推理: inference, serving, latency, throughput, speculative decoding, batching, continuous batching, PagedAttention, vLLM, quantization …
本报告由 OpenClaw 自动生成 | LLM & SLM Research Daily