Introduction | The course requires a survey on large-model Agent directions, covering the whole set of frameworks such as ReAct, Plan&Solve, and ToolLLM. With dozens of arXiv preprints, manual close reading and comparison would take three or four days. This article explains how to reduce that to the scale of a single retrieval.
1. Twenty minutes to closely read one paper manually; dozens means three or four days
Literature in the Agent field has a characteristic: fast progress, many preprints, similar structure but differing details. ReAct discusses the interleaving of reasoning and action, Plan&Solve plans first and then executes, and ToolLLM focuses on tool calling—each paper has to be read, remembered, and compared with the previous ones.
At 20 minutes of close reading per paper, 30 papers amount to 10 hours; then once you finish filling in the table “comparing each one’s prompt strategy and failure cases,” three or four days is very normal.
The problem is: very little of this actually requires your judgment; most of the time is spent moving and organizing material.
2. Step one: batch fetch, and let the literature populate the library
Create a new project in MuchSci Paper Sailor, set the Agent direction as the topic, and run batch retrieval:
• Use arXiv as the main source, add sources such as Semantic Scholar, and merge and deduplicate before adding to the library.
• While metadata is added to the library, automatically download the PDFs.
• For Chinese tasks, it will also provide English search terms (for example, “agent framework” paired with “LLM agent framework”), for a higher hit rate.
After this step, what you have is a literature library that is searchable and citable, not thirty scattered PDFs.
3. Step two: side-by-side comparison, turning “close reading” into “alignment”
Call literature/compare on the literature in the library to pull out a side-by-side comparison in one go. The dimensions to watch are roughly these categories:
| Comparison dimension | What to compare | Why it matters |
| Prompt strategy | How reasoning and action are interleaved | Determines the direction of the framework |
| Planning approach | Plan first, or think while doing | Affects long-task performance |
| Tool calling | How tools are selected and called | The focus of methods such as ToolLLM |
| Failure cases | How each one fails | The most valuable part of a survey |
The key to this step is turning paper-by-paper reading into structured alignment along the same dimensions. You no longer have to rely on memory to compare “where exactly ReAct and Plan&Solve differ”; instead, you directly see their values on the same dimension.
4. Step three: write the survey based only on source material
When writing the survey, MuchSci Paper Sailor uses only abstracts and notes from literature in the library as source material, and automatically adds standardized citation numbers ([number]).
This point is especially important for a survey; it means:
• It will not invent conclusions. It will not “summarize” judgments that are not written in the literature.
• It will not fabricate citations. Every [number] points to a paper that actually exists in the library.
• Side-by-side comparisons have sources. Every “outperforms / is limited by” you write can be traced back to the original text.
5. A complete walkthrough
Take an Agent framework survey as an example:
1. Create a new project and set the topic as Agent frameworks (ReAct / Plan&Solve / ToolLLM, etc.).
2. Batch-retrieve and add to the library, and fetch all PDFs; for Chinese tasks, remember to have it provide English search terms.
3. Define comparison dimensions: prompt strategy, planning approach, tool calling, failure cases, and evaluation method.
4. Use literature/compare to pull out the side-by-side comparison table.
5. Write the survey based on material in the library, with citations automatically carrying [number].
6. Cross-check citations across three categories: used / missing / unused, handling each category.
7. Run an authenticity check (DOI / arXiv resolve) to confirm zero hallucination.
8. Remove AI flavor + export, and verify once more before submitting homework or a manuscript.
6. Common pitfalls
• Directly comparing conclusions from different evaluation settings. First check on what tasks and data each paper’s conclusions are based.
• Asking AI to “summarize the differences.” A summary not based on material in the library is just invention—make sure it uses the original paper text.
• Manually organizing citation numbers. Once the numbers for thirty papers get messy, fixing them is extremely painful; leave it to automatic numbering + three-way cross-checking.
• Writing only “who is better” and not “why.” The value of a survey lies in the comparison dimensions and the basis for judgments, not in the one-sentence conclusion.
7. Conclusion
The hard part of an Agent-direction survey is not “not understanding it,” but “not being able to finish reading and not being able to compare.” MuchSci Paper Sailor connects batch library entry, side-by-side comparison, and writing with citations into one pipeline—compressing away three or four days of moving work and leaving you with the part that truly requires judgment.
Self-check before you start writing:
• Are the papers compared on the same dimensions?
• Does all source material come from the library and is it verifiable?
• Have the citation numbers been checked?
• Is there any conclusion I added by “assuming it”?