Aditya Channa's picture
5

Aditya Channa

Aditya0619
ยท

AI & ML interests

None yet

Recent Activity

Organizations

None yet

Aditya0619's activity

reacted to chansung's post with ๐Ÿ‘ 10 days ago
view post
Post
1974
Simple summarization of Evolving Deeper LLM Thinking (Google DeepMind)

The process starts by posing a question.
1) The LLM generates initial responses.
2) These generated responses are evaluated according to specific criteria (program-based checker).
3) The LLM critiques the evaluated results.
4) The LLM refines the responses based on the evaluation, critique, and original responses.

The refined response is then fed back into step 2). If it meets the criteria, the process ends. Otherwise, the algorithm generates more responses based on the refined ones (with some being discarded, some remaining, and some responses potentially being merged).

Through this process, it demonstrated excellent performance in complex scheduling problems (travel planning, meeting scheduling, etc.). It's a viable method for finding highly effective solutions in specific scenarios.

However, there are two major drawbacks:
๐Ÿค” An excessive number of API calls are required. (While the cost might not be very high, it leads to significant latency.)
๐Ÿค” The evaluator is program-based. (This limits its use as a general method. It could potentially be modified/implemented using LLM as Judge, but that would introduce additional API costs for evaluation.)

https://arxiv.org/abs/2501.09891