diff --git a/Exploring-DeepSeek-R1%27s-Agentic-Capabilities-Through-Code-Actions.md b/Exploring-DeepSeek-R1%27s-Agentic-Capabilities-Through-Code-Actions.md new file mode 100644 index 0000000..f45ece6 --- /dev/null +++ b/Exploring-DeepSeek-R1%27s-Agentic-Capabilities-Through-Code-Actions.md @@ -0,0 +1,19 @@ +
I ran a quick experiment examining how DeepSeek-R1 carries out on [agentic](https://investjoin.com) jobs, in spite of not [supporting tool](https://remotejobscape.com) use natively, and I was quite amazed by [preliminary outcomes](https://www.thisihavefound.com). This experiment runs DeepSeek-R1 in a [single-agent](https://festicia.com) setup, where the design not just [prepares](https://www.telewolves.com) the actions however likewise creates the actions as [executable Python](https://condominioentrelagos.com.br) code. On a subset1 of the [GAIA recognition](https://artbouquet-kolpashevo.ru) split, DeepSeek-R1 surpasses Claude 3.5 Sonnet by 12.5% outright, [wiki.myamens.com](http://wiki.myamens.com/index.php/User:MarylynEsmond) from 53.1% to 65.6% correct, and other [designs](http://sportowewywiady.pl) by an even bigger margin:
+
The [experiment](https://artiav.com) followed design use [standards](https://teyfcenter.com) from the DeepSeek-R1 paper and the model card: Don't use [few-shot](https://www.dutyperfume.co.il) examples, avoid including a system prompt, and [classihub.in](https://classihub.in/author/ixolina6716/) set the [temperature](http://square.la.coocan.jp) to 0.5 - 0.7 (0.6 was utilized). You can find further [evaluation details](https://beon.ind.in) here.
+
Approach
+
DeepSeek-R1's strong [coding abilities](https://samakcleaning.shop) allow it to [function](https://gitlab.innive.com) as an agent without being [explicitly trained](https://www.avglobaladvisory.com) for tool usage. By allowing the model to create [actions](http://waylandsepac.com) as Python code, it can flexibly interact with through code [execution](https://bardina.ch).
+
Tools are executed as [Python code](https://cswarzone.ro) that is [consisted](https://kommer-agf.nl) of [straight](http://rpg.harrypotterhaven.net) in the prompt. This can be a [basic function](https://davie.org) [meaning](http://adrianodisanto.com) or a module of a [larger package](http://losbremos.de) - any [legitimate Python](https://milliansburger.com.br) code. The design then [generates code](http://kringelholt.dk) [actions](https://reemsbd.com) that call these tools.
+
Arise from performing these [actions feed](http://planetexotic.ru) back to the model as [follow-up](http://www.sptinkgroup.com) messages, [driving](https://kando.tv) the next [actions](http://victorialakes-katy.com) up until a last response is [reached](https://tu-opt.com). The [agent structure](https://webetron.in) is an easy [iterative coding](https://nhadatsontra.net) loop that moderates the conversation in between the design and its [environment](http://theunbrokenwindow.com).
+
Conversations
+
DeepSeek-R1 is used as [chat model](http://durfee.mycrestron.com3000) in my experiment, where the [model autonomously](http://yosoy.squarespace.com) [pulls extra](http://124.160.76.16365000) [context](http://pinkyshogroast.com) from its [environment](https://www.askmuslima.com) by using tools e.g. by [utilizing](http://z.async.co.kr) an [online search](https://tehnomind.rs) engine or bring information from web pages. This drives the [discussion](http://fredwhite.se) with the [environment](https://tehnomind.rs) that continues until a last [response](http://wordpress.skippersamraadet.dk) is [reached](https://ginemed.first-simulation.com).
+
In contrast, o1 models are [understood](https://routingtable.cloud) to carry out improperly when used as chat models i.e. they do not [attempt](https://thedatingpage.com) to pull context throughout a [discussion](https://www.sesnicsa.com). According to the [connected](https://yovidyo.com) article, o1 models perform best when they have the full context available, with clear instructions on what to do with it.
+
Initially, I likewise tried a full [context](https://baramatizatka.com) in a [single prompt](http://neuronadvisers.com) [technique](https://fliesen-kroes.de) at each step (with arise from previous [actions consisted](http://appnormals.com) of), but this caused substantially [lower scores](https://thienphaptang.org) on the [GAIA subset](https://teamasshole.com). [Switching](https://hgarcia.es) to the [conversational technique](https://cswarzone.ro) [explained](https://www.lotusprotechnologies.com) above, I was able to reach the reported 65.6% [performance](https://pcigre.com).
+
This raises an intriguing question about the claim that o1 isn't a [chat model](https://brandfxbody.com) - possibly this [observation](http://www.polster-adam.de) was more appropriate to older o1 models that did not have tool use capabilities? After all, isn't [tool usage](https://baramatizatka.com) [support](https://bardina.ch) an important system for allowing designs to pull additional [context](https://gothamdoughnuts.com) from their [environment](https://moztube.com)? This [conversational method](https://innerforce.jp) certainly [appears](https://zeitfuer.abenstein.de) [effective](https://christianswhocursesometimes.com) for DeepSeek-R1, though I still [require](https://www.mafiscotek.com) to carry out similar try outs o1 models.
+
Generalization
+
Although DeepSeek-R1 was mainly [trained](https://m.my-conf.ru) with RL on math and coding tasks, [botdb.win](https://botdb.win/wiki/User:ThanhMacaulay89) it is [amazing](https://git.sasserisop.com) that [generalization](https://sossdate.com) to [agentic jobs](https://rikaluxury.com) with tool use through [code actions](https://artstroicity.ru) works so well. This [capability](http://www.xn--9m1b66aq3oyvjvmate.com) to [generalize](http://web463.webbox180.server-home.org) to agentic tasks [advises](http://wit-lof.com) of current research by [DeepMind](https://www.econofacturas.com) that shows that [RL generalizes](https://theideasbodega.com.au) whereas SFT remembers, although [generalization](http://cyprusurology.com) to tool use wasn't [investigated](https://syair.co.id) because work.
+
Despite its ability to generalize to tool use, DeepSeek-R1 [typically produces](https://vendulaburgrova.com) long [reasoning traces](https://gogs.qqck.cn) at each step, compared to other [designs](http://krise-kommunikation.dk) in my experiments, [restricting](https://www.dentalpro-file.com) the usefulness of this model in a [single-agent setup](https://i-print.com.ua). Even [simpler jobs](https://richardsongroupsclq.com) in some cases take a long time to complete. Further RL on [agentic tool](https://umyovideo.com) use, be it via code actions or not, might be one choice to [improve effectiveness](https://baramatizatka.com).
+
Underthinking
+
I also [observed](http://kuhnigarant.ru) the underthinking phenomon with DeepSeek-R1. This is when a thinking design regularly changes between different [reasoning ideas](https://commealatele.com) without sufficiently [exploring appealing](https://lesmetiersdessi.wp.imtbs-tsp.eu) [courses](https://heskethwinecompany.com.au) to reach a [proper service](https://integramais.com.br). This was a major factor for [excessively](https://iadgroup.co.uk) long [thinking traces](https://www.petrasuzanna-camino.blog) produced by DeepSeek-R1. This can be seen in the [recorded traces](https://artstroicity.ru) that are available for [download](https://cedricdaveine.fr).
+
Future experiments
+
Another common application of thinking [designs](https://topspeedliga.eu) is to [utilize](https://www.edulchef.com.ar) them for [preparing](https://www.pdmfalegnameria.com) only, while using other designs for generating code [actions](https://www.derklostertalerhof.com). This could be a prospective new function of freeact, if this [separation](http://janidocs.com) of functions proves helpful for more complex tasks.
+
I'm likewise [curious](https://www.reuna.cl) about how thinking models that currently [support](https://motelpro.com) [tool usage](http://www.globediscover.net) (like o1, o3, ...) perform in a [single-agent](https://beaubybo.nl) setup, with and without [generating code](https://en.studio-beretta.com) actions. Recent advancements like [OpenAI's Deep](http://philippefayeton.free.fr) Research or [Hugging](https://www.alorpos.com) [Face's open-source](https://boonbac.com) Deep Research, which likewise [utilizes code](https://git.sky123th.com) actions, look interesting.
\ No newline at end of file