Hey there! This blog site post is an intro to the job, not a claim that we've replicated R1 yet. We're developing in the open, so as soon as we have examination numbers, we'll share them. You can follow our development on Hugging Face and GitHub.


True, but it looks like there's absolutely nothing to be assessed as of right now. I presume the supreme goal is to train a brand-new reasoning model and then use the same assessment metrics as o1 and the DeepSeek-R1.


Well, there ought to be at least some peace of mind check and validation to ensure the design was trained properly.


Oh yes, if you are discussing the assessment number of deepseek's model it's coming extremely soon!


As discussed in the post there is no design called Open-R1 to check at all ... not yet anyway. This is a blog site outlining that Hugging face will take the R1 Deepseek model, exercise how it was constructed as described in the paper and from what they launched, and then duplicate that procedure.


in reality this is practically how science works ... A creates a plan, discovery or innovation and it is checked by B, C and D to see if it is reproduceable. Thats been the cornerstone of research study now for a couple of centuries.


This blog site is not stating they have actually already done so ... Its a blog describing an intent to begin training a model like R1 and calling it Open-R1.


Also DeepSeek-R1 was only released recently, and even in their paper they laid out the compute hours needed. While those are low compute hours for a SOTA model this does not indicate you can train said model in a week. I 'd personally enjoy to be able to train a transformer design in a week, however we might require to wait a while for that level of compute innovation.


So there are no benchmarks for a design that has not been developed yet right? As outlined in the blog, and again in reply to your question.


However fear not, there is a GitHub Repo already and factors (hell I might join myself), some prelim work done, and a strategy of attack. A good starting position.


n
@edbeeching
has examined the released models currently


( src: https://x.com/edwardbeeching/status/1884273209136275742)


R1 simply trained on o1 outputs, so jointly .../ s. This is what the brand-new AI czars are stating


Hi! This article is an introduction to the job, not a claim that we have actually replicated R1 yet. We will totally share the missing piece when we have them, you can anticipate the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo


That's nice and important to understand this tremendous hype that lacks technical comprehension and description. Science is about recreation, and if they claim to be open, let them fullfill the open part.


Please do publish the training cost.


We will!


Excalidraw Hi n
@bojan2501
thanks, we will indeed be striving to make sure this training recipe can work for small language models on consumer hardware because not everybody has a cluster of H100s at home:-RRB- The tool we utilized for the images was Excalidraw! https://excalidraw.com


eagerly anticipating it! WTF are your talking about?


need to be a joke


It's truly cool to see how the whole open source community comes together!


Ops ...


5.5 M is number press reporter in the deepseekv3 tech report (simply the training, not the experiment afaik), for R1 hard to estimate tbh however much less than 5.5 M imo


Historically, they have actually never ever released code or datasets of their LLM training, so I would not expect this time to be various. If they would launch it that would be fantastic of course!


Yes obviously!


So essentially you're asking to replace existing censorship with another flavour of censorship?


The code for the designs are inside the model repositories, e.g. for V3: https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/modeling_deepseek.py


Hello Team, I'm Ray Bernard, the author and developer of EQUATOR. My research team will be working on a paper concentrated on duplicating specific parts of DeepSeek R1. Our objective is to recreate the cold start and provide your group with a dataset that consists of COT and other methods to support these efforts. We like to contribute our work to assist. Please let me understand if you discover this useful. Best, Ray Bernard https://www.facebook.com/groups/1186310571520299/


Where is the assessment numbers? without it you can't call it reproduction.






8 replies


True, but it appears like there's absolutely nothing to be assessed since today. I assume the supreme objective is to train a brand-new thinking design and then utilize the very same evaluation metrics as o1 and the DeepSeek-R1.


That's quite intriguing, I was asking myself why the questions the author exposed here are not being asked by others? I think the work they have actually done is memorable however at the exact same time I wonder why they wouldn't put these missing out on pieces on if they are expected to be totally open.
Why even without reproduction and understanding of the development they could impact a lot the market in this way?






4 replies


Hi! This post is an introduction to the project, not a claim that we've reproduced R1 yet. We will completely share the missing out on piece when we have them, you can anticipate the designs and datasets to be upload in this Hugging Face org and the code to be in this GitHub repo


Interesting read, and it is good that we see more effort into this instructions: more optimization and less strength.
Also wonder what tool did the author use for producing action diagram.




2 replies


Excalidraw I'm so delighted that effort like this currently exist, I'm gon na try to contribute:-RRB- 1 reply


looking forward to it! So racist articel




2 replies


WTF are your discussing?


Awesome to have this open recreation started!


For Step # 1 check out https://github.com/open-thoughts/open-thoughts!


https://x.com/ryanmart3n/status/1884284101265612856


Let's do this thing!


1 reply


It's really cool to see how the entire open source community comes together!


Does anyone understand the real training expense of r1? I can't find it in the paper or the announcement post. Is the 6M cost reported by media just the number taken from v3's training cost?




2 replies


Ops ...


Has anyone asked the DeepSeek group to publish their training data and code, or a minimum of share them privately with an independent replication project like this? Have they rejected such a request?


A loyal replication depends upon utilizing the very same dataset and hyperparameters. Otherwise, any significant disparities with the published criteria would be hard to pin down-whether due to training information differences or the replication method itself.


1 reply


Historically, they have never ever released code or datasets of their LLM training, so I would not anticipate this time to be various. If they would release it that would be remarkable naturally!


In the meantime we have to make finest guess price quotes and see if we can arrive ourselves.


You provide great duplication process of Deepseek thinking training. I will try something comparable to it.


This is truly good information, can we fine tune with particular use case when code is released?


1 reply


Yes naturally!


Please think about removing prejudiced, polluted or unaligned training data and make an effort to remove copyrighted works from the crawl from intake. This will make the design more functional. If you recycled anthropic curation checks, this may likewise help, eliminate obviouslybiased information will likely add a lot of worth. We don't want another polluted, unaligned open source design, right? And no corporate would ever use deepseek or a model that recycles it, right?
We appreciate your work for the benefit of humankind, we hope.
Miike C from NJ


1 reply


So basically you're asking to change existing censorship with another flavour of censorship?


Can't wait! Hopefully the design will be uncensored however whatever you can do is alright! Love seeing open source structure itself up. I'm not wise enough to really help but I can contribute moral assistance lol


Hello guys, I am even just searching for code for DeepSeek-V2, in order to totally understand multi-head hidden attention. You do not appear to have code in Hugging Face even for that. Or am I missing out on something? Don't see anything in src/transformers/models. MLA is not appropriately explained in their paper, so it would be essential to have code for this.