Subscribe
Sign in
Start new thread
All
EleutherAI GPT-Neo
p/eleutherai-gpt-neo
Open-source GPT-3
Visit Product
Aditya
GPT-J — Open-source cousin of GPT-3, everyone can use it
Featured
36
•
GPT⁠⁠-⁠J-6B, a 6 billion parameter model trained on the Pile, is now available for use with our new codebase, Mesh Transformer JAX.
Replies
Best
Aditya
Hunter
Yep! Doors have been OPENED 🤯 An open-source cousin of GPT-3 is here 😇 - Performs on par with 6.7B GPT-3 - Performs better and decodes faster than GPT-Neo - repo + colab + free web demo Got to know about it through Towards Data Science article: https://towardsdatascience.com/c... More details in @arankomatsuzaki's article: https://arankomatsuzaki.wordpres...
Mayank Mishra
this is super good for folks to get started with GPT3 @adityavsc
Aditya
Hunter
@mishra_mayank absolutely!
Ankit Sharma
awesome bro. keep going. It is much needed. Congo 👍👍👍
Aditya
Hunter
@ankitsharmaofficial all credit goes to the open-source community :)
Blake Hunsicker
This looks really promising. Does GPT-J have a token limit like GPT-3 does?
Aditya
Hunter
@blakehunsicker yes, it can be fine-tuned at a rate of ~5000 tokens/second, which should be sufficient for small-to-medium-size datasets. Fine tuning instructions are here: https://github.com/kingoflolz/me...
Maxim Ignatev
Hey hope this is still relevant I find gpt-j quite alright in the generation but it provides silly results when it does summaries. are there any experts here that could help on how i can maybe train here to provide tl;dr's
Nik Hazell
Amazing! Nice one for hunting this down @adityavsc!
Aditya
Hunter
@nik_hazell thank you! Anything for open-source 😉
Pal
Fab find @adityavsc Thanks! How safe has GPT-J been in your usage - any issues of negative sentiments, foul language or worse?
Aditya
Hunter
@pallpakk some results were definitely weird but overall, it works great! Negative sentiment, foul language, etc are context specific outputs. So if an input is negative/abusive itself, the output is bound to reinforce the same sentiment.
Pal
@adityavsc Thank you for clarifying. I'm definitely looking into this some more!
Dillon Peterson
WOAH!! Way to go guys! Thank y'all for putting this together, really amazing.
Aditya
Hunter
@dillon_peterson all credit goes to the wonderful open-source community :)
Ajeya
🔥🔥🔥🔥🔥
Girdharee Saran
Useful and easy, good luck on launch
Kelvin Zhao
Will an api of this be created?
Swebliss
Wow this is amazing. Thank you SO much. Is there any way to DM you and ask you something? :)
Patrick Hamelin
*GPT-J is just as good as GPT-3.* It is more efficient, but with more quirks. In our JPRED scores, it did better with simple TCS tasks, but lost with the more complex tasks. By removing the Jordan Algorithm: Our next proposed change to a probability model is removing the Jordan Algorithm. The Jordan Algorithm is a special procedure used for simple TCS tasks that allows for fast analysis of different sequence pairs, as well as being able to easily analyze simple n-gram (aka word) models.It is more efficient, but with more quirks. In our JPRED scores, it did better with simple TCS tasks, but lost with the more complex tasks. ...
Matt Gordon
Will try this out
Nassim Abd
Great, we were waiting for this
Aditya
Hunter
@nassc for so long! Finally the wait is over :)
Fateh BENMERZOUG, Ph.D
This just blew up the door of textual content generation, awesome!
Aditya
Hunter
@fateh_benmerzoug IKR! This is literally giving super-powers to the Makers 🚀
Raju Akon
How to Use it?
Aditya
Hunter
@raju_akon super simple. Just visit https://6b.eleuther.ai/?ref=prod... write a prompt (the topic/reference/plot) on which you want an output, and then click on 'Run the model' 😍
Angsuman Chakraborty
Can we just use the pre-trained model on 1080Ti?
Pascal Weinberger
the world needs this :)
Zoya Brant
Gamechanger. Can't wait to try it in comparison to gpt3.