CodeCrafters (YC S22) - Recreate your favorite developer tools from scratch

Congrats on the launch buddy! :) @sarupbanskota

@angrezi_ Thank you!

@eunicemarginel Great question. If I'm being honest, when we first started out, we didn't think too deeply about which experience level the product is for. As we started getting users, we started learning from them. You definitely need to have programming experience, CodeCrafters is definitely not the right fit as an "intro to programming" platform. But you don't need to be an expert. ◆ At the Eastern Switzerland University of Applied Sciences, CodeCrafters is part of the curriculum for undergrad students to learn and master Haskell. ◆ My favorite customer story ever is of a final college student in the US who had never written a line of Java and had never heard of Redis. But he knew C. During his CodeCrafters adventures, he learnt about what Redis is, picked up Java, and Built his own Redis. ◆ Like I shared in my story, Paul and I hardly knew much serious programming when we set out to reimplement Git in Ruby 10 years back. During the early career level it definitely feels intimidating. But that's also when you stand to gain the most value. You should give it a try ;)

@sarupbanskota Thanks for sharing your insights and experiences with CodeCrafters! It's interesting to learn how the product evolved based on user feedback. The customer story about the college student who learned Java and built their own Redis implementation is inspiring. I appreciate your transparency about the required programming experience. I'll definitely consider giving it a try. Thanks again!

@eunicemarginel Can't wait

@sarupbanskota @gshewakr thanks Gautam! Handling the "I'm stuck" problem is super important for us. That's where most of the learning happens, and if we don't offer help people are likely to get frustrated with our product and drop off. There are a combination of features that we offer to help people when they're stuck: ◆ The instructions for each stage are carefully refined based on user feedback so that all the common questions are answered. It's common for us to make 10s of edits to each stage after launch. ◆ The instructions for each stage are also tailored to the language you're using. So if a stage is difficult in Rust for of a specific reason, we highlight that. We offer a concise explanation in-app and link to external articles for further reading. Here's an example from a stage in our Docker challenge, where if one uses Go they might run into a cryptic error: https://rohitpaulk.com/articles/.... ◆ We have a "Hints" feature, where users can submit hints based on their experience. Other users can upvote/downvote these hints and reply to them. ◆ If you're stuck and hints don't help, you can browse through solutions from other users. These solutions come with "Automatic Explanations", which are AI-generated comments that explain the solutions in detail. This was a recent launch that uses GPT-4 under the hood, you can read more about it here: https://codecrafters.io/blog/exp.... I also tweeted about how we made this: https://twitter.com/RohitPaulK/s.... ◆ For popular languages, we also offer a "Recommended Solution". These are authored by experts in the field (Maintainers of Next.js, for example). More on recommended solutions here: https://codecrafters.io/blog/sol.... ◆ Lastly - we offer personalized 1:1 debugging help over email. It's rare for people to do this, but when they do we respond with a thoughtful answer within a few business days. Summary: Meticulously crafted instructions, hints, user solutions with AI-powered explanations, expert solutions, and if all else fails - a live human to help. There are a lot of other interesting things we want to do here. One of the ideas I'm very excited about is automated bug fixes. What if we could analyze the thousands of mistakes users have made before you, and use that to suggest a fix to your code? Automated program repair is typically considered unreliable since it's hard to verify a fix, but in our case we've got a rock solid test suite so it's easy to say whether a fix is valid or not.

@manjush_vundemodalu Thanks Manjush. When you already know a language, I feel CodeCrafters is a great way to opportunities for mastery. Really appreciate your kind words and support!

@manjush_vundemodalu @sarupbanskota may you say more about what the system of automated tests is and how it helps?

@sarupbanskota @heythrive Basically when you push code to CodeCrafters, they run your code against a test suite (this test suite is open source btw, which is so cool). At times on the CLI response you can see what could have caused your code to fail. I've tried doing similar projects but through tutorials, and in the absence of tests, it's very easy to get lost and not know what to do. Causes me to abandon the project altogether 😂

@heythrive @manjush_vundemodalu Manjush explained it much better than I ever could! :)

@ruhi_mahajan thanks! We don't offer traditional certificates, however: ◆ Users get profile pages, like this one: https://app.codecrafters.io/user.... Users often to link to this from their LinkedIn profiles, for example. ◆ Users can publish their repositories to GitHub through our app - we automatically keep the repository updated with their progress and a cool animated banner on top. More on this here: https://codecrafters.io/blog/github, https://codecrafters.io/blog/ban.... ◆ We offer custom letters on demand

Wow! This could help them get hunted by headhunters like me :) I'll keep an eye on such GitHub repos for my prospects now.

@simranjitkaur For sure. Philosophically, we're lean more towards the mastery of fundamentals (vs. keeping up-to-date with the latest tech). There's plenty of resources for learning new frameworks and tools, and often the best place to learn about them are the documentation sites or learning material provided by their creators. We generally like to keep our focus on the "evergreen" aspects of learning programming — the different ways of designing software, paradigms, etc. That being said, we take great care to support the latest language versions, dependencies, and leverage the best-in-class tech ourselves to deliver the learning experience. A recent example is our AI-assisted explanations feature, which is built using GPT-4. https://codecrafters.io/blog/exp...

@sarupbanskota I think it’s smart to focus on the “evergreen” aspect. Using GPT-4 is impressive, and a nice forward-thinking approach that enhances the learning journey. I'm genuinely amazed by the idea and CodeCrafters. Good luck to you all!

@simranjitkaur Thank you! Can't wait to see you try it out.

@vishaag good question! The process we use for breaking down challenges into stages is similar to "feature slicing": https://handbook.infinityworks.c.... As an example, in our Git challenge, the first stage is to initialize a repository since that's usually the first step in using Git + it's relatively easy (just creating a few files and folders). Subsequent stages involve more complex features like making commits, cloning repositories and so on. We try to order stages based on difficulty. This isn't always possible though, since there might be features that are simple to implement but only once another complex feature is already in place. We highlight such cases in the app so users know what level of difficulty to expect from a stage.

@cayter_goh thanks! Our web-app is pretty standard - a Ruby on Rails backend, Ember.js front-end. The interesting components are the ones that handle executing tests based on Git pushes. We need to keep these test runs super fast, so that the feedback loop is quick. Our median test result time (for early stages) is 3 seconds - that includes receiving a Git push, running tests and streaming results back to the client. We do this by maintaining Firecracker VMs (the same technology behind AWS Lambda) for executing tests on each repository - these are cheap to keep around, and they boot in under 300ms. We initially built all of this in house, and then 3 months later Fly.io released their machines product: https://fly.io/blog/fly-machines/, which did pretty much exactly what we were doing but with elastic pricing & reliability benefits for us. Moving to Fly Machines reduced our costs significantly + I no longer worry about being paged at night 😅

@vera_mur Thanks Vera!

@niranjan94! This comment touched me. You've been one of our greatest supporters. Community oriented features we've rolled out recently: ◆ You can now look at how others have approached the same stage you tried, and get our AI to explain what's going on in their code https://codecrafters.io/blog/exp... ◆ If you're part of a team, you can enjoy being part of a private leaderboard and Slack, where you can see how your team is making progress and be inspired by their submissions ◆ We're exploring the idea of letting larger teams create custom courses that fit their talent engagement and up-skilling goals, whether for internal or external talent cc @rohitpaulk

@rohitpaulk @sarupbanskota I'm thrilled to hear that my comment resonated with you. It's great to have been able to support and witness your journey so far 😄 And those are some interesting features that you have mentioned there ! Will check them out :)

@rohitpaulk @niranjan94 ❤️🙏

@puspesh this wasn't too difficult to build, took us a couple of days experimenting with prompts to get good results. I tweeted the whole journey here: https://twitter.com/RohitPaulK/s... (includes the full prompt and how it evolved). We had to write two separate prompts - one to break a diff down into relevant chunks, and another to explain each chunk. We didn't need to do few-shot prompting or fine-tuning. The results are far better than I thought they'd be - GPT-4 is able to catch a lot of subtleties in code. One example that blew my mind is where it recognized that a variable was unused, and explained what purpose it might serve in _future_ changes to the code.

@rohitpaulk @salups27 Thanks Salil!

@deeptivchopra Haha so true. 6 years ago, the alternative was to follow tutorials. https://github.com/codecrafters-... has hundreds of them. (it's the 10th most popular repo on GitHub with 200k+ stars) But tutorials have these problems: ◆ They're only available in one language, the one the author knows ◆ Due to lack of tests and automated feedback, it's easy to go astray and lose motivation ◆ They're created at one point in time, and then go stale as the language gets new versions Those are some of the problems we hope to solve. Thank you for your kind support!

@gabriel_nori Thank you so much! When we started out, we thought we were teaching people the internals of popular software. Before CodeCrafters, you were a user of Git. After CodeCrafters, you knew how Git works. But we realised we were teaching something beyond software internals. ◆ Among the experienced engineer community (including staff/principal level engineers), they seem to enjoy CodeCrafters the most as a form of language mastery. Leetcode doesn't feel real-world enough. Other real-world sites don't feel complex and ambiguous enough. And books don't feel engaging enough. ◆ Among the early career community, they like the ability to explore areas of programming that they might not usually get a chance to explore at work. Reimplementing Redis is a journey into network programming. Docker is an adventure in OS and systems programming. grep is a deep dive into statecharts. etc ◆ Among Engineering Managers, they find CodeCrafters both a meaningful way to engage their team — but also to keep themselves up-to-date and hands on, especially when their dayjob causes their programming skills to go rusty. We're also exploring the idea of tailor-made programs for larger companies to engage talent, both internal and external. I hope this helps!

@mscarim Thank you!

@irfan_s Thanks Irfan! The Redis one is probably quickest :)

@rohitpaulk @anshul_mishra1 Thank you Anshul! So glad to have you as an early supporter.

@cindywu The Recurse center community is the best. Thank you for your continued support, and please keep the feedback coming. We want to make a product every Recurse batch will enjoy.

@wecandoit Absolutely Shane 🚀