Sam Altman: "I think if you have a smart person who has learned to do good research and has the right sort of mindset, it only takes about six months to make them, you know, take a smart physics researcher and make them into a productive AI researcher. So we don't have enough talent in the field yet, but it's coming soon. We have a program at OpenAI that does exactly this \[OpenAI Residency\]. And I'm astonished how well it works." (This quote refers specifically to people learning to become research engineers or research scientists.) ## Types of AI roles * Software Engineer: Build customer-facing features, optimize applications for speed and scale, use AI APIs. Prompt engineering expertise is generally helpful, but AI experience beyond using the APIs or using ChatGPT like an expert is generally not needed. * Machine Learning Engineer: Build pipelines for data management, model training, and model deployment, to improve models. And/or implement cutting-edge research papers. * Research Engineer: Build massive-scale distributed machine learning systems. Focus on massive-scale and large distributed systems. * Research Scientist: Develop new ML techniques to push the state of the art forward. ## Software Engineer focused For building products that use LLMs, or for software engineering roles. https://github.com/openai/openai-cookbook https://buildspace.so/builds/ai-avatar https://vercel.com/templates/ai https://erichartford.com/uncensored-models https://platform.openai.com/docs/guides/fine-tuning Prompting: 1. [[My template for a semi-complex GPT-4 prompt]] 3. https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api 4. https://learnprompting.org/docs/intro 5. https://platform.openai.com/docs/guides/gpt-best-practices 6. https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/ P.S. If you're a freelancer and are good at backend engineering plus prompt writing (a la the GPT best practices link), I'm [hiring](https://news.ycombinator.com/item?id=36154077). ## Machine Learning engineer focused For deploying models, or ML eng roles. https://fullstackdeeplearning.com https://pytorch.org/tutorials/ (Suggest focusing on pytorch, even for companies that use both tensorflow and pytorch that's generally they have some old models using tf and their new stuff uses pytorch) ## Research Engineering focused For research eng roles, though, still helpful if you're interested in research scientist roles. http://karpathy.github.io/2019/04/25/recipe/ https://karpathy.medium.com/yes-you-should-understand-backprop-e2f06eab496b https://course.fast.ai/ https://karpathy.ai/zero-to-hero.html These HN threads https://news.ycombinator.com/item?id=35114530 https://www.deeplearning.ai/ and https://www.coursera.org/collections/machine-learning Andrew Ng https://github.com/karpathy/nanoGPT ## Research Science focused For research scientist roles, though still helpful if you're interested in research eng roles. https://openai.com/research/spinning-up-in-deep-rl https://iconix.github.io/notes/2018/10/07/what-i-learned https://github.com/iconix/openai/blob/master/syllabus.md See also other OpenAI fellows/scholars blog posts ("We ask all Scholars to document their experiences studying deep learning to hopefully inspire others to join the field too.") eg https://openai.com/blog/openai-scholars-2021-final-projects https://80000hours.org/podcast/episodes/chris-olah-unconventional-career-path/ https://80000hours.org/podcast/episodes/richard-ngo-large-language-models/ John Schulman: https://www.youtube.com/watch?v=hhiLw5Q_UFg, http://joschu.net/blog/opinionated-guide-ml-research.html Alec Radford: https://www.youtube.com/watch?v=BnpB3GrpsfM, https://www.youtube.com/watch?v=3X3EY2Fgp3g, https://www.youtube.com/watch?v=S75EdAcXHKk, https://www.youtube.com/watch?v=VINCQghQRuM, https://www.youtube.com/watch?v=KeJINHjyzOU https://web.archive.org/web/20200813005847/http://wiki.fast.ai:80/index.php/Calculus_for_Deep_Learning and https://www.quantstart.com/articles/matrix-algebra-linear-algebra-for-deep-learning-part-2/ (via https://openai.com/blog/openai-scholars-2019) https://www.deepmind.com/learning-resources/introduction-to-reinforcement-learning-with-david-silver ## That's it. If you have [feedback on what you liked about this post or what could be improved](https://airtable.com/shr411VWRbl9og1xb), or if you'd [like to get notified about new posts via email](https://airtable.com/shr411VWRbl9og1xb), or if you'd [like to be a reviewer of posts before they're published](https://airtable.com/shr411VWRbl9og1xb), please click the relevant link and submit.