AI Engineer Roadmap¶
- Roadmap: https://roadmap.sh/ai-engineer
1. Introduction¶
1.1 What is an AI Engineer?¶
- 1.1.1 AI Engineer vs ML Engineer
1.2 Common Terminology¶
- 1.2.1 AI vs AGI
- 1.2.2 LLMs
- 1.2.3 Inference
- 1.2.4 Training
- 1.2.5 Embeddings
- 1.2.6 Vector Databases
- 1.2.7 AI Agents
- 1.2.8 RAG
-
1.2.9 Prompt Engineering
-
1.3 Impact on Product Development
- 1.4 Roles and Responsibilities
2. Using Pre-trained Models¶
2.1 Pre-trained Models¶
- 2.1.1 Benefits of Pre-trained Models
- 2.1.2 Limitations and Considerations
2.2 Popular AI Models¶
- 2.2.1 OpenAI Models
- 2.2.2 Open AI Models
- 2.2.3 Capabilities / Context Length
- 2.2.4 Cut-off Dates / Knowledge
- 2.2.5 Anthropic's Claude
- 2.2.6 Google's Gemini
- 2.2.7 Azure AI
- 2.2.8 AWS Sagemaker
- 2.2.9 Hugging Face Models
- 2.2.10 Mistral AI
- 2.2.11 Cohere
- 2.2.12 Replicate
3. Open AI Platform¶
3.1 OpenAI API¶
- 3.1.1 Chat Completions API
- 3.1.2 Writing Prompts
- 3.1.3 Maximum Tokens
- 3.1.4 Token Counting
- 3.1.5 Pricing Considerations
- 3.1.6 Managing Tokens
- 3.1.7 Open AI Playground
- 3.1.8 Fine-tuning
4. AI Safety and Ethics¶
- 4.1 Understanding AI Safety Issues
- 4.2 Prompt Injection Attacks
- 4.3 Bias and Fairness
- 4.4 Security and Privacy Concerns
- 4.5 OpenAI Moderation API
- 4.6 Adding end-user IDs in prompts
- 4.7 Conducting adversarial testing
- 4.8 Robust prompt engineering
- 4.9 Know your Customers / Usecases
- 4.10 Constraining outputs and inputs
- 4.11 Safety Best Practices
5. OpenSource AI¶
- 5.1 Open vs Closed Source Models
- 5.2 Popular Open Source Models
5.3 Hugging Face¶
- 5.3.1 Finding Open Source Models
- 5.3.2 Hugging Face Tasks
- 5.3.3 Hugging Face Hub
- 5.3.4 Using Open Source Models
- 5.3.5 Inference SDK
- 5.3.6 Transformers.js
5.4 Ollama¶
- 5.4.1 Ollama Models
- 5.4.2 Ollama SDK
6. Embeddings & Vector Databases¶
6.1 What are Embeddings¶
- 6.1.1 Semantic Search
- 6.1.2 Data Classification
- 6.1.3 Recommendation Systems
- 6.1.4 Anomaly Detection
- 6.1.5 Use Cases for Embeddings
6.2 Open AI Embedding Models¶
- 6.2.1 Pricing Considerations
6.3 Open AI Embeddings API¶
- 6.3.1 Sentence Transformers
- 6.3.2 Models on Hugging Face
6.4 Open-Source Embeddings¶
6.5 Vector Databases¶
- 6.5.1 Purpose and Functionality
- 6.5.2 Popular Vector DBs (pick one)
- 6.5.2.1 Chroma
- 6.5.2.2 Pinecone
- 6.5.2.3 Weaviate
- 6.5.2.4 FAISS
- 6.5.2.5 LanceDB
- 6.5.2.6 Qdrant
- 6.5.2.7 Supabase
- 6.5.2.8 MongoDB Atlas
- 6.5.3 Implementing Vector Search
- 6.5.3.1 Indexing Embeddings
- 6.5.3.2 Performing Similarity Search
7. RAG & Implementation¶
7.1 RAG Usecases¶
- 7.1.1 RAG vs Fine-tuning
7.2 Implementing RAG¶
- 7.2.1 Chunking
- 7.2.2 Embedding
- 7.2.3 Vector Database
- 7.2.4 Retrieval Process
- 7.2.5 Generation
7.3 Ways of Implementing RAG¶
- 7.3.1 Using SDKs Directly
- 7.3.2 Langchain
- 7.3.3 Llama Index
7.4 RAG Alternative¶
- 7.4.1 Open AI Assistant API
8. Agents Usecases¶
8.1 Prompt Engineering¶
- 8.1.1 ReAct Prompting
8.2 AI Agents¶
- 8.2.1 Building AI Agents
- 8.2.1.1 Manual Implementation
- 8.2.1.2 OpenAI Functions / Tools
- 8.2.1.3 OpenAI Assistant API
9. Multimodal AI¶
9.1 Multimodal AI Usecases¶
- 9.1.1 Image Understanding
- 9.1.2 Image Generation
- 9.1.3 Video Understanding
- 9.1.4 Audio Processing
- 9.1.5 Text-to-Speech
- 9.1.6 Speech-to-Text
- 9.1.7 Multimodal AI Tasks
9.2 Multimodal AI APIs¶
- 9.2.1 OpenAI Vision API
- 9.2.2 DALL-E API
- 9.2.3 Whisper API
- 9.2.4 Hugging Face Models
- 9.2.5 LangChain for Multimodal Apps
- 9.2.6 LlamaIndex for Multimodal Apps
- 9.2.7 Implementing Multimodal AI
10. AI Code Editors¶
- 10.1 Code Completion Tools