On-premise generative AI platform with GPU orchestration
Deploy a fully autonomous in-house generative AI platform to run, optimise and orchestrate open-source LLMs (Llama, Mistral…) without depending on public clouds.
Missions, technologies and objectives
Missions
- Deployment of open-source models
- Setting up an in-house prompt engine
- LoRA fine-tuning on ITEmpire data
- Secure web interface and API for controlled internal access
Relevant fields of study
AI, DevOps, cloud infrastructure, security
Technologies and skills
- Docker
- Kubernetes GPU
- Hugging Face
- Triton Inference Server
- FastAPI
Objectives
- Deploy a robust, scalable on-premise AI architecture.
- Guarantee the security, confidentiality and performance of AI processing.
- Optimise models through LoRA fine-tuning.
- Provide a unified internal platform for generative AI.