As part of my ongoing efforts to explore and implement self-hosted AI solutions, I recently deployed DeepSeek’s latest AI model, R1, on a virtualized environment within my infrastructure. This process involved several key steps, from provisioning hardware resources to configuring the AI host and testing its performance.
Setting Up the Virtual Machine
To begin, I allocated a virtual machine (VM) on my Proxmox hypervisor, provisioning it with 16 cores from an Intel Xeon processor along with 64GB of RAM. The choice of hardware was based on balancing available computational resources while ensuring the virtual machine could handle AI inference workloads effectively.
Following this, I installed Debian as the operating system on the VM. Debian’s stability and compatibility with AI toolchains made it a suitable choice for this deployment. Once the OS setup was complete, I proceeded with installing Ollama, which serves as the AI hosting framework.
Configuring Ollama and Network Access
With Ollama installed, the next step was configuring the AI host to allow local network access. I assigned a static IP address within my LAN using the 192.168.1.240 subnet, ensuring seamless connectivity across devices within the network. This step was crucial for enabling other applications to interface with the AI model.
To facilitate interaction with the AI instance, I installed Chatbox AI, which connects to the designated IP and port. This setup allowed for a more user-friendly method of sending prompts and receiving responses from the DeepSeek R1 model.
Performance Testing and Observations
After successfully establishing connectivity, I proceeded to test prompt responses. However, I quickly encountered performance limitations due to the hardware constraints of the Intel Xeon processor. Specifically, inference times ranged between two to four hours for each response, significantly impacting usability.
It is important to note that these performance results were based on the 7B parameter model, which is already a significantly reduced variant compared to the full 671B parameter model. Given the computational demands of even the smaller model, it became evident that my current hardware was not optimal for running large-scale AI inference workloads efficiently.
Conclusion
This deployment served as a valuable experiment in self-hosting AI models. While the setup was functional, the performance limitations underscore the necessity of GPU acceleration for practical AI workloads. Future iterations will involve optimizing the environment, potentially incorporating NVIDIA GPUs or dedicated AI inference hardware to improve response times and overall efficiency.
For those considering a similar setup, it is advisable to carefully assess hardware requirements and explore alternative hosting solutions that better accommodate the computational intensity of modern AI models.