How To Build Your Own Deep Learning Server From Scratch

With tech giants like Microsoft, Google and Amazon providing cloud solutions, why do we need an independent machine or server to do the job? The reason is simple: cloud solutions are far too expensive in the long run and it all depends on the usage of resources.

In this article, we will be concentrating on how to set up the deep learning server, as the components have already been described in one of our former articles. Also, make sure you check out the compatibility of the components.

Here is a list of top performing and affordable hardware components for a Deep Learning Server:

Subscribe to our Newsletter

Join our editors every weekday evening as they steer you through the most significant news of the day, introduce you to fresh perspectives, and provide unexpected moments of joy
Your newsletter subscriptions are subject to AIM Privacy Policy and Terms and Conditions.
  • Motherboard: Gigabyte Z370
  • GPU: GeForce Gtx 1080 Ti 
  • Processor: Intel Core i5 8600K
  • RAM: Corsair Vengeance LPX 16GB 2x8GB DDR4
  • Power Supply Unit: Corsair 600W

Now that we have all the components we will start building our Deep Learning Machine.

Choosing The OS

If you come from a software background you will know that a Linux machine can be your best work buddy. We will stick with Ubuntu Server 16.04 LTS as it provides the best support for the softwares that follow.

  • Download your Ubuntu from here.
  • If you are choosing the Ubuntu Desktop version instead of Ubuntu server you will need to switch to console mode manually.
  • Making Bootable Disk: This is a fairly simple step. Have your USB stick by your side and follow the simple instructions provided here.
  • Install the OS with the Bootable USB: This is a straightforward step for anyone who uses a computer fairly. All you need to do is to follow some simple steps. You can find plenty of materials online to get this part done. Restart your system once the installation is complete.

Installing Software Stack for Deep Learning

Before proceeding to install the Nvidia drivers, we must make sure to remove or disable the nouveau driver which is the default driver in Ubuntu. Although we have our GeForce Gtx 1080 Ti  inside the box, our machine still uses the inbuilt card that comes with the motherboard.

To activate our graphics card we need to install the drivers.

Installing CUDA

CUDA 10 is the latest version however, we recommend that you stick with the CUDA 9 to avoid dependency issues with cuDNN and Tensorflow

Follow the links below for instructions:

  1. Before installing check your device and OS compatibility.
  2. Download the installer for the OS  and follow the instructions
  3. Perform the post-installation actions

Installing cuDNN

  • Create a free Nvidia developers account
  • Download cuDNN specified for your CUDA and Ubuntu versions.

Make sure to download these three files:

  1. Runtime Library
  2. Developer Library
  3. Code Samples and User Guide
  •    Follow steps 2.3.2 and 2.4 from here

Installing TensorFlow

Follow the instructions here to install TensorFlow in your machine. Make sure to choose the GPU enabled version that is compatible with the versions of CUDA,cuDNN and Ubuntu that we have already installed in our machine.

Installing Jupyter Notebook

Install Jupyter Notebook in your Deep Learning Machine using pip

Type in the following commands in the terminal:

  • python3 -m pip install –upgrade pip
  • python3 -m pip install jupyter

To run Jupyter Notebook:

  • jupyter notebook

Setting up Remote Access

Enabling SSH

To access your machine remotely enable SSH in your machine by installing OpenSSH.Follow the Instructions here.

Accessing Jupyter Note remotely


Run Jupyter Notebook in your Deep Learning Machine


  • jupyter notebook –no-browser –port=8889


SSH from your local machine or a different machine


  • ssh -N -L localhost:8888:localhost:8889 user@serverip


Open port 8888 by typing in localhost:8888 in a browser in your local machine to access Jupyter Notebook.

Amal Nair
A Computer Science Engineer turned Data Scientist who is passionate about AI and all related technologies. Contact:

Download our Mobile App


AI Hackathons, Coding & Learning

Host Hackathons & Recruit Great Data Talent!

AIM Research

Pioneering advanced AI market research

Request Customised Insights & Surveys for the AI Industry


Strengthen Critical AI Skills with Trusted Corporate AI Training

Our customized corporate training program on Generative AI provides a unique opportunity to empower, retain, and advance your talent.

AIM Leaders Council

World’s Biggest Community Exclusively For Senior Executives In Data Science And Analytics.

3 Ways to Join our Community

Telegram group

Discover special offers, top stories, upcoming events, and more.

Discord Server

Stay Connected with a larger ecosystem of data science and ML Professionals

Subscribe to our Daily newsletter

Get our daily awesome stories & videos in your inbox