Machine Images Getting Started Guide

Getting started with deep learning, scientific computing, media processing, and more on Amazon Web Services (AWS) has never been easier by using Bitfusion’s pre-built Amazon Machine Images (AMIs). Check out the variety of AMIs that Bitfusion provides here.

Additional AMI Resources:

Bitfusion Public Slack Channel – join us here to discuss an questions you may have with regards to Bitfusion AMIs
Detailed AMI Documentation – each AMI comes with detailed documentation on latest software inclusions, versions, and more

This guide assumes you are new to spinning up machines on AWS and will walk through the process of deploying a Bitfusion AMI.


Table of Contents


Step 1: Create an AWS Account

If you don’t have one already, make sure you create an account on AWS. (If you already have an account, skip this step and just log in.)

  1. Go to the AWS Console. Navigate to http://console.aws.amazon.com, enter your e-mail address or mobile number, and select “I am a New User.”
    Create an AWS Account
  2. Enter Login Credentials and Contact Details. Enter your name, e-mail, and password, followed by either your company or personal details.
    Login CredentialsContact Information
  3. Enter Payment Information. There are no upfront fees — Amazon will only charge based on the cloud infrastructure services that you consume.
    Payment Information
  4. Verify your Identity. Amazon will call you and ask you to enter the PIN on the screen to ensure that you are who you say you are.Identity Verifiction
    Identity Verification Continued
  5. Choose your Support Plan. You can pay extra to have an AWS support representative you can call and other service add-ons. Most folks just choose the Basic plan at first, which is free/included — you only pay for the machines as you consume them. You can always change this later if you need to.
    Support Plan
  6. Congrats! Now login by clicking “Sign In to the Console” or navigating to http://console.aws.amazon.com.
    Welcome to Amazon Web Services

Step 2: Deploy a Bitfusion AMI

  1. Create a Key Pair. In order to quickly deploy an instance on AWS, you must have a key pair. If you already have a key pair that you plan to use, you can skip ahead to the Bitfusion marketplace page. Go to the EC2 dashboard, either by clicking this link to go there directly, or by going to the AWS Console and then clicking EC2.
    Create a Key PairClick on Key Pairs in the left sidebar.
    Key Pair Side BarThen click on the blue “Create Key Pair” button.
    Create Key PairGive it a name. For this example, I will call mine bitfusion-testing. Then hit “Create”.
    Name Key PairThis will download a .pem file to your Downloads folder (or wherever your web browser has been set to store donwloaded files to).

    Note: Make sure you take good care of this file – If you lose track of this file you will not be able to connect to any instances that you create with this key.

    Download Key Pair

  2. Go to the Bitfusion Marketplace Page: https://aws.amazon.com/marketplace/seller-profile?id=3b372560-86bf-4e3d-9ec0-016892a64bed.

    Note: You can also find the Bitfusion AMIs by going to https://aws.amazon.com/marketplace and searching for “Bitfusion”.

  3. Click the AMI to Deploy. In this example, we will deploy Bitfusion Ubuntu 14 TensorFlow by clicking on the top most product in the image below. The process will be similar for all other Bitfusion AMIs.
    Deploy AMI from AWS Marketplace
  4. Choose a Data Center Region and Click Continue. On the Product Details page you can learn more about the AMI, the pricing approach, and move forward with deployment. Bitfusion provides a 5 day free trial for each of our AMIs, however, you will still be responsible for any infrastructure costs payable to AWS.

    Note: It is a best practice to pick a data center that is regionally close by to your location. For example, if you are on the US East Coast, it is probably best to use US East (N. Virginia) or US East (Ohio). If you are in the UK, it is probably best to use EU West (Ireland).

    Select Region and Continue

  5. 1-Click Launch: In order to do a 1-Click launch, you must already have a key pair that you can use. If you have an angry red “Please select a key pair” message as shown in the image below, then you don’t have an available key pair yet, and need to create one. Please Go Back to Step 2 Item 1, where we walk you through how to Create a Key Pair.
    1-Click Launch No Key PairIf you have an available Key Pair created already, the screen will look like this with a big yellow button prompting you to Accept the Software Terms & Launch, and no red warning messages – Don’t press the yellow button just yet!
    1-Click Launch with Key Pair
  6. Choose Instance Type. In this example, we will select a t2.micro, which is a pretty small, inexpensive instance.

    Note: the Bitfusion Ubuntu 14 Tensorflow AMI default is p2.xlarge as GPU instances are optimal for deep learning applications, but for testing purposes you can choose a much smaller instance, such as the t2.micro that we are doing in this example. To get the full power of deep learning frameworks like TensorFlow as well as Bitfusion Boost™ software for GPU scale-out (which is bundled with some of our AMIs) — using GPU instances gives you a significant performance boost. Keep in mind — if you want to deploy P2, G2, or other larger instance types you may need to request a limit increase.

    Choosing an Instance Type

  7. Double Check the Key Pair. Scroll down to see which key pair is being used. If it did not default to the key pair you want to use for this instance, change it to the one you’d like.
    Double-Check Key Pair
  8. Accept Software Terms & Launch with 1-Click. Scroll back to the top and hit the big yellow button to continue with deployment.
    Accept Software Terms & Launch
  9. Subscription Successful, Instance Deploying. Subscription takes a few moments to complete, then your instance deploys immediately thereafter.
    Instance Launched
  10. Check out your New Bitfusion AMI Instance! To watch as your instance comes online, navigate to the EC2 dashboard and select the “Running Instances” link.
    Check Instance Status
    Check Instance Status Continued

Step 3: SSH into your Bitfusion AMI

SSH (secure shell) connections allow you to get to the command line of your AWS instance and run programs there. These instructions will work for Mac or Linux. If you are using Windows, follow these instructions to install PuTTY.

  1. Make sure your instance is running and checks have passed. (If you aren’t already there, go to the EC2 dashboard and click “Running Instances.”) Instance State should show a green circle and say “running,” and Status Checks should show a green check-mark and “2/2 checks passed.”
    Verify Running Instance
  2. Open your favorite Terminal application.
  3. Navigate to the directory that you saved your .pem file to. For me, it downloaded to my Downloads folder. You may want to stick your .pem files somewhere else, but for this example we’ll just navigate to the Downloads folder. (New to the UNIX command line or a little rusty? There are tons of guides like this one for the basics.)
    Navigate to pem File
  4. Make sure your .pem file is not publicly viewable. If your .pem file is publicly viewable, SSH will not connect. To change the permissions to make it private, type:
    chmod 400 bitfusion-testing.pem

    Don’t forget to replace bitfusion-testing.pem with the name of your .pem file.

    CHMOD pem File

  5. Copy your public DNS. In the EC2 Dashboard, copy the public DNS address of your instance. You can find it in the Description panel below or by clicking the Connect button.
    Copy Public DNS
  6. Run SSH command. Go back to your Terminal window, and type:
    ssh -i bitfusion-testing.pem ubuntu@ec2-54-172-94-141.compute-1.amazonaws.com

    Don’t forget to replace bitfusion-testing.pem with the name of your .pem file and ec2-54-172-94-141.compute-1.amazonaws.com with Your Public DNS that you copied in the previous step.

    The first time you connect, it will ask you “Are you sure you want to continue connecting (yes/no)?” Say yes.
    Run SSH Command

  7. BOOM! Welcome to your Bitfusion AMI. Make sure to register with Bitfusion.io when it prompts you so we can stay in touch and help support you. Stay on the lookout for additional tutorials coming soon where we’ll use our AMIs to walk through and explore various deep learning and data processing frameworks and use cases.

    Additional AMI Resources:

    Bitfusion Public Slack Channel – join us here to discuss an questions you may have with regards to Bitfusion AMIs
    Detailed AMI Documentation – each AMI comes with detailed documentation on latest software inclusions, versions, and more

    Welcome to Your Bitfusion AMI


Step 4: Getting Started with Jupyter (optional)

Jupyter is included in many of our Bitfusion AMIs, and is a great, user-friendly interface for performing many data science functions.

  1. Copy your Instance ID and your Public DNS. You can get the Instance ID and Public DNS from the EC2 dashboard by going to “Running Instances,” then selecting your Bitfusion AMI instance and grabbing the info from the Description panel at the bottom of the screen.
    Copy Instance ID and Public DNS
  2. Double-check that port 8888 is open. Click Security Groups in the left-hand side bar of the EC2 dashboard.
    Double-Check Port 8888Select the Security Group related to your Bitfusion AMI instance. Then select the “Inbound” tab.
    Select Security GroupIf there isn’t a rule for port 8888, create one now. Click “Edit” then “Add Rule.”
    If Needed, Create New RuleChoose Custom TCP Rule, enter port number 8888, and either choose “My IP” to do your current IP address, or choose “Anywhere” to put 0.0.0.0/0 and open it up to all IP addresses. (Just be aware that you’ll probably want to lock this down later to be more secure.) Then hit the blue “Save” button.
    Rule Settings
  3. Navigate to the Jupyter interface. Open a tab in the web browser of your choice and navigate to the following URL: http://{EC2 Instance Public IP}:8888 Except replace {EC2 Instance Public IP} with your Public DNS that you copied earlier.
  4. Enter the Jupyter password. Your password is your Instance ID that you copied earlier.
    Enter Password
  5. Welcome to the Jupyter interface! For most of our AMIs we include some example folders that contain code snippets and explanations of how to run analysis using Jupyter. The Tensorflow AMI contains Udacity course material. You can click the udacity folder to see some example notebooks.
    Welcome to JupyterLet’s click 1_notmnist.ipynb.
    Notebook ExamplesThis is the notebook view. If you’ve used Google Docs in the web browser before, it has a very similar look and feel. Text, code, and images can be weaved together for storytelling of textual explanations, links, code, data visualizations, and more in a single interactive page.
    Notebook View
  6. Upload a Notebook. Have an existing notebook ready on your personal computer that you’d like to bring into your Bitfusion AMI? Hit the “Upload” button and perform easy file transfers.
    Upload a Notebook
  7. Create a New Notebook. You can create a new notebook by hitting the “New” dropdown button and selecting either Python 2 or Python 3. For AMIs other than Tensorflow you can start other kernels such at iTorch, Julia, R, etc.
    Create a New NotebookIn this example, we’ll create a Python 2 notebook. After that, an Untitled notebook will show up.
    Python 2 ExampleWe’ll go ahead and name it by clicking the word “Untitled” near the Jupyter logo in the top left, type “Jupyter Testing” in the prompt, and hit “OK.”

    Notebook Naming
    Then we can type in some code. For simplicity sake, we’ll just type in:

    x = 1 + 1
    print x

    Simple ExampleHit the Run button to run that green selected code block, which looks like a play/pause icon 4-7-6-icon.Running a Cell1 + 1 = 2 … it worked!
    Simple Example Result

  8. Start a web shell. Want to pull in your own apt-get repositories or perform other command line actions? There’s no need to SSH into the machine, you can just use the Jupyter web shell instead.From the file browser view, click the “New” dropdown and then select “Terminal”.

    Starting a Web Shell
    Jupyter Terminal View
  9. Shutdown notebooks or terminal windows. Notebooks and terminals are a living, running processes. If later you want to “shutdown” a notebook or terminal, from the file browser view, you can either select the notebook you want to shutdown, then hit the yellow “Shutdown” button, or you can go to the “Running” tab and hit the yellow “shutdown” button to the right of the process you would like to end.
    Shutdown Notebooks
    Running Processes
  10. Additional Jupyter Resources
    Jupyter Documentation: https://jupyter.readthedocs.io/en/latest/index.html
    Jupyter Github: https://github.com/jupyter/help