Using Amazon Elastic Compute Cloud (EC2) To Add Rendering Capacity

From LuxRender Wiki

This tutorial will show you how to use Amazon Web Services Elastic Compute Cloud (EC2) [1] to provide inexpensive additional rendering capacity for those times when you really need/want that image or animation done promptly.

This tutorial will be of a Windows user using a Linux Ubuntu AMI from Alestic [2]. I chose this AMI because it is not as stripped down as the stock Fedora Core AMIs and makes installing and using LuxRender quick and simple. If any of this seems confusing at this point, I highly recommend you visit the link above to go to the EC2 home page and familiarize yourself with what EC2 is and can do.

Preparation

Use the EC2 Getting Started Guide [3] to set up your Amazon AWS account. Then follow the steps to get started with the Management Console all the way up to 'How to Run an Instance' (remembering to select your region first!).

If you are using windows and will be using Putty to SSH into your instances, as I did, see the 'Appendix: Putty' section of the guide to get that configured before moving on.

Now you must make a special modification to your security group to allow Luxconsole to use port 18018 (which is the port number that Lux uses to communicate between masters and slaves). Go back to the Security Groups section, LMB your security group to select it, and in the bottom panel add a 'Custom' rule and put the following into the four boxes: 'TCP', '18018', '18018' and either '0.0.0.0/0' to leave it wide open to all traffic or '<your IP address>/32' to close it off to only traffic from your PC. See Diagram below. I chose the latter because it is more secure, but I am just paranoid. :) Click the 'Save' button.

Next you are going to want to upload the 64-bit LuxRender package for Linux to webserver space you own so that your instance(s) can use the 'wget' command to download it. No need to extract it before doing so, it will just make the transfers longer. In this example (version 0.6.1) I used 'luxrender_v061_x64.tar.bz2'. Note the URL where you uploaded this for use later.

Down To Business

To launch an instance of your AMI:

1. Log in to the AWS Management Console and click the Amazon EC2 tab.

2. Select your region in the 'Region' dropdown.

3. Click AMIs in the Navigation pane and 'All Images' in the 'Viewing' dropdown. The console displays a list of running AMIs.

4. Cut and Paste ami-733d6c36 into the text search box for a Ubuntu 9.04 64-bit desktop AMI from Alestic (64-bit is required for c1.xlarge). The search will resolve to (hopefully) one AMI.

[EDIT January 18 2012] AMI are zone dependent. http://alestic.com/ have a nice list of Ubuntu AMI per zone. [END EDIT January 18 2012]

5. Mouse over the AMI when it comes up, RMB and select 'Launch Instance' or 'Request Spot Instance' depending on which you want. I use spot instances because they are typically less than half the price of on-demand instances (you can check current spot instance pricing here [4]), however you may have to wait a while for them to start and they WILL get terminated on you from time to time with no warning, but that is fine when they are a Lux slave.

6. Select the number of instances you want and the 'c1.xlarge' entry in the 'Instance Type' dropdown. You can pick whichever instance type you want but 'c1.xlarge' is the most appropriate for Lux in my opinion.

7. If selecting a Spot Instance, enter your max bid price. Click Continue.

8. The 'Advanced Instance Options' page will be next, just hit 'Continue'.

9. On the Keypair page select the proper keypair from the dropdown. Click Continue.

10. On the Configure Firewall page of the wizard, make sure the 'default' security group is selected and click Continue.

11. The Review page of the wizard appears. Confirm the data and click Launch. If you requested a spot instance it should show up in the 'Open' state. Once it goes to 'Active' (can take a while sometimes!) then you are in business. If you requested a regular instance, it should begin launching immediately. Click 'Close' on the window that pops up.

12. Once it becomes active, click on 'Instances' in the left hand column, select your new instance and from the bottom panel copy the hostname (called 'Public DNS') of the instance to the clipboard.

Connecting To Your Instance

1. Start PuTTY.

2. Under 'Session' enter 'root@<hostname> (for example: 'root@ec2-204-236-179-219.us-west-1.compute.amazonaws.com') or root@<ip_address>. I type 'root@' and then paste the hostname after it so as to prevent typos.

3. In the left-hand panel click 'Connection', then 'SSH', and select 'Auth'. The PuTTY Configuration dialog box appears. Click Browse, and select the PuTTY private key file you generated and named 'id_rsa-gsg-keypair.ppk' if you followed the instructions in the 'Appendix: Putty' section.

4. Click 'Open' to connect to your Amazon EC2 instance.

5. You may get a Putty Security Alert about a host key not being cached in the registry. Click Yes.

6. You should be presented with a terminal screen welcoming you to a Ubuntu session and ending with a command prompt that looks something like 'root@ip-<ip address> #' See screenshot image below.

NOTE: I have had it occur that the instance starts without Ubunu loading! Instead of the above you will get a command prompt that looks something like 'root@ip- #' (no IP address) and no Ubuntu welcome message. If that occurs, you may have to reboot the instance. Click on 'Instances' in the left hand column, select your new instance, RMB then select Reboot. After a few minutes restart your Putty session and confirm a correct Ubuntu start message and prompt.

Installing And Running Luxconsole

I then enter the following commands at the command prompt one at a time by copying them to the clipboard, mousing over the Putty terminal window, and an RMB pastes the command right to the current active prompt. Press enter and repeat for the next command. [I am sure there is a much better way to do this via shell script or user data, but being a complete UNIX/Linux noob, it is beyond me.]

wget http://onesolutionsjunction.com/ec2/luxrender_v061_x64.tar.bz2
[This downloads the Lux package to your instance]

tar xvf luxrender_v061_x64.tar.bz2
[This extracts it]

cd gcc442-x86_64-sse2-091220-lux-v061-091220-36533b0843bd
[This should move you to the directory created in the extraction. If not do an 'ls' command to find out what the directory name actually is.]

./luxconsole -s
[Starts Lux]

You should now see a normal start of Luxconsole stating that you have 8 willing cores (if you selected a 'c1.xlarge' instance type) chompin' at the bit.

Connecting LuxRender

Hopefully you started LuxRender on your PC by now and can go to the 'Network' tab.

Go back to your EC2 Management Console and your hostname should still be sitting there selected. Copy it again and paste it into the 'Server:' textbox at the top of the LuxRender Network window. Click the '+' symbol and check your Putty window. It should be filling with the entries of your scene data being transferred from your PC to the EC2 instance.

Want another instance or two? Go to Step 5 in the Down to Business section above and get it going!

Enjoy!