LuxFire - LuxRender Wiki
Luxrender GPL Physically Based Renderer

LuxFire

Personal tools

From LuxRender Wiki

Revision as of 22:34, 5 February 2011 by Dougal2 (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

LuxFire

At the present time, the LuxFire system is a work-in-progress. Installation and use shouldn't be attempted unless you have almost developer-level proficiency in software setup.

LuxFire is a multi-component distributed network rendering system for the LuxRender rendering engine. It is architectured such that some components can be run on different machines across the local network.

LuxFire aims to be compatible with both Python 2 and Python 3. The choice of which Python version to use depends on the LuxFire configuration you wish to use, since not all supporting libraries are yet available for Python 3.

Note that running components under different versions of Python on the same network is not supported.

Getting LuxFire

 $ hg clone http://src.luxrender.net/luxfire

or

 $ hg clone http://bitbucket.org/luxrender/luxfire


Required Python libraries

LuxFire requires sqlalchemy and jinja2 to operate. For MySQL databases, you also need the mysqldb library.

For ubuntu based linux systems you can install the required python libraries with these commands:

Python 2

 $ sudo apt-get install python-sqlalchemy python-jinja2

and optionally:

 $ sudo apt-get install python-mysqldb

Python 3

 $ sudo apt-get install python3-sqlalchemy python3-jinja2


Components

In the following sections, for the supplied commands to work it is assumed that the current directory relative to where you issued the hg clone command is:

 ./luxfire/src

Pyro

LuxFire comes with a copy of the Pyro networking library, which enables fancy local networking features. For the LuxFire system to operate, there must be exactly one Pyro name server running on the local network, with broadcasting enabled. Broadcasting is not enabled when the name server is running on the localhost address 127.0.0.1

 $ python3 -m Pyro.naming -n 192.0.X.X

LuxRender

LuxFire comes with a stub package for LuxRender, which contains some utility functions used by the system. In order to use the Renderer component, you should put your binary pylux file inside the correct folder in this package. Note that the pylux binary must be built against the correct python version for your LuxFire installation.

  • For python 2: add LuxRender/py2/pylux.[so|pyd]
  • For python 3: add LuxRender/py3/pylux.[so|pyd]

LuxFire

The main LuxFire package contains several sub-components as described below. Some components expose modules which can be executed directly to perform functions for LuxFire or to start servers.

LuxFire.Database

LuxFire uses a central database to hold data about users and jobs on the system. The interface to the database is abstracted via the sqlalchemy python library, so any database supported by sqlalchemy can be used by LuxFire. By default, LuxFire will use a local sqlite3 database file for storage.

So far, the following configurations have been tested:

  • Python 2: sqlite3 (built-in), MySQL (requires python-mysqldb package)
  • Python 3: sqlite3 (built-in)

The LuxFire.Database.Create command will initialise a new database installation and create a default luxfire.cfg configuration file:

 $ python3 -m LuxFire.Database.Create --help
 Usage: LuxFire.Database.Create [options]
 Creates the database required for LuxFire. If a non-default database
 connection string is specified, it will be saved into the luxfire.cfg file so
 that LuxFire can connect to it automatically the next time it is run
 
 Options:
   -h, --help            show this help message and exit
   -d STR, --database=STR
                         Database connection string to use. See sqlalchemy docs
                         for supported connection types
   -v, --verbose         Show database creation progress
   -b ADDR, --bind=ADDR  Specify the local IP/hostname to bind services to.
                         This would usually be the IP address of this machine's
                         main network adaptor. It is not advisable to bind to
                         IP addresses that are accessible on the internet.

When the database is created, it will also create a default administrative user, called root, with the password admin.

TODO: describe [LuxFire] section in luxfire.cfg

LuxFire.Dispatcher

The Dispatcher is a render queue manager for the LuxFire system. It will read the following items from the luxfire.cfg file:

 [Dispatcher]
 process_interval = 5       ; Number of seconds between processing the queue
 max_items_per_worker = 10  ; Max. number of items to process per interval
 [LocalStorage]
 ; TODO
 [NetworkStorage]
 ; TODO

The Dispatcher can be started as a server with the following command:

 $ python3 -m LuxFire.Dispatcher.Server

You should (at the present time) run only ONE Dispatcher in the LuxFire system.

Any of the LuxFire component servers can be stopped with CTRL-C or a SIGINT.

To connect to an existing Dispatcher on the network from a python console, use the client command:

 $ python3 -im LuxFire.Dispatcher.Client

LuxFire.Renderer

The Renderer module is responsible for creating LuxRender rendering processes on the network. It is configurable in the luxfire.cfg file:

 [Renderer]
 threads_per_server = 4   ; Number of rendering threads to use per server

Running this component as a server requires that a pylux module is put into the LuxRender package, and can be run using this command:

 $ python3 -m LuxFire.Renderer.Server

You can run as many Renderer.Servers on the network as needed, you can even run two or more on the same machine (providing the threads_per_server setting is set correctly!).

To connect to existing Renderers on the network from a python console:

 $ python3 -im LuxFire.Renderer.Client

LuxFire.Web

The Web module provides a web-based administration and monitoring interface for the LuxFire system. It uses the following luxfire.cfg section:

 [Web]
 port = 9080     ; Listening port for Web server

Running the server requires using the same database as the running Dispatcher; in the case of using sqlite, this means running on the same machine. These components can run on different machines if using a central MySQL server.

Running as a server:

 $ python3 -m LuxFire.Web.Server [-v]

This command takes an options -v/--verbose option to print a request log to the console. With the default configuration, you can now access the interface via http://127.0.0.1:9080/ using username root and password admin.

Easy startup

Also supplied is a "standalone" script which will, by default, run one instance of each of the server components using one command:

 $ ./luxfire_standalone.py --help
 Usage: luxfire_standalone.py [options]
 All-in-one launcher script for a standalone render queue. By default, this
 script will start a Pyro name server, a LuxFire.Dispatcher.Server,
 LuxFire.Renderer.Server and a LuxFire.Web.Server. You can disable any of these
 with the options below.
 
 Options:
   -h, --help           show this help message and exit
   -n, --no-nameserver  Do not start a Pyro name server.
   -d, --no-dispatcher  Do not start a LuxFire.Dispatcher.Server
   -r, --no-renderer    Do not start a LuxFire.Renderer.Server
   -w, --no-webserver   Do not start a LuxFire.Web.Server
   -v, --verbose        Show more output while running

Note, that before running this command you should have run LuxFire.Database.Create to initialise the database and LuxFire configuration.